Glaucoma detection method, medium and system based on feature fusion

Abstract

The application discloses a glaucoma detection method, medium and system based on feature fusion, wherein the method comprises the following steps: acquiring a fundus image and an optical coherence tomography image, and pre-processing the fundus image to obtain a first pre-processed image and pre-processing the optical coherence tomography image to obtain a second pre-processed image; inputting the first pre-processed image and the second pre-processed image into a pre-trained anomaly detection model to perform feature fusion to obtain a fusion feature map, and determining whether a to-be-detected sample is a glaucoma sample; if yes, inputting the first pre-processed image and the second pre-processed image into a pre-trained severity analysis model to analyze the severity of the to-be-detected sample through the severity analysis model, and generating a final detection result; the respective features of the fundus image and the optical coherence tomography image can be effectively utilized to effectively detect glaucoma and improve the accuracy of the glaucoma detection result.

Description

Technical Field

[0001] This application relates to the field of glaucoma detection technology, and in particular to a glaucoma detection method, medium and system based on feature fusion. Background Technology

[0002] Glaucoma is a chronic, irreversible eye disease characterized by optic nerve damage and visual field defects. Early detection and classification are of great clinical significance for preventing blindness.

[0003] In related technologies, glaucoma detection often employs single-modal analysis based on fundus images or optical coherence tomography (OCT) images; that is, analysis is performed using either fundus images alone or OCT images alone. When analyzing fundus images alone, it is difficult to quantify changes in the optic nerve fiber layer, while analysis using OCT images alone lacks a description of the overall optic nerve morphology and pathological manifestations. This results in inaccurate final analysis results. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one objective of this invention is to propose a glaucoma detection method based on feature fusion, which can effectively utilize the respective features of fundus images and optical coherence tomography (OCT) images to effectively detect glaucoma and improve the accuracy of glaucoma detection results.

[0005] In a first aspect, embodiments of the present invention propose a glaucoma detection method based on feature fusion, comprising the following steps: acquiring a fundus image and an optical coherence tomography (OCT) image corresponding to a sample to be detected, and preprocessing the fundus image to obtain a first preprocessed image, and preprocessing the OCT image to obtain a second preprocessed image; inputting the first preprocessed image and the second preprocessed image into a pre-trained anomaly detection model, so as to perform feature fusion on the first preprocessed image and the second preprocessed image through the anomaly detection model to obtain a fused feature map, and determining whether the sample to be detected is a glaucoma sample based on the fused feature map; if so, inputting the first preprocessed image and the second preprocessed image into a pre-trained severity analysis model, so as to analyze the severity of the sample to be detected through the severity analysis model, and generating a final detection result.

[0006] According to an embodiment of the glaucoma detection method based on feature fusion of the present invention, firstly, fundus image and optical coherence tomography (OCT) image corresponding to the sample to be detected are acquired, and the fundus image is preprocessed to obtain a first preprocessed image, and the OCT image is preprocessed to obtain a second preprocessed image; then, the first preprocessed image and the second preprocessed image are input into a pre-trained anomaly detection model, so that the anomaly detection model performs feature fusion on the first preprocessed image and the second preprocessed image to obtain a fused feature map, and determines whether the sample to be detected is a glaucoma sample based on the fused feature map; then, if so, the first preprocessed image and the second preprocessed image are input into a pre-trained severity analysis model, so that the severity of the sample to be detected is analyzed by the severity analysis model to generate a final detection result; thereby effectively utilizing the respective features of the fundus image and the OCT image to effectively detect glaucoma and improve the accuracy of glaucoma detection results.

[0007] In some embodiments, the pre-trained anomaly detection model includes a linear branch and a nonlinear branch. Determining whether the sample to be detected is a glaucoma sample based on the fused feature map includes: calculating the corresponding linear branch output based on the linear branch and the fused feature map; calculating the corresponding nonlinear branch output based on the nonlinear branch and the fused feature map; and performing element-wise addition on the linear branch output and the nonlinear branch output to obtain the final anomaly detection result.

[0008] In some embodiments, calculating the corresponding linear branch output based on the fused feature map based on the linear branch includes: performing global average pooling on the fused feature map to obtain a corresponding fused feature vector, and randomly discarding channels in the fused feature vector to obtain a linear input vector; applying an activation function to the linear input vector to obtain a nonlinear pre-transformation; and performing a linear transformation on the nonlinear pre-transformation based on a first weight matrix to obtain the linear branch output.

[0009] In some embodiments, calculating the corresponding nonlinear branch output based on the fused feature map based on the nonlinear branch includes: for each input dimension of the fused feature map, calculating the basis vector corresponding to the input dimension; concatenating the basis vectors to obtain the corresponding total basis representation; and linearly combining the total basis representation based on the second weight matrix to obtain the nonlinear branch output.

[0010] In some embodiments, the pre-trained severity analysis model includes a bi-branch feature extraction layer and an adaptive lightweight cross-attention layer; the bi-branch feature extraction layer is used to extract features from the first preprocessed image and the second preprocessed image to obtain corresponding branch features; the adaptive lightweight cross-attention layer is used to perform feature fusion on the branch features and suppress unfavorable features in the fused features based on attention interaction.

[0011] In some embodiments, the adaptive lightweight cross-attention layer includes a normalization module, a location-aware cross-attention module, and a feature enhancement module;

[0012] The output of the adaptive lightweight cross-attention layer is expressed by the following formula:

[0013]

[0014] in, This represents the output of the adaptive lightweight cross-attention layer. Indicates branching characteristics, Represents the adaptive fusion coefficient. This represents the output of the position-aware cross-attention module. This represents the output of the feature enhancement module.

[0015] In some embodiments, the loss function of the pre-trained severity analysis model is expressed by the following formula:

[0016]

[0017] in, Represents the loss function. Represents the category weight coefficient. This represents the model's predicted probability of the true class. This indicates a regulatory factor.

[0018] Secondly, embodiments of the present invention provide a computer-readable storage medium storing a glaucoma detection program based on feature fusion, which, when executed by a processor, implements the glaucoma detection method based on feature fusion as described above.

[0019] Thirdly, this invention proposes a glaucoma detection system based on feature fusion, characterized by comprising: a preprocessing module, which acquires a fundus image and an optical coherence tomography (OCT) image corresponding to the sample to be detected, preprocesses the fundus image to obtain a first preprocessed image, and preprocesses the OCT image to obtain a second preprocessed image; an anomaly detection module, which inputs the first preprocessed image and the second preprocessed image into a pre-trained anomaly detection model to perform feature fusion on the first preprocessed image and the second preprocessed image to obtain a fused feature map, and determines whether the sample to be detected is a glaucoma sample based on the fused feature map; and a severity analysis module, which, when the sample to be detected is a glaucoma sample, inputs the first preprocessed image and the second preprocessed image into a pre-trained severity analysis model to analyze the severity of the sample to be detected and generate a final detection result.

[0020] In some embodiments, the pre-trained anomaly detection model includes a linear branch and a nonlinear branch. Determining whether the sample to be detected is a glaucoma sample based on the fused feature map includes: calculating the corresponding linear branch output based on the linear branch and the fused feature map; calculating the corresponding nonlinear branch output based on the nonlinear branch and the fused feature map; and performing element-wise addition on the linear branch output and the nonlinear branch output to obtain the final anomaly detection result.

[0021] The glaucoma detection method based on feature fusion according to embodiments of the present invention effectively improves the accuracy of the final detection results by fusing features between fundus images and optical coherence tomography (OCT) images before judgment, and then judging glaucoma based on the fused feature map. Furthermore, after determining that the sample to be tested is a glaucoma sample, analyzing the severity of the glaucoma sample further improves the reliability of the detection results.

[0022] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of a glaucoma detection method based on feature fusion according to an embodiment of the present invention;

[0024] Figure 2 This is a block diagram of a glaucoma detection system based on feature fusion according to an embodiment of the present invention. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0026] The following describes a glaucoma detection method based on feature fusion according to an embodiment of the present invention with reference to the accompanying drawings.

[0027] Please see Figure 1 , Figure 1 This is a flowchart illustrating a glaucoma detection method based on feature fusion according to an embodiment of the present invention, as shown below. Figure 1 As shown, this glaucoma detection method based on feature fusion includes the following steps:

[0028] S101, acquire the fundus image and optical coherence tomography image corresponding to the sample to be detected, and preprocess the fundus image to obtain a first preprocessed image, and preprocess the optical coherence tomography image to obtain a second preprocessed image.

[0029] As an example, firstly, fundus image and optical coherence tomography (OCT) image corresponding to the sample to be tested are acquired; then, the fundus image is randomly cropped by 90%-110% in size and aspect ratio, and the size is adjusted to a preset pixel value (for example, the preset pixel value can be set to 512*512); then, the cropped fundus image is randomly flipped horizontally / vertically and randomly rotated to obtain a first preprocessed image. Next, the OCT image is cropped along the height dimension to obtain multiple slices, which are the second preprocessed image.

[0030] It should be noted that for the first preprocessed image, three corresponding input channels can be set; while for multiple slices in the second preprocessed image, 256 corresponding input channels can be set.

[0031] S102, the first preprocessed image and the second preprocessed image are input into the pre-trained anomaly detection model, so that the anomaly detection model can perform feature fusion on the first preprocessed image and the second preprocessed image to obtain a fused feature map, and determine whether the sample to be detected is a glaucoma sample based on the fused feature map.

[0032] In some embodiments, the pre-trained anomaly detection model includes a linear branch and a non-linear branch. The determination of whether a sample to be detected is a glaucoma sample based on the fused feature map includes: calculating the corresponding linear branch output based on the fused feature map based on the linear branch; calculating the corresponding non-linear branch output based on the fused feature map based on the non-linear branch; and summing the linear branch output and the non-linear branch output by the principal element to obtain the final anomaly detection result.

[0033] In some embodiments, calculating the corresponding linear branch output based on the fused feature map based on the linear branch includes: performing global average pooling on the fused feature map to obtain the corresponding fused feature vector, and randomly discarding channels in the fused feature vector to obtain a linear input vector; applying an activation function to the linear input vector to obtain a nonlinear pretransform; and performing a linear transformation on the nonlinear pretransform based on the first weight matrix to obtain the linear branch output.

[0034] In some embodiments, the nonlinear branch output is calculated based on the fused feature map, including: for each input dimension of the fused feature map, calculating the basis vector corresponding to the input dimension; concatenating the basis vectors to obtain the corresponding total basis representation; and linearly combining the total basis representation based on the second weight matrix to obtain the nonlinear branch output.

[0035] As an example, in the anomaly detection phase, a pre-trained EfficientNet-B2 is used to extract features from the first and second pre-processed images.

[0036] It's worth noting that EfficientNet-B2 is a lightweight convolutional neural network based on a compound scaling strategy. It jointly expands the network's depth, width, and input resolution, achieving a high efficiency balance between model size and performance. Compared to traditional convolutional networks, EfficientNet-B2 maintains high feature extraction capabilities and generalization performance while significantly reducing the number of parameters and computational cost. The main body of the network consists of MobileInverted Bottleneck Convolution (MBConv) modules. This structure introduces depthwise separable convolutional and expansion layers, effectively reducing redundant computation and enhancing inter-channel information interaction. EfficientNet-B2 uses the Swish (SiLU) activation function in all convolutional layers. Compared to ReLU, Swish retains nonlinear responses in the less-than-zero range, mitigating gradient vanishing and improving model convergence speed. SE attention structures are embedded in parts of MBConv to recalibrate channel features using global information, enhancing the model's sensitivity to key pathological features. Through composite scaling, EfficientNet-B2 can capture local details in shallow layers and extract high-level semantic features in deep layers, providing a solid feature representation foundation for differential modeling of subsequent fundus images / OCT (Optical Coherence Tomography) branches.

[0037] Specifically, firstly, feature fusion is performed on fundus images and optical coherence tomography (OCT) images to obtain a fused feature map. ,in, Represents the real number field. Represents the number of channels (feature dimension). Indicates feature height, This indicates the feature width. Next, for... Perform global average pooling to obtain the fused feature vector. Then, Dropout processing is applied to the fused feature vector, that is, some channels in the fused feature vector are randomly discarded to reduce overfitting. Next, the processed fused feature vector is first subjected to the Swish activation function to obtain the corresponding nonlinear pre-change; then, it is projected through the first weight matrix to obtain the linear branch output. This process can be expressed by the following formula:

[0038]

[0039] in, Indicates linear branch output. This represents the first weight matrix. Represents the fused feature vector. The bias vector of the linear layer representing the linear branch.

[0040] Then, during the nonlinear branching process, each input dimension... Expanding the basis functions using B-spline over the preset interval [a, b] yields locally smooth basis representations, which are then linearly combined to approximate complex nonlinear mappings. Specifically, regarding the basis function parameters, the number of non-repeating control points can be set to... B-spline order is Then the length of the node vector is Node vectors are typically distributed uniformly or semi-uniformly; the boundaries are clamped to ensure the endpoint interpolation properties.

[0041] In the specific calculation process, firstly, for each input dimension... Calculate its corresponding basis vectors ,in, express The Middle The characteristic values ​​of each channel, Represents the family of B-spline basis functions. Each basis function in the input The response value at that location, Indicates the first basis vectors of input dimension, The transpose is then represented; next, the basis vectors are concatenated to obtain the total basis representation. Then, based on the second weight matrix (the learnable weight matrix)... A linear combination is performed to obtain the nonlinear branch output; this nonlinear branch output can be calculated using the following formula:

[0042]

[0043] in, Indicates non-linear branch output. This represents the second weight matrix. The total basis is represented. The bias vector represents the linear layer of the nonlinear branch.

[0044] After obtaining the nonlinear branch output, the two branch outputs can be added element-wise to obtain the original score of the final output:

[0045]

[0046] Probability output (anomaly detection is binary classification):

[0047]

[0048] in, This represents the original score of the final output. This indicates the probability output. This represents the Sigmoid function.

[0049] S103, if so, the first preprocessed image and the second preprocessed image are input into the pre-trained severity analysis model to analyze the severity of the sample to be detected and generate the final detection result.

[0050] In some embodiments, the pre-trained severity analysis model includes a bi-branch feature extraction layer and an adaptive lightweight cross-attention layer; the bi-branch feature extraction layer is used to extract features from the first preprocessed image and the second preprocessed image to obtain corresponding branch features; the adaptive lightweight cross-attention layer is used to fuse the branch features and suppress unfavorable features in the fused features based on attention interaction.

[0051] In some embodiments, the adaptive lightweight cross-attention layer includes a normalization module, a location-aware cross-attention module, and a feature enhancement module;

[0052] The output of the adaptive lightweight cross-attention layer is expressed by the following formula:

[0053]

[0054] in, This represents the output of the adaptive lightweight cross-attention layer. Indicates branching characteristics, Represents the adaptive fusion coefficient. This represents the output of the position-aware cross-attention module. This represents the output of the feature enhancement module.

[0055] In some embodiments, the loss function of the pre-trained severity analysis model is expressed by the following formula:

[0056]

[0057] in, Represents the loss function. Represents the category weight coefficient. This represents the model's predicted probability of the true class. This indicates a regulatory factor.

[0058] As an example, after differentiating glaucoma samples, in order to further differentiate the disease stage of the detected glaucoma samples, a severity analysis method based on dual-branch EfficientNet-B2 and adaptive lightweight cross-attention module (ALCA) is further proposed. This aims to achieve fine-grained interaction between fundus images and OCT features, enhance the ability to distinguish lesion sites, and improve the sensitivity to early subtle lesions.

[0059] Specifically, firstly, the pre-trained severity analysis model includes a bi-branch feature extraction layer and an adaptive lightweight cross-attention layer; the bi-branch feature extraction layer is used to extract features from the first and second pre-processed images corresponding to the glaucoma samples to obtain the corresponding branch features; the adaptive lightweight cross-attention layer is used to fuse the branch features and suppress unfavorable features in the fused features based on attention interaction.

[0060] The adaptive lightweight cross-attention layer includes a normalization module, a position-aware cross-attention module, and a feature enhancement module. At the output, the outputs of the position-aware cross-attention module and the feature enhancement module are combined using learnable adaptive fusion coefficients, as expressed by the formula:

[0061]

[0062] in, This represents the output of the adaptive lightweight cross-attention layer. Indicates branching characteristics, Represents the adaptive fusion coefficient. This represents the output of the position-aware cross-attention module. This represents the output of the feature enhancement module.

[0063] In the normalization module, for the input For each spatial location (h, w), the mean and variance are calculated independently along the channel dimension:

[0064]

[0065] Next, normalization is performed and a learnable scale is applied. With bias :

[0066]

[0067] in, This represents the number of channels in the feature map. This represents the input feature tensor of the normalization module. Indicates the first The characteristic values ​​of each channel, This represents the mean of all channels. This represents the variance of all channels. This indicates that numerically stable terms should avoid division by zero. This represents the final normalized output. Each channel has an independent scaling factor and offset parameter, with initial values... .

[0068] In the position-aware cross-attention module, a 1×1 convolution + a 3×3 depthwise separable convolution (DWConv) is performed on the main branch input to obtain the query:

[0069]

[0070] Perform the same processing on the auxiliary branch and obtain the Key and Value using chunk2 (splitting the channel into two parts):

[0071]

[0072] Multi-head segmentation and shape transformation: Let the number of attention heads be h, the dimension of each head be d = C / h, and the number of spatial pixels be N = H × W.

[0073] Reshape Q, K, V into , This represents the number of samples processed in parallel simultaneously. Indicates the height of the feature map. Indicates the width of the feature map.

[0074] Content attention calculation: Norm normalization of Query and Key (L2 normalization for each header):

[0075]

[0076] Calculate dot product attention and use scaling factor The initial value is set to 1 to control the numerical range. Finally, a softmax operation is performed to obtain the attention weight matrix calculated based on content features (similarity between pixels). :

[0077]

[0078] Attention-map weighted value:

[0079]

[0080] in, Context-enhanced features are features obtained by reweighting and aggregating the Value feature based on the correlation between feature contents.

[0081] Positional attention calculation: First, add the positional embedding P (a learnable parameter):

[0082]

[0083] in, P is used to provide a learnable location information encoding for each spatial location (h, w). Its role is to supplement the insufficient global location information in convolution operations, enabling the attention mechanism to simultaneously focus on spatial relationships and channel features. At the start of training, P is typically initialized randomly.

[0084] right Projection And split it into multiple positions using the method described above, and calculate:

[0085]

[0086]

[0087]

[0088] in, Indicates location-enhanced features The query vector obtained by projection onto the vector. Indicates location-enhanced features The key vector obtained by projection onto the middle. and This represents the normalized query vector and key vector. This represents the spatial attention weights calculated based on location information.

[0089] Content and position attention fusion involves summing the two outputs one by one and restoring their shapes to obtain the fused features. :

[0090]

[0091] A new fused feature representation is obtained through 1×1 convolutional projection.

[0092] In the feature enhancement acceleration, a lighter convolutional sequence is used to enhance semantic features, and Swish is used to improve training stability and performance.

[0093] The specific steps are as follows:

[0094] Basic features are extracted through preliminary convolution. :

[0095]

[0096] exist Based on this, spatial features are extracted through depthwise convolution enhancement. :

[0097]

[0098] exist Based on the non-linear semantic features activated by Swish :

[0099]

[0100] Finally, for Perform another 3×3 convolution to obtain the enhanced feature map. :

[0101]

[0102] Finally, the optimized features of the two modalities are concatenated to obtain... To reshape GAP into Then perform Dropout, and finally apply the learnable weight matrix. Categorize:

[0103]

[0104] In classification tasks, since the number of normal samples far exceeds the number of early and mid-to-late stage samples, directly using cross-entropy loss for training can easily lead to the model being overly biased towards the majority class, thereby reducing its ability to identify the minority class (especially early-stage glaucoma). Therefore, to address the class imbalance problem, this invention introduces the FocalLoss loss function in both training stages. Specifically, in the first stage of anomaly detection (normal vs. glaucoma) and the second stage of severity classification (early vs. mid-to-late stage), this loss function is used to improve the model's attention to hard-to-classify samples.

[0105] The mathematical expression for Focal Loss is as follows:

[0106]

[0107] in, Represents the loss function. This represents the category weight coefficient, used to balance the contributions of different categories in the loss calculation. This represents the model's predicted probability of the true class. Indicates the regulating factor, when At the same time, it can reduce the loss contribution to easily classified samples and amplify the impact of difficult-to-classify samples.

[0108] Thus, by using Focal Loss in both the anomaly detection and severity grading stages, the overall training stability is maintained while significantly improving the performance degradation caused by class imbalance, thereby enhancing the robustness and clinical applicability of the model.

[0109] Additionally, it should be noted that using the Focal Loss function to optimize model parameters includes the following steps:

[0110] 1) Initialize parameters: At the start of training, all parameters of the network (including agent vectors and other model weights) are randomly initialized.

[0111] 2) Forward propagation: The input image is represented by network features and then the classification probability is output by the classifier.

[0112] 3) Calculate the total loss using the Focal Loss function.

[0113] 4) Backpropagation: By calculating the gradient of the loss function, all learnable parameters of the model are updated, thereby gradually optimizing the representational ability of the network.

[0114] 5) Parameter update: The model parameters are updated using the optimization algorithm (Adam), and the training is iterated until the model converges.

[0115] In summary, the glaucoma detection method based on feature fusion according to embodiments of the present invention firstly acquires a fundus image and an optical coherence tomography (OCT) image corresponding to the sample to be detected, and preprocesses the fundus image to obtain a first preprocessed image, and preprocesses the OCT image to obtain a second preprocessed image; then, the first preprocessed image and the second preprocessed image are input into a pre-trained anomaly detection model, so that the anomaly detection model performs feature fusion on the first preprocessed image and the second preprocessed image to obtain a fused feature map, and determines whether the sample to be detected is a glaucoma sample based on the fused feature map; then, if so, the first preprocessed image and the second preprocessed image are input into a pre-trained severity analysis model, so that the severity of the sample to be detected is analyzed by the severity analysis model, and a final detection result is generated; thereby effectively utilizing the respective features of the fundus image and the OCT image to effectively detect glaucoma and improve the accuracy of glaucoma detection results.

[0116] Secondly, embodiments of the present invention provide a computer-readable storage medium storing a glaucoma detection program based on feature fusion, which, when executed by a processor, implements the glaucoma detection method based on feature fusion as described above.

[0117] Thirdly, embodiments of the present invention propose a glaucoma detection system based on feature fusion, such as... Figure 2 As shown, the glaucoma detection system based on feature fusion includes: a preprocessing module 10, an anomaly detection module 20, and a severity analysis module 30.

[0118] The preprocessing module 10 is used to acquire the fundus image and optical coherence tomography image corresponding to the sample to be detected, and to preprocess the fundus image to obtain a first preprocessed image, and to preprocess the optical coherence tomography image to obtain a second preprocessed image.

[0119] The anomaly detection module 20 is used to input the first preprocessed image and the second preprocessed image into the pre-trained anomaly detection model, so as to perform feature fusion on the first preprocessed image and the second preprocessed image through the anomaly detection model to obtain a fused feature map, and determine whether the sample to be detected is a glaucoma sample based on the fused feature map.

[0120] The severity analysis module 30 is used to input the first preprocessed image and the second preprocessed image into the pre-trained severity analysis model when the sample to be detected is a glaucoma sample, so as to analyze the severity of the sample to be detected through the severity analysis model and generate the final detection result.

[0121] In some embodiments, the pre-trained anomaly detection model includes a linear branch and a non-linear branch. The determination of whether a sample to be detected is a glaucoma sample based on the fused feature map includes: calculating the corresponding linear branch output based on the fused feature map based on the linear branch; calculating the corresponding non-linear branch output based on the fused feature map based on the non-linear branch; and summing the linear branch output and the non-linear branch output by the principal element to obtain the final anomaly detection result.

[0122] It should be noted that the above description of the glaucoma detection method based on feature fusion also applies to this glaucoma detection system based on feature fusion, and will not be repeated here.

[0123] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0124] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0125] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0126] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0128] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0129] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0130] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A feature fusion based glaucoma detection method, characterized in that, The method comprises the following steps: obtaining fundus image and optical coherence tomography image corresponding to a sample to be detected, and pre-processing the fundus image to obtain a first pre-processed image and pre-processing the optical coherence tomography image to obtain a second pre-processed image; inputting the first pre-processed image and the second pre-processed image into a pre-trained anomaly detection model to obtain a fusion feature map by feature fusion of the first pre-processed image and the second pre-processed image through the anomaly detection model, and judging whether the sample to be detected is a glaucoma sample based on the fusion feature map; if yes, inputting the first pre-processed image and the second pre-processed image into a pre-trained severity analysis model to analyze the severity of the sample to be detected through the severity analysis model, and generating a final detection result; wherein the pre-trained severity analysis model comprises a double-branch feature extraction layer and an adaptive lightweight cross-attention layer; the double-branch feature extraction layer is used for feature extraction of the first pre-processed image and the second pre-processed image to obtain corresponding branch features; the adaptive lightweight cross-attention layer is used for feature fusion of the branch features, and suppresses the adverse features in the fused features based on attention interaction; the pre-trained anomaly detection model comprises a linear branch and a nonlinear branch, wherein judging whether the sample to be detected is a glaucoma sample based on the fusion feature map comprises: calculating a corresponding linear branch output based on the linear branch according to the fusion feature map; calculating a corresponding nonlinear branch output based on the nonlinear branch according to the fusion feature map; performing principal element addition on the linear branch output and the nonlinear branch output to obtain a final anomaly detection result.

2. The feature fusion based glaucoma detection method of claim 1, wherein, calculating a corresponding linear branch output based on the linear branch according to the fusion feature map comprises: performing global average pooling on the fusion feature map to obtain a corresponding fusion feature vector, and randomly discarding channels in the fusion feature vector to obtain a linear input vector; applying an activation function to the linear input vector to obtain a nonlinear pre-transformation; performing linear transformation on the nonlinear pre-transformation based on a first weight matrix to obtain the linear branch output.

3. The feature fusion based glaucoma detection method of claim 1, wherein, calculating a corresponding nonlinear branch output based on the nonlinear branch according to the fusion feature map comprises: for each input dimension of the fusion feature map, calculating a corresponding basis vector of the input dimension; splicing the basis vectors to obtain a corresponding total basis representation; performing linear combination on the total basis representation based on a second weight matrix to obtain the nonlinear branch output.

4. The feature fusion based glaucoma detection method of claim 1, wherein, The adaptive lightweight cross-attention layer comprises a normalization module, a position-aware cross-attention module and a feature enhancement module; the output of the adaptive lightweight cross-attention layer is expressed by the following formula: ； wherein, denotes the output of the adaptive lightweight cross-attention layer, denotes the branch feature, denotes the adaptive fusion coefficient, denotes the output of the position-aware cross-attention module, denotes the output of the feature enhancement module.

5. The feature fusion based glaucoma detection method of claim 1, wherein, the loss function of the pre-trained severity analysis model is expressed by the following formula: ； wherein, represents a loss function, represents a class weight coefficient, represents a predicted probability of the model for the true class, represents a regulation factor.

6. A computer-readable storage medium, characterized in that, A storage medium having stored thereon a feature fusion based glaucoma detection program which, when executed by a processor, implements the feature fusion based glaucoma detection method of any one of claims 1-5.

7. A feature fusion based glaucoma detection system characterized by, Comprise: a preprocessing module for obtaining fundus photograph images and optical coherence tomography images corresponding to a sample to be detected, and pre-processing the fundus photograph images to obtain first pre-processed images, and pre-processing the optical coherence tomography images to obtain second pre-processed images; an anomaly detection module for inputting the first pre-processed images and the second pre-processed images into a pre-trained anomaly detection model to obtain a fusion feature map by feature fusion of the first pre-processed images and the second pre-processed images through the anomaly detection model, and determining whether the sample to be detected is a glaucoma sample based on the fusion feature map; a severity analysis module for inputting the first pre-processed images and the second pre-processed images into a pre-trained severity analysis model to analyze the severity of the sample to be detected through the severity analysis model when the sample to be detected is a glaucoma sample, and generating a final detection result; wherein the pre-trained severity analysis model comprises a double-branch feature extraction layer and an adaptive lightweight cross-attention layer; the double-branch feature extraction layer is used for feature extraction of the first pre-processed images and the second pre-processed images to obtain corresponding branch features; the adaptive lightweight cross-attention layer is used for feature fusion of the branch features, and suppresses adverse features in the fused features based on attention interaction; the pre-trained anomaly detection model comprises a linear branch and a nonlinear branch, wherein determining whether the sample to be detected is a glaucoma sample based on the fusion feature map comprises: calculating a corresponding linear branch output based on the linear branch according to the fusion feature map; calculating a corresponding nonlinear branch output based on the nonlinear branch according to the fusion feature map; performing principal element addition on the linear branch output and the nonlinear branch output to obtain a final anomaly detection result.

Images