Multi-label lesion classification and identification method based on wide-angle fundus image

Abstract

The application discloses a multi-label lesion classification and identification method based on a wide-angle fundus image and belongs to the technical field of medical image processing. The method comprises the following steps: constructing a heterogeneous graph structure that fuses spatial distribution information of lesions and clinical semantic relationships; performing cross-scale feature fusion on the original wide-angle fundus image through the heterogeneous graph structure to extract fine-grained features of the optic disc region, the macular region and the peripheral retinal region; determining a class-level dynamic weight calibration mechanism for different lesion categories according to the fine-grained features to adaptively adjust the classification threshold of each lesion category; combining the heterogeneous graph structure and medical prior knowledge constraints to mine the synergy and inhibition relationships between different lesions by using a graph attention network; and performing joint identification on the multiple lesions based on the adjusted classification threshold and the mined relationships to output a multi-label classification result.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method for multi-label lesion classification and identification based on wide-angle fundus images. Background Technology

[0002] Wide-angle fundus images can cover a large area of ​​the retina and its surrounding anatomical structures, providing rich information for screening various fundus diseases such as diabetic retinopathy, glaucoma, and age-related macular degeneration. Fundus images often contain multiple lesions simultaneously, and these lesions are highly complex in terms of spatial distribution, morphological characteristics, and interrelationships. Achieving accurate classification and identification of multi-labeled lesions remains a challenge in current research.

[0003] Existing methods for identifying multiple fundus lesions typically employ uniform grid division or global feature extraction to process images, neglecting the spatial regional differences guided by anatomical structures. The optic disc, macula, and peripheral retina exhibit significantly different levels of clinical attention and lesion patterns. Uniform feature extraction leads to the loss of fine-grained information in key areas, making it difficult to capture discriminative representations of minute lesions. Furthermore, conventional methods largely rely on fixed classification thresholds to handle multi-label prediction problems. Due to variations in lighting conditions, imaging angles, and individual patient differences during wide-angle fundus image acquisition, image quality fluctuates considerably. A uniform threshold cannot adapt to different image quality levels and the sample distribution characteristics of various lesions, resulting in both missed detections of low-frequency lesions and misdiagnosis of high-frequency lesions.

[0004] Another prominent issue is that existing technologies often classify lesions in isolation, failing to fully utilize the pathophysiological connections between lesions within medical knowledge. In clinical practice, some lesions often coexist, while others rarely do due to conflicting pathogenesis mechanisms. Ignoring these structured lesion relationships can cause model predictions to violate basic medical logic. Although some works have attempted to introduce graph networks to model label relationships, most are based solely on statistical co-occurrence, without deep binding to the spatial occurrence patterns of lesions or explicit constraints on medical rules in model optimization, resulting in a lack of clinical interpretability in the discovered relationships. Therefore, the core issue for achieving accurate multi-label lesion identification in wide-angle fundus images is how to organically integrate prior knowledge of retinal spatial distribution with clinical semantic knowledge to enhance feature expression, and on this basis, establish a joint reasoning mechanism that combines image quality adaptation and lesion relationship constraints. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-label lesion classification and identification method based on wide-angle fundus images, in order to solve the problems in the existing technology where the uniform feature extraction strategy ignores the scale differences of different anatomical regions of the retina, the fixed classification threshold is difficult to take into account the image quality fluctuations and the imbalance of lesion categories, and the lesion relationship modeling is deviated from the constraints of spatial distribution and medical rules, resulting in insufficient accuracy and clinical consistency in the joint identification of multiple lesions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The present invention discloses a multi-label lesion classification and identification method based on wide-angle fundus images, comprising: preprocessing the acquired wide-field-of-view fundus images, enhancing contrast using adaptive histogram equalization, extracting effective fundus regions and normalizing their sizes via an eyeball boundary segmentation network to obtain standardized original wide-angle fundus images; constructing a heterogeneous graph structure that integrates lesion spatial distribution information and clinical semantic relationships; performing cross-scale feature fusion through the heterogeneous graph structure to extract fine-grained features of the optic disc region, macular region, and peripheral retinal region; adaptively adjusting the classification threshold of each lesion category using a category-level dynamic weight calibration mechanism based on the fine-grained features; combining the heterogeneous graph structure with medical prior knowledge constraints extracted from clinical guidelines, and using a graph attention network to mine synergistic and inhibitory relationships between different lesions; jointly identifying multiple lesions in the image based on the adjusted classification thresholds and the mined lesion relationships, and outputting multi-label classification results and a correlation matrix representing the mutual influence between lesions. This method deeply integrates anatomical structural characteristics, image quality factors, and clinical logic rules, which improves the detection accuracy of various lesions in wide-angle fundus images while enhancing the interpretability of the recognition results.

[0007] Preferably, in the preprocessing step, adaptive histogram equalization is performed on wide-angle fundus images with an imaging angle greater than or equal to 120 degrees to obtain an image with enhanced contrast; a pre-trained eyeball boundary segmentation network is used to obtain the complete eyeball boundary contour, and invalid areas outside the orbit are cropped and removed based on this contour before scaling to a preset uniform size. After preprocessing, image noise is suppressed, and the optic disc, macula, and surrounding structures are clearer, laying a consistent and reliable foundation for subsequent heterogeneous image construction and feature extraction.

[0008] In constructing the heterogeneous graph structure, this invention divides the original wide-angle fundus image into multiple spatial regions, with each spatial region serving as a spatial node. Based on the semantic descriptions of lesions in the clinical knowledge base, each lesion category is designated as a semantic node. The historical probability of each lesion category appearing within each spatial region is statistically analyzed, and spatial-semantic edges are constructed between spatial nodes and semantic nodes, with their weights determined by this historical probability. Simultaneously, based on the clinical co-existence probabilities between different lesion categories recorded in the clinical knowledge base, semantic-semantic edges are constructed between different semantic nodes, with their weights determined by the clinical co-existence probabilities. The spatial nodes, semantic nodes, spatial-semantic edges, and semantic-semantic edges together constitute the heterogeneous graph structure. This heterogeneous graph organically encodes the spatial distribution patterns and semantic associations of lesions, providing a structured multi-source prior for subsequent multi-scale feature extraction and lesion relationship mining. As a more preferred implementation, the spatial region division employs a non-uniform grid partitioning strategy based on retinal anatomy, where the grid density of the optic disc and macula regions is higher than that of the peripheral retinal regions. This classification enables key anatomical sites to achieve higher spatial resolution, which is beneficial for capturing small lesions such as microaneurysms and drusen during the fine-grained feature extraction stage, thereby improving the overall detection sensitivity.

[0009] Regarding cross-scale feature fusion, this invention locates the spatial nodes corresponding to the optic disc region, macula region, and peripheral retinal region from the heterogeneous image structure; features are extracted from the optic disc region using a first convolution kernel size to obtain an optic disc feature map; features are extracted from the macula region using a second convolution kernel size smaller than the first convolution kernel size to obtain a macular feature map, preserving a more refined structure; features are extracted from the peripheral retinal region using a third convolution kernel size larger than the first convolution kernel size to obtain a peripheral feature map, capturing large-scale exudation or degeneration areas; the optic disc feature map, macular feature map, and peripheral feature map are concatenated along the channel dimension, and global average pooling is performed to obtain fine-grained features that represent the entire image and take into account multi-scale information. This cross-scale strategy effectively adapts to the inherent scale differences of different retinal regions, making the feature representation rich in both local texture details and overall morphological information.

[0010] The implementation process of the category-level dynamic weight calibration mechanism is as follows: The fine-grained features are input into a category-level attention network consisting of two fully connected layers and one non-linear activation layer connected in series, generating an initial weight coefficient for each lesion category. Simultaneously, an image-level quality score, evaluated by a lightweight quality assessment network based on the original wide-angle fundus image, is obtained. This score is a scalar between 0 and 1, representing the overall image sharpness and contrast. The initial weight coefficients for each lesion category are scaled according to the image-level quality score to obtain the dynamic weights for each lesion category. The dynamic weights for each lesion category are multiplied by a global baseline classification threshold to obtain an adaptive classification threshold. Through the above settings, different lesion categories can autonomously correct their decision boundaries under different image quality conditions: automatically increasing the threshold to suppress false positives when image quality is poor, and appropriately decreasing the threshold to avoid missing true lesions when the image is clear. Furthermore, the global baseline classification threshold is predetermined in the following way: a training dataset containing multiple lesion categories is collected, and the sample frequency of each lesion category is counted; the sample frequency is input into a threshold mapping function, which maps low sample frequencies to higher baseline thresholds and high sample frequencies to lower baseline thresholds, thereby effectively mitigating the bias caused by long-tail distribution in the initial state.

[0011] As a key aspect of this invention, when using graph attention networks to mine the synergistic and inhibitory relationships between lesions, all semantic nodes and the semantic-semantic edges connecting them are extracted from the heterogeneous graph structure to form a lesion relationship subgraph. The initial lesion feature vector corresponding to each semantic node is used as the initial hidden state of each node in the graph attention network. At each layer of the network, attention coefficients are calculated for the neighboring nodes of each semantic node. These coefficients are jointly determined by the hidden states of the current node and its neighboring nodes. The attention coefficients are multiplied by the weights of the semantic-semantic edges to obtain the fused attention weights. These fused attention weights are then used to perform a weighted summation of the hidden states of neighboring nodes to update the hidden state of the current semantic node. Simultaneously, the lesion coexistence rules and lesion mutual exclusion rules from medical prior knowledge are used as additional constraints. A rule penalty term is added to the loss function to penalize prediction results that violate these rules, forcing the model to learn lesion relationships that conform to clinical reality. The calculation method for the rule penalty is as follows: for each lesion coexistence rule, when the prediction result shows the first lesion as positive and the second lesion as negative, a first penalty value is accumulated; for each lesion mutual exclusion rule, when the prediction result shows both the first and second lesions as positive, a second penalty value is accumulated. The lesion coexistence and mutual exclusion rules in the aforementioned medical prior knowledge can be automatically obtained through natural language parsing of logically related statements in publicly available clinical guidelines for fundus diseases. This identifies the names of the first and second lesions, as well as the coexistence or mutual exclusion association type, and sets corresponding probability thresholds accordingly. This method eliminates the need for manual annotation of each rule, ensuring that the rules are easily expandable and updated.

[0012] When outputting multi-label classification results, the hidden state of each semantic node after the graph attention network update is fed into the classifier to obtain the original activation value for each lesion category. The generated adaptive classification threshold is used to binarize each original activation value: if the original activation value is greater than or equal to the adaptive classification threshold, the predicted label is set to positive; otherwise, it is set to negative. The predicted labels for all lesion categories are combined into a binary vector, which serves as the final multi-label classification result. Simultaneously, the synergistic and inhibitory relationships between lesions learned by the graph attention network are transformed into an association matrix and output to help explain the mutual influence between different lesion categories, enhancing the credibility of the intelligent diagnostic conclusions.

[0013] As a preferred approach, the process of the graph attention network mining lesion relationships is further optimized through iterative interaction with the category-level dynamic weight calibration mechanism. Specifically, the lesion relationship feature vector output by the graph attention network in the current round is used as a feedback signal and input into the category-level dynamic weight calibration mechanism. The category-level dynamic weight calibration mechanism recalculates the dynamic weight of each lesion category based on this feedback signal and updates the adaptive classification threshold. The updated adaptive classification threshold is then used to recalculate the classification loss in the next round of training of the graph attention network. The weighted classification loss is backpropagated to the graph attention network to guide the network in updating its internal attention parameters. This iterative process is repeated until the classification loss converges. This closed-loop mechanism enables the learning of dynamic threshold calibration and lesion co-inhibition relationships to mutually guide and co-evolve, allowing the model to achieve higher accuracy and stronger scene adaptability on complex wide-angle fundus images with multiple coexisting lesions.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: By constructing a heterogeneous graph structure that integrates spatial distribution information of lesions with clinical semantic relationships, feature analysis of fundus images is no longer limited to uniform processing. Instead, it involves non-uniform region division based on retinal anatomy, coupling spatial location with semantic concepts. In the feature extraction stage, the heterogeneous graph is used to locate spatial nodes corresponding to different regions. Based on the physiological scale differences of the optic disc, macula, and peripheral retina, convolutional kernels of different sizes are adapted for independent feature extraction, and multi-scale fused feature maps are formed through channel concatenation. This approach ensures the ability to capture features of small lesions within the optic disc, fine structures in the macular region, and large-scale lesions in the peripheral retina, thereby enhancing spatially specific fine-grained characterization from the source.

[0015] After obtaining multi-scale fusion features, a category-level dynamic weight calibration mechanism is further introduced. This mechanism takes fine-grained features as input, generates initial weights for each lesion category through a lightweight attention network, and scales the weights in conjunction with image-level quality scores, ultimately calculating an adaptive classification threshold for each lesion category. This adaptive adjustment gives the model the ability to dynamically change the judgment criteria based on the input image quality and category characteristics, effectively alleviating the global threshold mismatch problem caused by changes in imaging conditions, and significantly improving the response sensitivity to low-frequency lesions without sacrificing the accuracy of high-frequency category discrimination.

[0016] By constructing a lesion relationship subgraph based on the aforementioned heterogeneous graph structure, a graph attention network is used to transmit information between semantic nodes. Furthermore, a penalty term for the loss function is constructed using coexistence and mutual exclusion rules from medical prior knowledge, ensuring that the network is strictly constrained by clinical logic when mining synergistic and inhibitory relationships between lesions. This design prevents predictive combinations that conflict with known medical principles from being generated by the model output. The learned lesion relationship matrix exhibits clear clinical interpretability, and the consistency of multi-label prediction results in adhering to pathological relationships is fundamentally improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of a multi-label lesion classification and identification method based on wide-angle fundus images; Figure 2 This is a flowchart of the adaptive classification threshold generation process based on image quality perception; Figure 3 This is a flowchart of the training process for an attention network that integrates rules for the coexistence and mutual exclusion of lesions. Figure 4 This is a flowchart illustrating the iterative interaction between the graph attention network and the category-level dynamic weight calibration mechanism. Figure 5 This is a schematic diagram of the spatial grid division of the optic disc region, macula region, and peripheral retina region in a wide-angle fundus image. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1 This invention provides a method for multi-label lesion classification and identification based on wide-angle fundus images, including constructing a heterogeneous graph structure that integrates spatial distribution information of lesions with clinical semantic relationships; performing cross-scale feature fusion on the original wide-angle fundus image through the heterogeneous graph structure to extract fine-grained features of the optic disc region, macular region, and peripheral retinal region; determining a category-level dynamic weight calibration mechanism for different lesion categories based on the fine-grained features, and adaptively adjusting the classification threshold for each lesion category; combining the heterogeneous graph structure with medical prior knowledge constraints, using a graph attention network to mine the synergistic and inhibitory relationships between different lesions; and jointly identifying multiple lesions in the original wide-angle fundus image based on the adjusted classification threshold and the mined synergistic and inhibitory relationships, outputting multi-label classification results.

[0021] Example 1: In practice, a raw, wide-angle fundus image is acquired. The image angle of this raw, wide-angle fundus image is greater than or equal to 120 degrees, completely covering the optic disc region, macula region, and peripheral retina region. Adaptive histogram equalization is performed on the raw, wide-angle fundus image to obtain an image with enhanced contrast. This adaptive histogram equalization involves segmenting the raw, wide-angle fundus image into multiple non-overlapping local blocks, calculating a gray-level histogram for each local block, performing histogram equalization based on a contrast constraint, and then using bilinear interpolation to eliminate boundary effects between local blocks, thereby enhancing the contrast between the boundaries of blood vessels and lesions.

[0022] Optionally, the enhanced contrast image is input into a pre-trained eyeball boundary segmentation network to segment the complete eyeball boundary contour from the enhanced contrast image. The pre-trained eyeball boundary segmentation network adopts an encoder-decoder structure. The encoder contains four downsampling stages, each consisting of two 3×3 convolutional layers and a 2×2 max-pooling layer connected sequentially. The decoder contains four upsampling stages, each consisting of a 2×2 transposed convolutional layer and two 3×3 convolutional layers connected sequentially. The corresponding layers of the encoder and decoder are concatenated along the channel dimension via skip connections. The training process of the pre-trained eyeball boundary segmentation network is as follows: a batch of wide-angle fundus image samples are collected, and the binary mask of the eyeball boundary for each wide-angle fundus image sample is manually labeled, with pixels inside the eyeball boundary labeled as 1 and pixels outside labeled as 0; the pre-trained eyeball boundary segmentation network is trained using the cross-entropy loss function until the loss value converges. After training, the pre-trained eyeball boundary segmentation network receives an enhanced contrast image as input and outputs a probability map of the same size as the input. The value of each pixel in the probability map represents the probability that the pixel belongs to the inside of the eyeball boundary. The probability map is binarized with a threshold of 0.5 to obtain the eyeball boundary contour mask. The bounding rectangle of the largest connected region is extracted from the eyeball boundary contour mask as the complete eyeball boundary contour.

[0023] Based on the eyeball's boundary contour, the enhanced contrast image is cropped to remove invalid areas outside the eye socket, resulting in an effective fundus region image. Specifically, the bounding rectangle corresponding to the eyeball's boundary contour is used as the cropping box to crop a rectangular area from the enhanced contrast image. Then, the pixels outside the eyeball's boundary contour in the cropped image are filled with pure black to generate the effective fundus region image. The effective fundus region image is then scaled to a preset uniform size, such as 800×800 pixels, to generate the original wide-angle fundus image.

[0024] The construction process of the heterogeneous graph structure integrating lesion spatial distribution information and clinical semantic relationships is as follows. In specific implementation, the original wide-angle fundus image is divided into multiple spatial regions, each of which serves as a spatial node. The division of spatial regions adopts a non-uniform grid partitioning strategy based on retinal anatomy, where the grid density of the optic disc and macula regions is higher than that of the peripheral retinal regions. To implement the non-uniform grid partitioning strategy, a pre-trained optic disc and macula detection model is first used to locate the coordinates of the optic disc center and the fovea of ​​the macula in the original wide-angle fundus image. The optic disc and macula detection model uses a lightweight convolutional neural network. This lightweight convolutional neural network takes the original wide-angle fundus image as input and outputs a heatmap representing the probability of the optic disc center and a heatmap representing the probability of the fovea of ​​the macula. The coordinates corresponding to the maximum probability are taken as the coordinates of the optic disc center and the fovea of ​​the macula. A circular region with a radius of 120 pixels, centered at the optic disc center, is defined as the optic disc region. A circular region with a radius of 100 pixels, centered at the macula center, is defined as the macular region. The portion of the retina outside the optic disc and macular regions in the original wide-angle fundus image is defined as the peripheral retinal region. Within the optic disc and macular regions, a square grid with a side length of 20 pixels is used for partitioning, with each grid cell representing a spatial region block. Within the peripheral retinal region, a square grid with a side length of 60 pixels is used for partitioning, with each grid cell representing a spatial region block. Each spatial region block constitutes a spatial node in the heterogeneous graph.

[0025] Based on the semantic descriptions of lesions in the clinical knowledge base, each lesion category is treated as a semantic node. The clinical knowledge base stores information such as standard names, morphological descriptions, common locations, and associations with other lesions for various fundus lesions. All lesion categories, such as diabetic retinopathy, retinal vein occlusion, age-related macular degeneration, and glaucomatous optic neuropathy, are extracted from the clinical knowledge base, and each lesion category is used to construct a semantic node.

[0026] Spatial-semantic edges are constructed between spatial nodes and semantic nodes, with the weights determined by historical probabilities. The specific method for calculating the historical probability of each lesion category appearing within each spatial region block is as follows: A historical dataset containing a large number of labeled wide-angle fundus images is pre-constructed. For each historical wide-angle fundus image in the historical dataset, the lesion category label and the pixel-level location of each lesion instance in the image are recorded. The historical wide-angle fundus images are divided into spatial regions using the aforementioned non-uniform grid partitioning strategy. For each spatial region block, it is determined whether a lesion instance of a certain lesion category exists within the block. The frequency of a certain lesion category appearing at each spatial region block location in the entire historical dataset is counted. The proportion of this frequency to the total number of times the spatial region block appears as a valid fundus region in all historical images is calculated as the historical probability of the corresponding lesion category appearing within the spatial region block. The following formula is used to calculate the spatial node... With semantic nodes Spatial-semantic edge weights : in, This represents the spatial nodes obtained from statistics based on the historical dataset. Semantic nodes appear within the corresponding spatial region block The historical probability of the corresponding lesion type This represents the total number of semantic nodes. For smoothing coefficients, The range of values ​​is This is used to balance the weights between historical statistical probabilities and uniform distributions, preventing the weights of spatial-semantic edges with zero historical probabilities from degenerating to zero. When a certain lesion category has never appeared within a spatial region block... =0, Will degenerate into The historical dataset and the subsequent training dataset are the same dataset, and the historical probabilities are pre-calculated and stored before model training.

[0027] Based on the clinical co-existence probabilities between different lesion categories recorded in the clinical knowledge base, semantic-semantic edges are constructed between different semantic nodes, with the weights of these edges determined by the clinical co-existence probabilities. A description of the co-existence relationship between each pair of lesion categories is extracted from the clinical knowledge base. If a quantitative co-existence probability value exists, it is directly used as the clinical co-existence probability; if only a qualitative description exists, it is converted into a probability value according to a preset mapping rule, for example, "common co-existence" is mapped to 0.7, and "occasional co-existence" is mapped to 0.3. Semantic nodes... With semantic nodes The semantic-semantic edge weights are set to the corresponding clinical coexistence probabilities. If the clinical knowledge base does not record the association between the two lesion categories, the weight is set to 0, indicating no edge connection. The spatial nodes, semantic nodes, spatial-semantic edges, and semantic-semantic edges together form a heterogeneous graph structure.

[0028] The process of performing cross-scale feature fusion on the original wide-angle fundus image to extract fine-grained features of the optic disc region, macula region, and peripheral retina region is as follows: Locating spatial nodes corresponding to the optic disc region, macula region, and peripheral retina region from the heterogeneous map structure: Based on the category labels of the optic disc region, macula region, and peripheral retina region used when dividing the spatial region blocks during the construction of the heterogeneous map structure, all spatial nodes belonging to the optic disc region, the macula region, and the peripheral retina region are labeled.

[0029] In practice, a shared feature extraction backbone network is used to perform initial feature extraction on the original wide-angle fundus image to obtain a basic feature map. The shared feature extraction backbone network adopts the first four stages of a deep convolutional neural network, such as ResNet-50, and outputs a basic feature map with a spatial size of one-sixteenth of the original image after inputting the original wide-angle fundus image.

[0030] Based on the set of spatial nodes corresponding to the visual disk region, the mapping region of the visual disk region on the basic feature map is determined. Feature extraction is performed using the first convolutional kernel size to obtain the visual disk feature map. On the basic feature map, a region of interest calibration layer for the visual disk region is used. The features belonging to the visual disk region in the basic feature map are preserved using a mask of the visual disk region, while the features in other regions are set to zero. Then, the calibrated feature map is input into a visual disk network consisting of three convolutional layers. All convolutional layers use the first convolutional kernel size, which is 7×7, and the number of output channels is 256, to obtain the visual disk feature map.

[0031] Based on the set of spatial nodes corresponding to the macular region, the mapping region of the macular region on the basic feature map is determined. Feature extraction is performed using the second convolutional kernel size to obtain the macular feature map. On the basic feature map, a region of interest calibration layer is used to preserve features belonging to the macular region in the basic feature map using a mask, while setting the remaining regions to zero. The calibrated feature map is then input into a macular sub-network consisting of three convolutional layers. All convolutional layers use the second convolutional kernel size of 3×3, with 256 output channels, resulting in the macular feature map. The second convolutional kernel size of 3×3 is smaller than the first convolutional kernel size of 7×7 to accommodate the fine texture and subtle lesion features of the macular region.

[0032] Based on the set of spatial nodes corresponding to the peripheral retinal region, the mapping region of the peripheral retinal region on the basic feature map is determined. Feature extraction is performed using a third convolutional kernel size to obtain the peripheral feature map. On the basic feature map, a peripheral retinal region interest region calibration layer is used. A mask of the peripheral retinal region is used to retain features belonging to the peripheral retinal region in the basic feature map, while other regions are set to zero. The calibrated feature map is then input into a peripheral sub-network consisting of three convolutional layers. All convolutional layers use a third convolutional kernel size of 11×11, with 256 output channels, to obtain the peripheral feature map. The third convolutional kernel size of 11×11 is larger than the first convolutional kernel size of 7×7 to capture the distribution patterns of lesions such as hemorrhage and exudation over a larger area of ​​the peripheral retinal region.

[0033] The optic disc feature map, macular feature map, and peripheral retinal feature map are concatenated along the channel dimension to generate a multi-scale fused feature map with 768 channels. Global average pooling is then performed on the multi-scale fused feature map, compressing the feature map of each channel into a scalar value, resulting in a 768-dimensional feature vector, which serves as the fine-grained feature. This fine-grained feature encodes the semantic information of the optic disc region, macular region, and peripheral retinal region under different receptive fields, and will be subsequently used in a category-level dynamic weight calibration mechanism.

[0034] See Figure 5 This figure illustrates the results of spatial region segmentation of the original wide-angle fundus image based on the non-uniform grid partitioning strategy described in Example 1. The horizontal and vertical axes represent the horizontal and vertical pixel positions of the fundus image, respectively, ranging from 0 to 1200 pixels, covering the main visual field of the image. The legend identifies the grid cell distribution characteristics of the three types of spatial region blocks: The circles (○) represent grid cells in the visual disk region, located in the lower left of the image, and are distributed in a near-circular pattern. The spacing between the circle grid cells is a square grid with a side length of 20 pixels, reflecting a relatively dense division strategy within the visual disk region. The square (■) represents the grid cell of the macular region, which is adjacent to the right side of the visual disc region and is located in the lower right part of the image. It uses a square grid with the same side length of 20 pixels as the visual disc region, reflecting the need for fine-grained feature extraction of the macular region. Triangles (▲) represent grid units in the peripheral retinal region, distributed around the optic disc and macula, covering a large area of ​​the remaining image. The grid spacing is a square grid with a side length of 60 pixels. The larger grid size is used to capture wide-area lesion features in the peripheral retinal region.

[0035] Overall, the grid density in the optic disc and macula regions is significantly higher than that in the surrounding retinal regions, which meets the requirements of the non-uniform grid division strategy based on retinal anatomy in Example 1, ensuring that key anatomical structures receive a more refined spatial feature representation. This figure clearly illustrates the spatial distribution and size differences of the grid units in the three regions.

[0036] Example 2: In specific implementation, please refer to Figure 2 The fine-grained features obtained by global average pooling of the multi-scale fused feature map generated in Example 1 are used as the input of this example. The fine-grained features are fixed-dimensional feature vectors, denoted as... .

[0037] Fine-grained features are input into a category-level attention network, which consists of two fully connected layers and a non-linear activation layer cascaded together. The first fully connected layer inputs the fine-grained features... The dimension is mapped from the input dimension to a hidden dimension. The hidden dimension The setting is 128, based on the principle that the hidden dimension is chosen between the input and output dimensions and is close to the geometric mean of the input and output dimensions. This is suitable when the input dimension is 768 and the output dimension is the total number of lesion categories. hour, Choosing 128 provides a suitable model capacity. The output of the first fully connected layer is processed by a non-linear activation layer using the ReLU activation function. The second fully connected layer hides the dimension. The activation values ​​are mapped to the output dimension, and the output dimension is equal to the total number of lesion categories. , The specific value is determined by the total number of lesion categories included in the clinical knowledge base. The output of the second fully connected layer is normalized by the Sigmoid function, generating an initial weight coefficient for each lesion category. The initial weight coefficients of all lesion categories form a vector. ,in Indicates the first The initial weighting coefficients for each lesion category, The value ranges from 0 to 1.

[0038] Image-level quality scores are obtained for the original wide-angle fundus image. These scores are generated by a lightweight quality assessment network. This network takes the original wide-angle fundus image as input and outputs a scalar value between 0 and 1, representing the overall image sharpness and contrast. The core architecture of this lightweight quality assessment network consists of: a feature extraction section composed of five stacked 3×3 convolutional layers, each followed by a batch normalization layer and a ReLU activation function. The number of output channels for each convolutional layer is 16, 32, 64, 128, and 256, respectively. A global average pooling layer is connected after the fifth convolutional layer to compress the spatial dimension to a scalar. Following the global average pooling layer is a regression head containing two fully connected layers. The first fully connected layer has an input dimension of 256, an output dimension of 64, and is followed by a ReLU activation function. The second fully connected layer has an input dimension of 64, an output dimension of 1, and is activated by a Sigmoid function to output the image-level quality score. The lightweight quality assessment network is trained as follows: a batch of wide-angle fundus images are collected, and multiple ophthalmologists subjectively score each image based on its sharpness and contrast, with scores ranging from 0 to 1. The average score from multiple ophthalmologists is taken as the image-level quality score label for each image. The lightweight quality assessment network is trained using a mean squared error loss function until the loss function converges. After training, during the inference stage of multi-label lesion classification and recognition, the original wide-angle fundus images are input into the lightweight quality assessment network to obtain the corresponding image-level quality score, denoted as . .

[0039] The initial weights for each lesion category are scaled based on the image-level quality score to obtain the dynamic weights for each lesion category. The scaling process is implemented using the following formula: in, Indicates the first Dynamic weights for each lesion category, The range of values ​​is arrive ; This indicates that the category-level attention network is used for the first... Initial weighting coefficients generated for each lesion category; This represents the image-level quality score output by the lightweight quality assessment network. The range of values ​​is ; The quality modulation coefficient, Set to 0.5. The setting of 0.5 is based on striking a balance between the impact of image quality scoring and the preservation of basic weights. When the image quality score is 0.5, If the image quality score reaches its maximum value of 1 and the scaling factor is 1.0, then... The scaling factor is reduced to a minimum of 0, thus moderately reducing the dynamic weights rather than completely suppressing them when image quality deteriorates. This is achieved when the image quality score... At higher levels, the scaling factor approaches 1, and the dynamic weights... close to the initial weight coefficient When image quality score At lower levels, the scaling factor decreases, and the dynamic weights... The corresponding adjustment is lowered, reflecting the decrease in classification confidence when the overall image quality is poor.

[0040] The dynamic weight of each lesion category is multiplied by a global baseline classification threshold to obtain the adaptive classification threshold for each lesion category. Each lesion category has an independent adaptive classification threshold, denoted as _i_. The adaptive classification threshold for each lesion category is: , No. The global baseline classification threshold for each lesion category is: , We obtain it from the following formula: .

[0041] The pre-determination process of the global baseline classification threshold is as follows: A training dataset containing multiple lesion categories is collected, where each training image is labeled with a real multi-label lesion. The sample frequency of each lesion category in the training dataset is counted, defined as the proportion of images containing a certain lesion category in the training dataset to the total number of images in the training dataset. The sample frequency is input into a threshold mapping function, which maps low sample frequencies to high baseline thresholds and high sample frequencies to low baseline thresholds. The threshold mapping function adopts an exponential decay function form: ,in Indicates the first The sample frequency of each lesion category in the training dataset. The range of values ​​is ; The amplitude parameter is set to 0.4, based on the dynamic change of the control baseline threshold within an appropriate range. The decay rate parameter is set to 5.0, based on the principle that the baseline threshold should produce a significant difference when the sample frequency changes from the minimum to the maximum value. The baseline bias parameter is set to 0.1, based on the principle of ensuring that even lesion categories with extremely high sample frequencies still retain a non-zero activation threshold baseline. The base of the natural logarithm is used. The mapping result corresponding to each lesion category is used as the global baseline classification threshold for that lesion category. For rare lesion categories with low sample frequency, the mapped global baseline classification threshold is higher, which can suppress false positive predictions; for common lesion categories with high sample frequency, the mapped global baseline classification threshold is lower, which can improve the recall of positive samples.

[0042] Example 3: In specific implementation, please refer to Figure 3 From the heterogeneous graph structure, all semantic nodes and semantic-semantic edges connecting semantic nodes are extracted to form a lesion relationship subgraph. The heterogeneous graph structure, constructed in Example 1, includes spatial nodes, semantic nodes, spatial-semantic edges, and semantic-semantic edges. All nodes of type semantic node are selected from the heterogeneous graph structure, and all semantic-semantic edges between semantic nodes are retained to form a lesion relationship subgraph, denoted as […]. ,in Represents a set of semantic nodes. , This represents the total number of lesion categories. This represents the set of semantic-semantic edges, and the weights of the semantic-semantic edges are determined by the clinical coexistence probability.

[0043] The initial lesion feature vector corresponding to each semantic node is used as the initial hidden state of the corresponding node in the graph attention network. The method for obtaining the initial lesion feature vector is as follows: for the ... For each semantic node corresponding to a lesion category, semantic description text for that lesion category is extracted from a clinical knowledge base. A pre-trained medical text encoder is used to convert this semantic description text into a fixed-length dense vector, which serves as the initial lesion feature vector. The pre-trained medical text encoder employs the BioBERT model, which is pre-trained on a large-scale biomedical literature corpus. The input is the semantic description text of the lesion category, and the output is the hidden state corresponding to the [CLS] token in the last layer of the BioBERT model. The output vector has a dimension of 768. The initial lesion feature vector of each semantic node is denoted as... , This vector serves as the first vector in the graph attention network. The initial hidden state of each node.

[0044] In each layer of the graph attention network, attention coefficients are calculated for the neighboring nodes of each semantic node. These attention coefficients are determined by the initial hidden state of the node and the initial hidden states of its neighboring nodes. The graph attention network contains a total of [number missing] layers. layer, Set to 2, The rationale for setting it to 2 is that excessively deep graph networks can easily lead to oversmoothing in small-scale graphs such as lesion relationship subgraphs. A two-layer structure can achieve a balance between aggregating information from neighboring nodes and preserving the characteristics of the nodes themselves. Each layer of the graph attention network contains a shared linear transformation matrix and an attention vector. In the layer, the first The hidden state of each semantic node is represented as follows: ,when hour ; in the In the layer, the first The hidden state of each semantic node is represented as follows: , No. The semantic node and the first Attention coefficient between semantic nodes Calculate using the following formula: in, Indicates the first The linear transformation matrix of the layer, , Given the input hidden state dimension, when hour For 768, when hour for , This is the output dimension of layer 1; To output the hidden state dimension, the first layer Set to 384, Level 2 The dimensionality is set to 256, with the dimensionality decreasing layer by layer, based on the principle of progressively compressing the feature representation and removing redundant information. Indicates the first The attention vector of the layer, , These are trainable parameters, updated via backpropagation, with initial values ​​randomly initialized using a Xavier uniform distribution. This represents a vector concatenation operation; This represents the activation function of the rectified linear unit with leakage, with a negative slope set to 0.2; Indicates the first The first in the layer The semantic node is the first The original attention coefficients of the semantic node. After calculating the original attention coefficients of all neighboring nodes of a semantic node, the Softmax function is used to normalize them to obtain the normalized attention coefficients. .

[0045] The attention coefficient is multiplied by the weights of the semantic-semantic edges to obtain the fused attention weights. The weights of the semantic-semantic edges are denoted as... , This is a scalar value pre-determined based on the clinical co-existence probability between different lesion categories in the clinical knowledge base. The range of values ​​is Integrating attention weights The calculation method is as follows The process of using fused attention weights for subsequent calculations is as follows: normalized attention coefficients... Replace with fused attention weights Then, the fusion attention weights are re-normalized over the set of neighboring nodes to ensure that the sum of all weights is 1. The re-normalized fusion attention weights are then used to perform a weighted summation of the hidden states of neighboring nodes to update the hidden state of the current semantic node. ,in Indicates the first The set of neighboring nodes of a semantic node in the lesion relationship subgraph This represents the ELU activation function. After... After processing by the layered graph attention network, the first The final hidden state of each semantic node is The final hidden states of all semantic nodes constitute the output of the graph attention network.

[0046] Optionally, the rules for lesion coexistence and mutual exclusion in prior medical knowledge are obtained by extracting multiple logical association statements between lesions from publicly available clinical guidelines for fundus diseases. Each logical association statement is then parsed using natural language to identify the first lesion name, the second lesion name, and the association type. Natural language parsing is implemented using a rule-based text parser, which includes a lesion name dictionary and an association type trigger word list. The lesion name dictionary includes the official names and synonyms of all known fundus lesions. The association type trigger word list includes "common comorbidities," "often associated," and "may be associated" corresponding to coexistence types, and "rare comorbidities," "very rarely associated," and "usually not associated" corresponding to mutual exclusion types. When the association type is coexistence, a lesion coexistence rule is generated. This rule stipulates that the probability of the second lesion name appearing in an image containing the first lesion name is higher than a preset coexistence threshold. The preset coexistence threshold is set to 0.5, based on the fact that a probability greater than 0.5 indicates that the two lesions are more likely to coexist rather than appear independently. When the association type is mutually exclusive, a lesion mutual exclusion rule is generated. This rule stipulates that the probability of the second lesion name appearing in an image containing the first lesion name is lower than a preset mutual exclusion threshold. The preset mutual exclusion threshold is set to 0.05, based on the fact that a probability lower than 0.05 constitutes an extremely low probability event in medical statistics and can be considered a mutually exclusive relationship.

[0047] The rules of lesion coexistence and lesion mutual exclusion from prior medical knowledge are added as additional constraints to the loss function of the graph attention network. These rules are used to penalize predictions that violate either the lesion coexistence or mutual exclusion rules. The penalty term is calculated as follows: for each lesion coexistence rule, a first penalty value is accumulated when the prediction result shows a positive result for the first lesion and a negative result for the second lesion; this first penalty value is set to 1.0. For each lesion mutual exclusion rule, a second penalty value is accumulated when the prediction result shows a positive result for both the first and second lesions; this second penalty value is also set to 1.0. Formal definition ,in This represents the set of rules for coexisting lesions. Represents the set of mutually exclusive rules for lesions. This represents the predicted probability of the first lesion category in each rule. This represents the predicted probability of the second lesion category in each rule. The parenthesis represents an indicator function, which takes a value of 1 when the condition within the parentheses is true, and 0 otherwise. The total loss function of the graph attention network consists of the sum of the multi-label classification loss and the rule penalty term. The multi-label classification loss uses the binary cross-entropy loss function. The total loss function is... ,in This is the rule penalty weighting coefficient. Set to 0.1, The rationale for setting the learning rate to 0.1 is to moderately introduce prior knowledge constraints while primarily optimizing classification accuracy, thus avoiding the degradation of classification performance caused by rule-based penalty terms dominating the loss function. The training process uses the Adam optimizer, with an initial learning rate of 0.001. The learning rate decays to 0.5 times its current value every 20 training epochs, with a total of 100 training epochs and a batch size of 16.

[0048] Example 4: In specific implementation, the hidden state of each semantic node after the graph attention network is updated is input into a classifier, which outputs an original activation value for each lesion category. After training in Example 3, the graph attention network outputs the... The final hidden state of each semantic node , ,in The range of values ​​is arrive , This represents the total number of lesion categories. The classifier contains a fully connected layer with an input dimension of 256 and an output dimension of 1. The weight matrix of the fully connected layer is denoted as... The bias term is denoted as , Initialization using Xavier uniform distribution. Initialize to 0. For the first... Lesion category, original activation value Calculate using the following formula: in, Indicates the first The original activation value for each lesion category, It is a real scalar and has not been subject to any activation function constraints; The graph attention network is represented by the first... After the layer output, the first The final hidden state of each semantic node; This represents the weight matrix of the fully connected layer of the classifier; This represents the bias term of the fully connected layer of the classifier.

[0049] The original activation values ​​are binarized using the generated adaptive classification threshold for each lesion category. The adaptive classification threshold for each lesion category is generated and stored in Example 2. The adaptive classification threshold for each lesion category is denoted as . , It is dynamically calculated by a category-level dynamic weight calibration mechanism based on fine-grained features and image quality scores. The binarization determination process is as follows: when the original activation value... Greater than or equal to the adaptive classification threshold At that time, the first The predictive label for each lesion category is set to positive; when the original activation value Less than the adaptive classification threshold At that time, the first The predictive label for each lesion category is set to negative. The predictive label uses a binary symbol. It means that among them Indicates a positive result. It indicates a negative result.

[0050] The predicted labels for all lesion categories are combined into a binary vector, which serves as the multi-label classification result. Specifically, the predicted labels for each lesion category are arranged sequentially according to a fixed index order, forming a binary vector of length [length missing]. binary vector This binary vector is the final multi-label classification result, where each bit corresponds to the existence of a lesion category.

[0051] Simultaneously, based on the cooperation and inhibition relationships output by the graph attention network, an association matrix is ​​generated. This association matrix is ​​used to help explain the mutual influence between different lesion categories in the multi-label classification results. The graph attention network in the... In each layer, a fusion attention weight is calculated for each pair of semantic nodes with semantic-semantic edge connections. This fusion attention weight incorporates both data-driven attention coefficients and clinically prior coexistence probabilities, characterizing the synergistic and inhibitory relationships between lesions. For any two lesion categories... and If the semantic nodes in the lesion relation subgraph With semantic nodes If semantic-semantic edges exist, then the normalized fused attention weights are extracted from the last layer of the graph attention network and used as elements of the corresponding positions in the association matrix; if semantic nodes... With semantic nodes If there are no edges connecting the elements, the corresponding element in the association matrix is ​​set to 0. The association matrix is ​​denoted as... , of which elements The calculation method is as follows: when semantic node With semantic nodes When there are semantic-semantic edges, , The graph attention network is represented by the first... semantic nodes in layers semantic nodes The renormalized fusion attention weights; when semantic nodes With semantic nodes When there are no edges connecting them, Relationship matrix Middle elements The larger the value, the higher the lesion category in the multi-label classification process. Lesion type The stronger the synergistic effect produced by the feature representation, the weaker the relationship or the absence of a direct correlation. The association matrix is ​​output together with the multi-label classification results, allowing clinicians to refer to the potential association logic between lesions when interpreting the prediction results.

[0052] Example 5: In specific implementation, please refer to Figure 4 The lesion relationship feature vector output by the graph attention network in the current round is used as a feedback signal and input into the category-level dynamic weight calibration mechanism. The structure and training process of the graph attention network have been described in detail in Example 3. The current round is denoted as... Graph attention network in the first After each training round, the final hidden state of each semantic node is output. The final hidden state of each semantic node is The lesion relationship feature vector is obtained by aggregating the final hidden states of all semantic nodes. The aggregation method is to combine all... The final hidden states of each semantic node are averaged along the channel dimension to obtain a global lesion relationship feature vector with dimension 256. : in, Indicates the total number of lesion categories. The specific value is determined by the total number of all lesion categories included in the clinical knowledge base; Indices representing semantic nodes The range of values ​​is arrive ; Indicates the first Round Graph Attention Network After the layer output, the first The final hidden state of each semantic node. ; Indicates the first Global lesion relationship feature vectors for each round, .

[0053] The category-level dynamic weight calibration mechanism recalculates the dynamic weights for each lesion category based on the lesion relationship feature vector. The internal structure of this mechanism, described in detail in Example 2, includes a category-level attention network consisting of two fully connected layers and one non-linear activation layer connected in series. In each round, the input to the category-level attention network is the original fine-grained features. Based on this, additional feature vectors relating lesions are spliced ​​together. This results in enhanced fine-grained features. ,in This represents a vector concatenation operation, enhancing fine-grained features. The dimension is This will enhance fine-grained features. The input is a category-level attention network. The input dimension of the first fully connected layer of the category-level attention network is adjusted to 1024, while the output dimension remains at 128. The output dimension of the second fully connected layer remains at 128. The category-level attention network generates an updated initial weight coefficient for each lesion category, denoted as . Using the image-level quality score obtained in Example 2. and scaling formula Scaling is performed to obtain the first... Rounds Dynamic weights for each lesion category Using dynamic weights Update the adaptive classification threshold; the updated adaptive classification threshold is: ,in The first one predetermined in Example 2 Global baseline classification threshold for each lesion category.

[0054] Using the updated adaptive classification threshold, the classification loss in the next training round of the graph attention network is recalculated with weights. The classification loss of the graph attention network adopts the binary cross-entropy loss function. In the training rounds, for the first For each lesion category, the model's predicted probability is... The real label is The standard binary cross-entropy loss is The weighted calculation method is as follows: using the updated adaptive classification threshold... For reference, different weighting coefficients are applied to positive and negative samples respectively. When the true label... When the model predicts probability Below Then the loss value of that sample is multiplied by the amplification factor. , Set to 2.0. The setting of 2.0 is based on applying a larger penalty gradient to positive samples that are easily misclassified as negative; when the true label... When the model predicts probability Higher than Then the loss value of that sample is multiplied by the amplification factor. , Set to 1.5. The rationale for setting it to 1.5 is to apply a moderate penalty to false positives, avoiding excessive suppression that could lead to a decrease in recall. The weighted binary cross-entropy loss is denoted as... The weighted loss of all lesion categories is averaged to obtain the first... Classification loss per round The total loss function of the graph attention network is composed of the classification loss and the rule penalty term defined in Example 3: ,in This is the rule penalty weighting coefficient. Set it to 0.1.

[0055] Weighted classification loss Backpropagation is performed to the graph attention network, guiding it to update its internal attention parameters. The internal attention parameters of the graph attention network include the linear transformation matrix of each layer. and attention vector These two types of parameters are clearly defined in Example 3. The backpropagation process uses the Adam optimizer with a learning rate of 0.001 and a gradient clipping threshold of 1.0. The gradient clipping threshold is set to 1.0 to prevent gradient explosion from causing training instability. During backpropagation, the internal attention parameters of the graph attention network are updated based on the loss gradient. The updated parameters will be used for the... Forward propagation of rounds.

[0056] Repeat the above steps until the classification loss converges. The convergence criterion is the classification loss over five consecutive training epochs. The decrease in all values ​​was less than 0.001, and the iteration stopped after reaching the convergence criterion. The initial state of the iterative interaction process was: in round 0, the lesion relationship feature vector... Assuming a zero vector, the category-level dynamic weight calibration mechanism relies solely on the original fine-grained features. Initial dynamic weights are generated. In the first round, the graph attention network performs complete forward and backward propagation under the initial adaptive classification threshold. After the first round, meaningful lesion relationship feature vectors are generated, and iterative interactions in subsequent rounds are initiated. The iterative interactions do not change the weight values ​​of semantic-semantic edges in the graph attention network. The weight values ​​of semantic-semantic edges are always determined by the clinical co-existence probabilities in the clinical knowledge base and are initialized before training begins.

[0057] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A multi-label lesion classification and identification method based on wide-angle fundus images, characterized in that, The multi-label lesion classification and identification method based on wide-angle fundus images includes: A heterogeneous graph structure integrating spatial distribution information of lesions and clinical semantic relationships is constructed, specifically as follows: The original wide-angle fundus image is divided into multiple spatial regions, each region serving as a spatial node; based on the semantic descriptions of lesions in a clinical knowledge base, each lesion category is designated as a semantic node; the historical probability of each lesion category appearing within each spatial region is statistically analyzed, and spatial-semantic edges are constructed between the spatial nodes and the semantic nodes, with the weights of the spatial-semantic edges determined by the historical probabilities; based on the clinical co-existence probabilities between different lesion categories recorded in the clinical knowledge base, semantic-semantic edges are constructed between different semantic nodes, with the weights of the semantic-semantic edges determined by the clinical co-existence probabilities; the spatial nodes, semantic nodes, spatial-semantic edges, and semantic-semantic edges together form the heterogeneous graph structure. Through the heterogeneous graph structure, cross-scale feature fusion is performed on the original wide-angle fundus image to extract fine-grained features of the optic disc region, macular region and peripheral retinal region. Based on the fine-grained features, a category-level dynamic weight calibration mechanism is determined for different lesion categories to adaptively adjust the classification threshold for each lesion category. Specifically, the fine-grained features are input into a category-level attention network, which consists of two fully connected layers and a non-linear activation layer connected in series. An initial weight coefficient is generated for each lesion category through the category-level attention network. An image-level quality score is obtained corresponding to the original wide-angle fundus image, which is used to characterize the overall image sharpness and contrast. The initial weight coefficient for each lesion category is scaled according to the image-level quality score to obtain the dynamic weight for each lesion category. The dynamic weight for each lesion category is multiplied by a global baseline classification threshold to obtain the adaptive classification threshold for each lesion category. By combining the aforementioned heterogeneous graph structure with medical prior knowledge constraints, graph attention networks are used to explore synergistic and inhibitory relationships between different lesions; Based on the adjusted classification threshold and the discovered synergistic and inhibitory relationships, multiple lesions in the original wide-angle fundus image are jointly identified, and multi-label classification results are output. The spatial region blocks are divided using a non-uniform grid division strategy based on retinal anatomy, wherein the grid density of the optic disc region and the macula region is higher than that of the peripheral retinal region.

2. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 1, characterized in that, The process of performing cross-scale feature fusion on the original wide-angle fundus image to extract fine-grained features of the optic disc region, macular region, and peripheral retinal region is as follows: Locate the spatial nodes corresponding to the optic disc region, the macula region, and the peripheral retina region from the heterogeneous graph structure; Feature extraction is performed on the visual disk region using the first convolution kernel size to obtain the visual disk feature map; The macula region is used to extract features using a second convolution kernel size to obtain a macula feature map, wherein the second convolution kernel size is smaller than the first convolution kernel size; Feature extraction is performed on the peripheral retinal region using a third convolution kernel size to obtain a peripheral feature map, wherein the third convolution kernel size is larger than the first convolution kernel size; The visual disc feature map, the macula feature map, and the peripheral feature map are stitched together along the channel dimension to generate a multi-scale fused feature map; Global average pooling is performed on the multi-scale fused feature map to obtain the fine-grained features.

3. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 2, characterized in that, The image-level quality score is generated by a lightweight quality assessment network that takes the original wide-angle fundus image as input and outputs a scalar value between 0 and 1.

4. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 3, characterized in that, The global baseline classification threshold is predetermined in the following manner: Collect a training dataset containing multiple lesion categories, wherein each training image in the training dataset is labeled with a real multi-label lesion; Statistically analyze the sample frequency of each lesion category in the training dataset; The sample frequencies are input into a threshold mapping function, which maps low sample frequencies to high baseline thresholds and high sample frequencies to low baseline thresholds. The mapping result corresponding to each lesion category is used as the global baseline classification threshold for that lesion category.

5. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 4, characterized in that, Combining the aforementioned heterogeneous graph structure with medical prior knowledge constraints, the process of using graph attention networks to mine the synergistic and inhibitory relationships between different lesions is as follows: Extract all semantic nodes and semantic-semantic edges connecting the semantic nodes from the heterogeneous graph structure to form a lesion relationship subgraph; The initial lesion feature vector corresponding to each semantic node is used as the initial hidden state of the corresponding node in the graph attention network; In each layer of the graph attention network, attention coefficients are calculated for the neighboring nodes of each semantic node. These attention coefficients are jointly determined by the initial hidden state of the node and the initial hidden state of the neighboring nodes. The attention coefficient is multiplied by the weight of the semantic-semantic edge to obtain the fused attention weight; The hidden states of the adjacent nodes are weighted and summed using the fused attention weights to update the hidden state of the current semantic node; The lesion coexistence rule and lesion mutual exclusion rule in the medical prior knowledge are used as additional constraints. A rule penalty term is added to the loss function of the graph attention network. The rule penalty term is used to penalize the prediction results that violate the lesion coexistence rule or the lesion mutual exclusion rule.

6. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 5, characterized in that, The calculation method for the rule penalty item is as follows: for each lesion coexistence rule, when the prediction result shows that the first lesion is positive and the second lesion is negative, the first penalty value is accumulated; for each lesion mutual exclusion rule, when the prediction result shows that the first lesion and the second lesion are both positive, the second penalty value is accumulated.

7. The multi-label lesion classification and identification method based on wide-angle fundus images according to claim 6, characterized in that, The process of outputting multi-label classification results is as follows: The hidden state of each semantic node after the graph attention network is updated is input into a classifier, and the classifier outputs an original activation value for each lesion category. The original activation value is binarized using the adaptive classification threshold of each generated lesion category. When the original activation value is greater than or equal to the adaptive classification threshold, the predicted label of the lesion category is set to positive; otherwise, it is set to negative. The predicted labels for all lesion categories are combined into a binary vector, which serves as the result of the multi-label classification. Simultaneously, based on the cooperative and inhibitory relationships output by the graph attention network, an association matrix is ​​generated. This association matrix is ​​used to help explain the mutual influence between different lesion categories in the multi-label classification results.

Images