Rapid semantic segmentation system and method for pneumonia DR image

Abstract

The invention discloses a rapid semantic segmentation system and method for a pneumonia DR image. The rapid semantic segmentation system comprises a context graph construction module, a context information aggregation module, a detail and context information fusion module and a decoding fusion feature segmentation module. According to the method, when a to-be-segmented DR image is processed, a context information aggregation module gradually collects and integrates long-distance context features and detail features outside an image block based on a context graph neural network model of an end-to-end training mode; then, based on a detail and context fusion module, effectively fusing context information and detail features of the image blocks; and the fusion feature is segmented by a decoding fusion feature segmentation module so as to generate an accurate segmentation mask. According to the method, accurate segmentation and rapid evaluation of the pneumonia focus in the DR image are realized, and powerful assistance and support are provided for diagnosis of doctors.

Description

Technical Field

[0001] The present invention relates to a semantic segmentation system, and in particular to a fast semantic segmentation system and method for pneumonia DR images. Background Art

[0002] As a common respiratory infection disease, the timely and accurate diagnosis of pneumonia is crucial for the treatment and rehabilitation of patients. Traditional pneumonia diagnosis methods mainly rely on doctors' clinical experience and manual analysis of medical imaging data. However, this method is not only time-consuming and laborious, but also difficult to guarantee the accuracy and consistency of the diagnosis results. With the continuous development of medical imaging technology and the continuous innovation of computer algorithms, the semantic segmentation technology based on DR (Digital Radiography) images provides a new solution for the rapid evaluation of pneumonia.

[0003] As an important imaging method in the field of medical imaging, DR images have the advantages of high resolution, fast imaging speed, low radiation dose, etc., and are widely used in the diagnosis of various diseases. However, in the face of a large amount of DR image data, how to accurately extract pneumonia-related information from it has become a major challenge in the current field of medical image processing. The semantic segmentation technology based on DR images is an effective solution proposed for this challenge.

[0004] Semantic segmentation is a computer vision technology whose goal is to classify each pixel in an image to achieve a fine understanding of the image content. In DR images, the semantic segmentation technology can be applied to distinguish the lung region, pneumonia lesion region, and other regions such as the background, so as to achieve accurate identification and positioning of pneumonia lesions.

[0005] Existing semantic segmentation methods based on DR images adopt a strategy based on image patches to improve the calculation efficiency in view of the high resolution of DR images (for example, reaching the level of tens of thousands of pixels). However, the main limitation of this method is that it restricts the model's access to information outside the image patches, which will affect the accuracy of segmentation. Moreover, due to the diverse shapes and sizes of pneumonia lesions in DR images, and they are often interfered by surrounding tissues, organs, and noise, etc., the segmentation task becomes particularly complex. In addition, different patients, different shooting conditions, and different diagnostic requirements also pose higher requirements on the adaptability, robustness, and real-time performance of the segmentation algorithm. Summary of the Invention

[0006] In order to solve the deficiencies of the above technologies, the present invention provides a fast semantic segmentation system and method for pneumonia DR images.

[0007] To solve the above technical problems, the technical solution adopted by the present invention is: a fast semantic segmentation system for pneumonia DR images. The fast semantic segmentation system includes a context graph construction module, a context information aggregation module, a detail and context information fusion module, and a decoded fusion feature segmentation module;

[0008] The context graph construction module is used to construct a context graph through the context graph construction module;

[0009] Based on the context graph neural network model, the context information aggregation module is used to model the detail features and context relationships of each image patch of the DR image;

[0010] The detail and context information fusion module is used to fuse the long-distance context features and detail features outside the target region of the image patch;

[0011] After obtaining the fusion features that fuse the detail features and context features of the image patch, the decoded fusion feature segmentation module decodes the fusion features using a decoder and performs feature segmentation to generate an accurate segmentation mask.

[0012] Preferably, the processing process of the context graph construction module is as follows:

[0013] Construct a context graph G=(V, E), where V represents the set of image patch features and E represents the set of undirected edges connecting the image patches;

[0014] Extract the complete foreground region from the DR image to obtain a set P={p i |i = 1...N} containing N image patches, where p i represents a single image patch;

[0015] Encode each image patch p i into a feature vector v of dimension 1024 i ∈R 1024 ; a DR image is represented as a set V={v i |i = 1...N} containing N nodes;

[0016] Based on the spatial connectivity of the image patches, determine the set of undirected edges E={v i v i v j |(i, j) ∈ M} of the image patch p i and v j where v i and v

[0017] Preferably, the extraction process of the complete foreground region is as follows: Filter the foreground of the DR image according to the edge detection method, and segment it into non-overlapping image blocks. Apply the edge detection operator to calculate the gradient of the image, find the edge pixels of the image, connect the edge pixels into continuous edge curves through the edge connection algorithm, fill the foreground region according to the edge curves, and extract the complete foreground region.

[0018] Preferably, the processing process of the context information aggregation module for modeling long-distance context features is as follows:

[0019] The adjacency matrix A = [a i,j n×n is derived from the connection relationship between image block nodes; the elements [a i,j of the adjacency matrix are defined as follows:

[0020]

[0021] where E represents the set of undirected edges of the image blocks; v i , v j are any two feature vectors encoded by the image blocks;

[0022] The feature matrix represents the initial feature map of N nodes, and each

[0023] The context information aggregation beyond t steps can be expressed as:

[0024]

[0025] where F GCN represents the graph convolutional neural network; δ represents the activation function; P represents the weight matrix; represents the normalized adjacency matrix, which is used to balance the number of neighbors of each node, D represents the degree matrix, and I represents the identity matrix;

[0026] The target node x i aggregates features from its neighbor nodes and gradually expands the scope of its context information; after T0 steps of aggregation, the feature of node i changes from to and integrates context information from neighbors that are farther and farther away;

[0027] Starting from the i-th node, the neighbor set within t steps is represented as Nera t (x i ) = {x j |d(i,j) = t}, where d(i,j) represents the shortest path length between the i-th node and the j-th node;

[0028] Therefore, the features of the $i$-th node are updated based on the union of all neighbor features within $T_0$ steps, expressed as:

[0029]

[0030] where is a graph convolution operation that updates the features of node $i$ by aggregating information from increasingly distant neighbors; this process enables the features of the node to capture long-range context information Finally, a richer representation of the target image patch is learned.

[0031] Preferably, the context information aggregation module models the processing process of detailed features as:

[0032] The context information aggregation module uses an encoder composed of a convolutional layer, an activation function, a max pooling layer, and feature doubling to extract detailed features where represents the number of tokens of image patch $p$ i , $H$ is the image length, $W$ is the image width, and $l$ is the size used for segmenting the image patch.

[0033] Preferably, before inputting the detailed features and the long-range context features into the detailed and context fusion module for fusion, the detailed features are first position-encoded. By adding the position encoding, the uniqueness of the overall context in the detailed features is enhanced.

[0034] Preferably, the process of feature fusion by the detailed and context fusion module is:

[0035] The position-encoded detailed features and the long-range context features are concatenated into where $e$ pos represents the operation of position-encoding .

[0036] The detailed and context fusion module consists of layers of multi-head attention MSA and a fully connected block; finally, the feature fusion calculation method of the detailed features and the long-range context features is as follows:

[0037]

[0038] where LayerNorm represents layer normalization; represents $z$ iThe entire set, i.e., the output of the attention hidden layer; is the final output of semantic feature fusion.

[0039] Preferably, the processing process of the decoding fusion feature segmentation module is as follows:

[0040] Suppose the token features and context features in have been fully fused, select the features corresponding to the token positions, denoted as

[0041] Input these features into the decoder of TransUNet to predict the segmentation mask where k represents the predefined number of classes of the segmentation target, H is the image length, and W is the image width.

[0042] Preferably, the decoder of TransUNet upsamples the encoded features through transposed convolution operations, simultaneously obtains high-resolution feature maps from the corresponding layers of the encoder, and combines the feature maps with the upsampled features through skip-connection to improve the accuracy and quality of segmentation.

[0043] An image processing method for a fast semantic segmentation system of pneumonia DR images, the processing method is as follows:

[0044] When processing a DR image to be segmented, the context information aggregation module gradually collects and integrates long-range context features and detailed features outside the image patches based on the context graph neural network model in the end-to-end training mode;

[0045] Subsequently, based on the detail and context fusion module, effectively fuse the context information and detailed features of the image patches;

[0046] The decoding fusion feature segmentation module performs segmentation of the fusion features to generate an accurate segmentation mask.

[0047] The present invention discloses a fast semantic segmentation system and method for pneumonia DR images. When processing an image patch to be segmented, long-range and fine context information outside the image patch is gradually collected and integrated based on the graph neural network. Subsequently, based on the detail and context fusion module, the context information and detailed features of the image patch are effectively combined to generate an accurate segmentation mask, thereby realizing accurate segmentation and rapid evaluation of pneumonia lesions in DR images, providing strong assistance and support for doctors' diagnosis. The proposal of the present invention not only is expected to improve the accuracy and efficiency of pneumonia diagnosis, but also is expected to promote the further development of the field of medical image processing and intelligent diagnosis. Brief Description of the Drawings

[0048] Figure 1The overall graph neural network proposed for this invention.

[0049] Figure 2 Graph showing the influence of changing the number of graph convolution network layers on four evaluation metrics.

[0050] Figure 3 Graph showing the influence of changing the spatial pixel resolution on the evaluation metrics. Detailed implementation manners

[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0052] The present invention proposes a fast semantic segmentation system for pneumonia DR images, which performs semantic segmentation on pneumonia DR images based on a graph neural network, aiming to accurately outline the boundaries of pneumonia lesions at different pathological stages (similar to different maturation stages of TLS, but specifically referring to the early stage, progression stage, and stable / late stage of pneumonia lesions) from DR images, and achieve precise segmentation and identification of different development stages of pneumonia lesions in DR images.

[0053] During the semantic segmentation process, due to the high resolution of DR images and the complexity of lesions, a pneumonia lesion may be divided into multiple image patches for processing. Among these image patches, some contain key discriminatory information (such as specific internal structures or texture features of the lesion) that determines the development stage of the lesion. Therefore, for the image patches to be segmented (i.e., the target regions), the semantic segmentation system proposed by the present invention, based on the graph neural network model, can perceive and integrate the context information outside the target region to accurately capture and identify these key discriminatory features.

[0054] As Figure 1 shown, the graph neural network proposed by the present invention includes two key steps: First, using multi-layer convolutional network (GCN) technology, iteratively aggregate the long-distance and fine context information outside the target region. This step aims to construct a comprehensive context information library to provide a solid foundation for subsequent feature extraction and segmentation. Second, a detail and context information fusion module (DCFusion) is designed, which can effectively integrate the context information and detail features of the target region at the semantic level to generate an accurate segmentation mask.

[0055] Specifically, the graph neural network on which the semantic segmentation system of this invention is based includes the following four modules, namely a context graph construction module, a context information aggregation module, a detail and context information fusion module, and a decoding and fusion feature segmentation module.

[0056] (1) Context graph construction module

[0057] To model the context relationships of all image patches in the DR image, the present invention constructs a context graph G = (V, E) through a context graph construction module, where V represents the set of image patch features and E represents the set of undirected edges connecting the image patches.

[0058] Specifically, the present invention first filters the foreground of the DR image according to the edge detection method and divides it into non-overlapping image patches. The edge detection operator is applied to calculate the gradient of the image, the edge pixels of the image are found, and the edge pixels are connected into continuous edge curves through the edge connection algorithm. The foreground region is filled according to the edge curves, and the complete foreground region is extracted. Thus, a set P = {p i |i = 1...N} containing N image patches is obtained, where p i represents a single image patch. Secondly, each p i is encoded into a feature vector v i ∈R 1024 with a dimension of 1024 using UNI encoding. UNI is a computer pathology visual encoder based on Transformer, which has been pre-trained using self-supervised methods on millions of pathology images. Therefore, a DR image can be represented as a set V = {v i |i = 1...N} containing N nodes. Next, based on the spatial connectivity of the image patches, the present invention determines the set of undirected edges E = {v i v j |(i, j) ∈ M}, where v i and v j represent any two feature vectors encoded by p i , and M represents the set of nodes connected by 4-connectivity natural connection.

[0059] (2) Context Information Aggregation Module

[0060] The context information aggregation module is based on the context graph model and is used to model the features and context relationships of each image patch in the DR image;

[0061] Specifically, the adjacency matrix A = [a i,j n×n is derived from the connection relationship between the image patch nodes. The elements [a i,j of the adjacency matrix are defined as follows:

[0062]

[0063] where E represents the set of undirected edges of the image patches; v i , v j have the same meaning as in the previous set of undirected edges; the feature matrix represents the initial feature map of N nodes, each

[0064] The aggregation of context information beyond t steps can be expressed as:

[0065]

[0066] Where F GCN represents the graph convolutional neural network; δ represents the activation function; P represents the weight matrix; represents the normalized adjacency matrix, which is used to balance the number of neighbors of each node, D represents the degree matrix, and I represents the identity matrix.

[0067] The target node x i aggregates features from its neighbor nodes and gradually expands the scope of its context information. After T0 steps of aggregation, the feature of node i changes from to fusing context information from neighbors that are farther and farther away. Starting from the i-th node, the set of neighbors within t steps is denoted as Nera t (x i ) = {x j | d(i,j) = t}, where d(i,j) represents the shortest path length between the i-th node and the j-th node. Therefore, the feature of the i-th node is updated based on the union of all neighbor features within up to T0 steps, which can be expressed as:

[0068]

[0069] Where is the graph convolution operation, which updates the feature of node i by aggregating information from neighbors that are farther and farther away. This process enables the feature of the node to capture long-range context information and ultimately learn a richer representation of the target image patch. Therefore, multiple graph convolution aggregation steps enable v i to learn context information that is farther and farther away.

[0070] As Figure 1 shown, after obtaining the long-range context features i ∈ R H×W×3 of the image patch p this invention uses an encoder composed of a convolutional layer, an activation function, a max pooling layer, and feature doubling to extract detailed features Where represents the number of markers of the image patch p i , H is the image length, W is the image width, and l is the size used for segmenting the image patch.

[0071] (3) Detail and Context Information Fusion Module

[0072] The detail and context information fusion module is mainly used to fuse the long-range context features and detail features of the image patches;

[0073] Before inputting the detail features and the long-range context features into the detail and context fusion module for fusion, the detail features are first position-encoded. By adding the position encoding, the distinctiveness of the overall context in the detail features is enhanced.

[0074] Then, the position-encoded detail features and the long-range context features are concatenated into where e pos represents the operation of performing position encoding on .

[0075] The detail and context fusion module consists of layers of multi-head attention (MSA) and a fully connected block. The final detail features and the long-range context features are fused as follows:

[0076]

[0077] where LayerNorm represents layer normalization; represents the entire set of z i , i.e., the output of the attention hidden layer; is the final output of semantic feature fusion. MLP is a learnable fully connected layer where each layer is fully connected to the next layer. The output layer accepts the input data, and the hidden layers usually contain a certain number of neurons that perform linear transformation and non-linear activation on the input data, and then perform linear or non-linear transformation on the output of the last hidden layer to obtain the final output result.

[0078] (4) Decoding the fused feature segmentation module

[0079] After obtaining the fused feature i that fuses the detail features and context features of the image patch p , the invention assumes that the token features and context features in it have been fully fused. Therefore, the invention only selects the features corresponding to the token positions, denoted as Subsequently, these features are input into the decoder of TransUNet to predict the segmentation mask Among them, k represents the number of pre-defined categories for the segmentation target, H is the image length, and W is the image width. The decoder upsamples the encoded features through operations such as transposed convolution, and at the same time obtains high-resolution feature maps from the corresponding layers of the encoder, and combines these feature maps with the upsampled features through skip-connection to ensure that more detailed information is retained during the segmentation process and improve the accuracy and quality of the segmentation.

[0080] Finally, the cross-entropy is used to calculate the segmentation loss, and the network is optimized through backpropagation:

[0081]

[0082] The graph neural network proposed by this invention adopts an end-to-end training mode.

[0083] In summary, for the fast semantic segmentation system and method of pneumonia DR images proposed by this invention, different from traditional binary segmentation, this semantic segmentation method can accurately identify the boundaries and development stages of pneumonia lesions in DR images, which requires the system to integrate the context information in the image to extract discriminative features. Given the high resolution of DR images (for example, reaching the level of tens of thousands of pixels), the segmentation of pneumonia lesions usually adopts a patch-based strategy to improve computational efficiency. However, a major limitation of this method is that it restricts the model's access to information outside the patches, thus affecting the segmentation accuracy. To overcome this challenge, this invention is based on a context learning network of graph neural networks, and a system and method specifically designed for semantic segmentation of pneumonia lesions in DR images. When processing a patch to be segmented, this graph neural network first gradually collects and integrates long-distance and fine context information outside the patch. Subsequently, based on the detail and context fusion module, the context information and detail features of the patch are effectively combined to generate an accurate segmentation mask.

[0084] In addition, it should also be noted that for the processing of the pneumonia DR image dataset involved in the establishment of the semantic segmentation system of this invention, the following strategy is adopted:

[0085] First of all, the invention pays special attention to the CoronaHack-Chest X-Ray dataset, the Chest X-ray Imaging (Pneumonia) dataset, and the COVID-19CHEST X-RAY DATABASE dataset. For these three datasets, the invention carefully selected two adjacent DR images from each patient. One of the images maintains a standard grayscale display to provide basic imaging information; the other image has undergone specific image enhancement processing to more clearly reveal the lung structure and potential inflammatory areas. With the assistance of image enhancement technology, radiologists accurately marked the boundaries of pneumonia lesions in full-field digital imaging (WFD), and subdivided them into three stages according to the severity of the disease, providing an important basis for subsequent disease analysis and treatment.

[0086] In addition, the invention also constructed a collection called "Large Pneumonia DR Dataset". The DR images in this dataset are all from a tertiary hospital in Beijing and have undergone strict quality screening to ensure image quality. The invention excludes images containing motion artifacts, overexposure or blur to ensure the accuracy and reliability of the data. Without the assistance of image enhancement technology, radiologists annotated the severity of pneumonia lesions in the remaining images, dividing them into three levels: mild, moderate and severe, providing strong support for the assessment of pneumonia.

[0087] At the same time, the invention also utilizes two other DR image datasets closely related to pneumonia, respectively referred to as "Dataset A" and "Dataset B". These two datasets provide public annotation resources for lung areas and potential pneumonia lesions, but do not subdivide the severity of the lesions. In order to apply these datasets to DR image semantic segmentation, the invention performed preprocessing work. First, the invention excludes images that do not contain pneumonia lesions; secondly, based on the characteristics of the lesions (such as whether there is obvious consolidation or exudation), the invention divides pneumonia lesions into two categories: typical pneumonia lesions (TPL) and atypical pneumonia lesions (NTPL). In datasets A and B, the invention defines pneumonia lesions as two categories, TPL and NTPL, providing a clear classification standard for subsequent semantic segmentation tasks.

[0088] Finally, the invention integrates these four pneumonia DR image datasets for the semantic segmentation task of DR images. These four datasets are respectively responsible for two different segmentation tasks: four-class semantic segmentation (Seg4) and three-class semantic segmentation (Seg3). For the datasets that require four-class semantic segmentation (including the specially processed pneumonia DR image dataset, the large pneumonia DR dataset, and the eligible images in the CoronaHack-Chest X-Ray dataset), the invention divides the images into four categories: background (BG), mild pneumonia lesions (LPL), moderate pneumonia lesions (MPL), and typical pneumonia lesions (TPL). For the datasets A and B for three-class semantic segmentation (as well as the eligible images in the chest X-ray imaging (pneumonia) dataset and the COVID-19 CHEST X-RAY DATABASE dataset), the invention divides them into three categories: background (BG), atypical pneumonia lesions (NTPL), and typical pneumonia lesions (TPL). To ensure the reliability and effectiveness of the research, the invention randomly divides the data of each dataset into a training set, a validation set, and a test set according to the ratio of 6:2:2 for subsequent DR image semantic segmentation research and rapid assessment of pneumonia.

[0089] The performance of the model in three-class semantic segmentation and four-class semantic segmentation is shown in Table 1.

[0090] Table 1 Performance comparison of each model

[0091]

[0092]

[0093] As can be seen from the above table, the proposed graph neural network model of the invention has the highest prediction performance in all aspects. To further improve the prediction performance of the model, the invention has completed the following ablation study:

[0094] Figure 2Shows the results of ablation experiments that explored the impact of changing the number of graph convolutional network (aggregation layers) on its performance in the CoronaHack-Chest X-Ray dataset. Let Nc = (0, 1,..., 6) represent the number of graph convolutional network layers, where the baseline corresponds to Nc = 0. The number of graph convolutional network layers affects how far the contextual information of the target patch can propagate. The invention kept all other parameters unchanged, iteratively adjusted Nc, and compared the corresponding segmentation performance. The figure shows that the model performance is sensitive to the number of GCN layers. As the distance of aggregated information decreases, the segmentation performance drops significantly, and the performance is the worst when no contextual information is used. However, as the distance of aggregated information further increases, the performance will decrease slightly and then tend to be stable. Through multiple experiments with changing the number of GCN layers, the invention found that the model achieved the best results when the number of GCN layers was set to 3.

[0095] Figure 3 Shows the impact of information granularity on the CoronaHack-Chest X-Ray dataset. The invention set the number of GCN layers to 3, the patch size to 224×224, and the pixel spatial resolution to mpp = (0.5, 1, 2, 4, 8). The mF1 scores of the segmentation results are shown in the table. As the spatial resolution increases, the model becomes better at distinguishing the background, but the semantic segmentation performance for pneumonia lesions decreases significantly.

[0096] After determining the optimal resolution and the number of graph convolutional layers, the invention conducted experiments on the CoronaHack-Chest X-Ray dataset using a pixel resolution of 1.0μm / px and 3 graph convolutional layers for various fusion methods to integrate the detailed information and contextual information of the target. The invention adopted a variety of fusion methods, including: no contextual information, which does not incorporate contextual information and serves as a baseline for comparison. Among the fusion strategies, concatenation performed better than other basic strategies.

[0097] The above embodiments are not limitations of the present invention, and the present invention is not limited to the above examples either. Changes, modifications, additions, or substitutions made by those skilled in the art within the scope of the technical solution of the present invention also fall within the protection scope of the present invention.

Claims

1. A fast semantic segmentation system for pneumonia DR images, characterized in that: The fast semantic segmentation system includes a context graph construction module, a context information aggregation module, a detail and context information fusion module, and a decoded fusion feature segmentation module; The context graph construction module is used to construct a context graph through the context graph construction module; Based on the context graph neural network model, the context information aggregation module is used to model the detail features and context relationships of each image patch of the DR image; The detail and context information fusion module is used to fuse the long-range context features and detail features outside the target region of the image patch; After obtaining the fusion features that fuse the detail features and context features of the image patch, the decoded fusion feature segmentation module decodes the fusion features using a decoder and performs feature segmentation to generate an accurate segmentation mask.

2. The rapid semantic segmentation system for pneumonia DR images according to claim 1, characterized in that: The processing process of the context graph construction module is as follows: Construct a context graph G=(V, E), where V represents the set of image patch features and E represents the set of undirected edges connecting the image patches; Extract the complete foreground region from the DR image to obtain a set P = {p i | i = 1... N}, where p i represents a single image patch; Encode each image patch p using UNI encoding i into a feature vector v of dimension 1024 i ∈R 1024 ; A DR image is represented as a set V of N nodes = {v i | i = 1... N}; Based on the spatial connectivity of image patches, determine the image patch p i The set of undirected edges E = {v i v j | (i, j) ∈ M}, where v i and v j represent any two feature vectors encoded by the image patch p i and M represents the set of nodes connected by 4-connectivity naturally.

3. The rapid semantic segmentation system for pneumonia DR images according to claim 2, characterized in that: The extraction process of the complete foreground region is as follows: Filter the foreground of the DR image according to the edge detection method, and divide it into non-overlapping image patches. Apply the edge detection operator to calculate the gradient of the image, find the edge pixels of the image, connect the edge pixels into continuous edge curves through the edge connection algorithm, and fill the foreground region according to the edge curves to extract the complete foreground region.

4. The rapid semantic segmentation system for pneumonia DR images according to claim 1, wherein: The processing process of the context information aggregation module for modeling long-range context features is as follows: Adjacency matrix \(A = [a i,j _{n\times n}\) is derived from the connection relationship between image block nodes; the elements \([a i,j \) of the adjacency matrix are defined as follows: where E represents the set of undirected edges of the image block; v i , v j any two feature vectors encoded by the image block; Feature matrix represents the initial feature map of N nodes, each The aggregation of context information beyond step t can be expressed as: Among them, F GCN represents a graph convolutional neural network; δ represents an activation function; P represents a weight matrix; represents a normalized adjacency matrix, which is used to balance the number of neighbors of each node, D represents a degree matrix, and I represents an identity matrix; Target node x i Aggregate features from its neighbor nodes and gradually expand the scope of its context information; after T0 steps of aggregation, the feature of node i changes from to and incorporates context information from neighbors that are increasingly far away; Starting from the $i$-th node, the neighbor set within $t$ steps is denoted as $Nera$. t (x i ) = {x j | d(i, j) = t}, where d(i, j) represents the shortest path length between the $i$-th node and the $j$-th node; Therefore, the feature of the i-th node is updated based on the union of all neighbor features within up to step T0, expressed as: Among them, is a graph convolution operation that updates the features of node i by aggregating information from increasingly distant neighbors; this process enables the features of the node to capture long-range context information Finally, a richer representation of the target image patch is learned.

5. The rapid semantic segmentation system for pneumonia DR images according to claim 1, wherein: The processing process of the context information aggregation module for modeling detail features is as follows: The context information aggregation module uses an encoder composed of a convolutional layer, an activation function, a max pooling layer, and feature doubling to extract detailed features where represents the number of markers of the image patch p i H is the image length, W is the image width, and l is the size used for segmenting the image patches 6. The rapid semantic segmentation system for pneumonia DR images according to claim 1, wherein: Before inputting the detailed features and the long-distance context features into the detail-context fusion module for fusion, first perform positional encoding on the detailed features By adding positional encoding, the uniqueness of the overall context in the detailed features is enhanced.

7. The rapid semantic segmentation system for pneumonia DR images according to claim 6, wherein: The process of the detail and context fusion module for feature fusion is as follows: The detailed features after positional encoding and the long-range context features are concatenated into where, e pos represents the operation of performing positional encoding; The detail and context fusion module consists of l layers of multi-head self-attention (MSA) and a fully-connected block; the final detail features and the long-range context features are fused as follows: Among them, LayerNorm represents layer normalization; represents the entire set of z i i.e., the output of the attention hidden layer; is the final output of semantic feature fusion.

8. The rapid semantic segmentation system for pneumonia DR images according to claim 1, characterized in that: The processing process of the decoded fusion feature segmentation module is as follows: Assume the marked features and context features in These features are input into the decoder of TransUNet to predict the segmentation mask where k represents the number of predefined categories of the segmentation target, H is the image length, and W is the image width.

9. The rapid semantic segmentation system for pneumonia DR images according to claim 8, characterized in that: The decoder of the TransUNet upsamples the encoded features through transposed convolution operations, and at the same time obtains high-resolution feature maps from the corresponding layers of the encoder, and combines the feature maps with the upsampled features through skip-connection to improve the accuracy and quality of segmentation.

10. An image processing method for a rapid semantic segmentation system of pneumonia DR images according to any one of claims 1-9, characterized in that: The processing method is as follows: When processing a DR image to be segmented, the context information aggregation module gradually collects and integrates the long-range context features and detail features outside the image patches based on the context graph neural network model in an end-to-end training mode; Subsequently, based on the detail and context fusion module, the context information and detail features of the image patches are effectively fused; The decoded fusion feature segmentation module performs segmentation of the fusion features to generate an accurate segmentation mask.

Images