Parallel detection method and system for multiple types of retinal diseases

Abstract

The invention relates to the technical field of retinal disease detection, in particular to a parallel detection method and system for multiple types of retinal diseases. According to the method, a binary classifier of a Faster R-CNN model is replaced by a multi-label classifier of which the output dimension is the total number of retinal disease types, and a single-slice prediction module is constructed; for each lesion area bounding box of each OCT slice, calculating the intersection-union ratio of the lesion area bounding box of each OCT slice and the same kind of disease bounding box in the adjacent slice, and constructing a cross-slice lesion area bounding box set; performing weighted average on the probability distribution of the multiple disease types in the set to obtain the probability distribution of the multiple disease types of a target; and expanding the lesion area based on the bounding box coordinates in the set to obtain a bounding box of the target lesion area. In addition, a supervision domain adaptive framework BV-DA-CNN is designed to train the model, and the cross-device disease prediction capability is enhanced. Through multi-label classification and cross-slice information integration, accurate detection of multiple retinal diseases and complete identification of focus areas are realized, and the accuracy and reliability of detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of retinal disease detection, and in particular to a method and system for parallel detection of multiple types of retinal diseases. Background Art

[0002] Retinal diseases are one of the leading causes of visual impairment and blindness worldwide. Early diagnosis and precise treatment are key to preventing irreversible visual impairment. With the advancement of medical imaging technology, color fundus photography (CFP) and optical coherence tomography (OCT) have become core tools for the clinical diagnosis of retinal diseases. In the early days, due to the complexity of interpreting OCT images, most research focused on CFP image analysis. However, CFP images can only provide two-dimensional retinal surface information and are not sensitive enough to pathological changes in deep structures. In contrast, OCT technology can reveal the microstructural characteristics of each layer of the retina through three-dimensional imaging, providing richer morphological basis for disease diagnosis. It can clearly show the location and morphological changes of choroidal neovascularization (CNV) and drusen in AMD, providing richer structural information for disease diagnosis.

[0003] Currently, research on retinal disease detection mainly focuses on the classification task of a single or a small number of diseases. Early detection methods based on OCT images, by identifying three pathological features such as choroidal neovascularization and diabetic macular edema, can only distinguish between health and a single disease state, and cannot meet the complex needs of clinical diagnosis. Later, some studies used three-dimensional OCT datasets to build a referral recommendation system. Although it can cover a variety of retinal diseases, the coverage of diseases is still limited to less than 7 categories, making it difficult to fully respond to diverse retinal pathological conditions. Another study proposed a detection system that can support four-stage referral recommendations for 7 diseases. However, its disease coverage is still limited. In actual clinical scenarios, when faced with more types of retinal diseases, it shows obvious limitations.

[0004] While these studies have demonstrated the feasibility of AI technology in OCT image analysis, practical applications present numerous drawbacks. Existing studies typically employ a single-slice independent analysis model, performing lesion detection and classification independently on each OCT slice. This is because 3D OCT volumetric data consists of hundreds of 2D slices. Manually labeling lesion location and type on each slice consumes considerable time and effort, requiring a high level of expertise from the annotators, making it difficult to implement on a large scale. Furthermore, developing 3D algorithms is challenging, making single-slice analysis a more practical option. However, this single-slice approach completely ignores the spatial correlation between adjacent slices, significantly compromising the consistency of lesion detection. Relying solely on single-slice analysis can easily lead to missed lesions. For example, lesions that are not readily apparent on a single slice but are characteristic in consecutive slices may be missed. It can also lead to boundary misjudgment; the lack of reference from adjacent slices makes it difficult to accurately define the actual boundaries of lesions. Furthermore, it is susceptible to false positives, where nonspecific signals present on a single slice may be misclassified as lesions, reducing the accuracy of retinal disease detection. Furthermore, existing systems are extremely limited in their ability to detect multiple diseases simultaneously. In clinical practice, patients often suffer from multiple retinal diseases at the same time, and most existing systems perform multi-disease detection within a limited range of diseases. They are still unable to effectively deal with multiple disease combinations beyond the coverage range, or more complex coexistence of multiple diseases. In essence, they have not yet gotten rid of the limitations of single-disease design thinking and cannot meet multi-task diagnosis needs. This has greatly limited the widespread application of retinal disease detection systems in clinical practice. Summary of the Invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defects of existing OCT image-based retinal disease detection methods that mostly adopt a single-slice independent analysis mode, ignore the spatial correlation between adjacent slices, resulting in poor retinal disease detection accuracy, and are limited to detection of a limited range of diseases, and cannot meet the diagnostic needs of multiple diseases in clinical practice.

[0006] To solve the above technical problems, the present invention provides a method for parallel detection of multiple types of retinal diseases, comprising:

[0007] Each OCT slice image of the patient to be tested is input into the single-slice prediction module of the multi-class retinal disease parallel detection model. The coordinates of the lesion region bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types are output, and the disease type to which each lesion region bounding box belongs is determined. The binary classifier in the Faster R-CNN model is replaced with a multi-label classifier to construct the single-slice prediction module. The output dimension of the multi-label classifier is the total number of retinal disease types.

[0008] For each lesion bounding box in each OCT slice image, the intersection-and-union ratio (IoU) of the lesion bounding box with the same disease type in the adjacent slices of the slice is calculated. The lesion bounding boxes with an IoU ratio greater than a preset threshold are retained as the same-type bounding boxes of the lesion bounding box. Each lesion bounding box and its same-type bounding boxes are collectively regarded as a cross-slice lesion bounding box set.

[0009] Expanding each lesion region bounding box based on the bounding box coordinates of the cross-slice lesion region bounding box set corresponding to each lesion region bounding box to obtain the coordinates of each target lesion region bounding box;

[0010] The mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box is used as the target multi-disease type probability distribution of each target lesion area bounding box;

[0011] The coordinates of the bounding boxes of each target lesion area in each OCT slice image and the probability distribution of target multi-disease types are used as the output of the multi-class retinal disease parallel detection model.

[0012] The method for parallel detection of multiple retinal diseases according to claim 1, wherein the adjacent slices of each OCT slice image include:

[0013] Arrange N OCT slices into a sequence in order, and the entire sequence is in a cyclic state; extract the k slices before and after each OCT slice image in the sequence as the adjacent slices of each OCT slice image; where N is the total number of OCT slice images and k is a hyperparameter.

[0014] Preferably, based on the bounding box coordinates of the cross-slice lesion area bounding box set corresponding to each lesion area bounding box, each lesion area bounding box is expanded to obtain the coordinates of the target lesion area bounding box of each lesion area, including:

[0015] For each lesion region bounding box, the minimum horizontal coordinate in the corresponding set of cross-slice lesion region bounding boxes is used as the horizontal coordinate of the lower left corner and the upper left corner of the lesion region bounding box;

[0016] The maximum horizontal coordinate in the corresponding cross-slice lesion region bounding box set is used as the horizontal coordinate of the lower right corner and the upper right corner of the lesion region bounding box;

[0017] The minimum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the vertical coordinate of the lower left corner and the lower right corner of the lesion region bounding box;

[0018] The maximum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the upper left corner vertical coordinate and the upper right corner vertical coordinate of the lesion region bounding box.

[0019] Preferably, the step of inputting each OCT slice image of the patient to be tested into the single slice prediction module of the multi-class retinal disease parallel detection model and outputting the coordinates of the lesion area bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types includes:

[0020] Each OCT slice image is passed through a feature extraction module to extract a feature map of each OCT slice image;

[0021] The feature map of each OCT slice image is passed through the candidate region generation network to generate the coordinates of the candidate box of the lesion area in the feature map of each OCT slice image;

[0022] The feature map of each OCT slice image and the coordinates of the candidate box of the lesion area are input into the bounding box regression module, the coordinates of the candidate box of the lesion area are adjusted, and the coordinates of the bounding box of the lesion area of ​​each OCT slice image are output;

[0023] The coordinates of the lesion region bounding box of each OCT slice image are mapped to the feature map of the OCT slice image, and the lesion feature vector of each lesion region bounding box of each OCT slice image is extracted through the region of interest pooling module;

[0024] The lesion feature vector of each lesion area bounding box of each OCT slice image is passed through a multi-label classifier to obtain the probability distribution of multiple disease types corresponding to the lesion area bounding box of each OCT slice image.

[0025] Preferably, the multi-label classifier includes:

[0026] Multiple parallel fully connected branches, each fully connected branch includes a fully connected layer and a Sigmoid function connected in sequence; wherein the number of fully connected branches is the total number of retinal disease types.

[0027] Preferably, the lesion feature vector of each lesion area bounding box of each OCT slice image is passed through a multi-label classifier to obtain a probability distribution of multiple disease types corresponding to the lesion area bounding box of each OCT slice image. The calculation formula for the probability distribution of multiple disease types is:

[0028]

[0029] Among them, P(y c,i =1|x i ) represents the probability that the i-th OCT slice image has the c-th retinal disease, x i is the i-th OCT slice image, φ(x i) is the lesion feature vector of each lesion area bounding box in the i-th OCT slice image, y c,i Indicates whether the i-th slice has the c-th retinal disease, v c The transpose of v c is the weight vector of the cth fully connected branch, e is a natural constant, C is the total number of retinal disease types, c is the retinal disease type index, and i is the OCT slice image index.

[0030] Preferably, the training method of the multi-class retinal disease parallel detection model is:

[0031] Obtaining a target domain device dataset, the target domain device dataset including: all OCT slice images of different patients, coordinates of a true lesion region bounding box of each OCT slice image, and corresponding true multi-disease type probability distribution;

[0032] All OCT slice images of the patient are input into the multi-class retinal disease parallel detection model, and the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of the target multi-disease type are output;

[0033] A multi-label classification loss is constructed based on the difference between the target multi-disease type probability distribution and the true multi-disease type probability distribution of each lesion area bounding box in all OCT slice images;

[0034] Based on the intersection-over-union ratio of the bounding box of each lesion area in all OCT slice images and the bounding box of the true lesion area, a bounding box tightness constraint loss is constructed;

[0035] Based on multi-label classification loss and bounding box tightness constraint loss, a total loss function is constructed; the total loss function is used to train a parallel detection model for multiple categories of retinal diseases to obtain the target parallel detection model for multiple categories of retinal diseases.

[0036] Preferably, the bounding box tightness constraint loss is:

[0037]

[0038] Among them, L ins is the bounding box tightness constraint loss, N is the total number of OCT slice images, i is the OCT slice image index, is the lesion area bounding box of the i-th OCT slice image and the actual lesion area bounding box IoU(·) is the intersection-over-union ratio, is the lesion area bounding box of the i-th OCT slice image, is the true lesion area bounding box of the i-th OCT slice image ‖·‖2 is the Euclidean norm.

[0039] Preferably, the total loss function is:

[0040] L=L multi-label +λL ins ,

[0041] Among them, L is the total loss function, L multi-label is the multi-label classification loss, L ins is the bounding box tightness constraint loss, and λ is the bounding box tightness constraint loss weight.

[0042] A multi-type retinal disease parallel detection system, comprising:

[0043] A single-slice prediction module is used to input each OCT slice image of the patient to be tested into the single-slice prediction module of the multi-class retinal disease parallel detection model, output the coordinates of the lesion region bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types, and determine the disease type to which each lesion region bounding box belongs. The single-slice prediction module is constructed by replacing the binary classifier in the Faster R-CNN model with a multi-label classifier; the output dimension of the multi-label classifier is the total number of retinal disease types;

[0044] A set construction module is configured to calculate, for each lesion region bounding box in each OCT slice image, the intersection-and-union ratio of the lesion region bounding box with the same disease type in adjacent slices of the slice, retaining the lesion region bounding box with an intersection-and-union ratio greater than a preset threshold as the same type of bounding box of the lesion region bounding box; and combining each lesion region bounding box with its same type of bounding box as a cross-slice lesion region bounding box set;

[0045] an expansion module, configured to expand each lesion region bounding box based on the bounding box coordinates of the set of cross-slice lesion region bounding boxes corresponding to each lesion region bounding box, to obtain the coordinates of each target lesion region bounding box;

[0046] A probability prediction module is used to calculate the mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box as the target multi-disease type probability distribution of each target lesion area bounding box;

[0047] The output module is used to output the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of its target multi-disease type as the output of the multi-class retinal disease parallel detection model.

[0048] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0049] The present invention describes a method and system for parallel detection of multiple types of retinal diseases. The present invention replaces the binary classifier in the Faster R-CNN model with a multi-label classifier whose output dimension is the total number of retinal disease types, and constructs a single-slice prediction module to achieve the distinction between multiple disease types in OCT slices. For each lesion area bounding box in each OCT slice image, the intersection-and-union ratio of the lesion area bounding box with the same disease type in the adjacent slices of the slice is calculated. This can accurately determine whether the lesion spans multiple slices. When the same type of lesion exists in adjacent slices and the spatial position is continuous, it can be determined as a cross-slice lesion. This fully utilizes the three-dimensional spatial correlation of the OCT image and effectively avoids the problem of missed lesions due to the lack of significant lesion features in a single slice during single slice analysis. At the same time, it can also filter out isolated noise or artifacts through cross-slice consistency analysis, significantly reducing false positive interference. Based on the set of cross-slice lesion area bounding boxes, the target lesion area bounding box is expanded to completely cover all lesion area bounding boxes in the set, thereby accurately defining the actual range of the lesion. By taking the average of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box as the target multi-disease type probability distribution, and combining the single slice prediction results with the contextual information of adjacent slices, the consistency and accuracy of cross-slice lesion detection are improved, thereby improving the reliability and accuracy of retinal disease detection.

[0050] In addition, the present invention adopts a fully supervised domain adaptation mechanism, directly using the full amount of labeled data of the target domain device for end-to-end fine-tuning, replacing the traditional semi-supervised strategy, eliminating the noise interference caused by unlabeled data, enhancing the model's adaptability to the target domain data, and effectively improving the retinal disease prediction ability. Due to the use of fully supervised training, the consistency constraint item in the traditional semi-supervised strategy is removed, and a bounding box tightness constraint loss is designed. This loss calculates the Euclidean norm mean of the intersection over union (IoU) of the predicted lesion area bounding box and the true bounding box. By constraining the spatial overlap between the predicted box and the true box, it can accurately reflect the actual coverage of the lesion area by the bounding box. The introduction of the Euclidean norm strengthens the penalty for small deviations, prompting the model to continuously optimize the coordinate parameters of the prediction box. It effectively improves the positioning accuracy of the lesion area, enables the model to more accurately define the spatial range of the lesion, and improves the accuracy of retinal disease detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0052] Figure 1 It is a flow chart of a method for parallel detection of multiple types of retinal diseases according to the present invention.

[0053] Figure 2 This is a structural diagram of the parallel detection model for multiple types of retinal diseases.

[0054] Figure 3 It is the structural diagram of the domain adaptation strategy BV-DA-CNN.

[0055] Figure 4 It is the annotation of different categories of retinal diseases, including the bounding box of the real lesion area and its corresponding real multi-disease type probability distribution, Figure 4 (a) in the figure is the pigment epithelial detachment (PED) mark. Figure 4 (b) in the figure is the mark of posterior vitreous detachment (PVD). Figure 4 (c) in the figure is the epiretinal membrane (ERM). Figure 4 (d) in the figure is the subretinal fluid (SRF) mark. Figure 4 (e) in the figure is the choroidal neovascularization (CNV) mark. Figure 4 The (f) in the figure is the mark for drusen. Figure 4 (g) in the figure is marked as Retinoschisis. Figure 4 (h) in the figure is the mark of cystoid macular edema (CME). Figure 4 (i) in the figure is hard exudate (HE). Figure 4 (j) in the figure is the mark of full-thickness macular hole (FTMH). Figure 4 (k) in the figure is the retinal detachment (RD) mark. Figure 4 (l) in the figure is the mark of internal segment / external segment fracture (IS / OS). Figure 4 The (m) in the figure is the non-macular epiretinal membrane (ERM (Non-Macular)). Figure 4 The (n) in the figure is the lamellar macular hole (LMH).

[0056] Figure 5 This is a comparison diagram of the intersection-over-union ratio between the target lesion area bounding box predicted by the multi-class retinal disease parallel detection model and the actual target lesion area bounding box.

[0057] Figure 6 This is a schematic diagram of the multi-label classification results of retinal diseases using a parallel detection model for multiple retinal diseases. Figure 6 (a) in the figure shows the accuracy result. Figure 6 (b) in the figure shows the precision result. Figure 6 (c) in the figure shows the F1 score result. Figure 6 (d) in the figure is the false positive rate (FPR) result. Figure 6(e) in the figure is the false negative rate (FNR) result. Figure 6 (f) in the figure is a schematic diagram of the Kappa coefficient results.

[0058] Figure 7 This is a structural comparison diagram of different domain adaptation strategies.

[0059] Figure 8 This is a schematic diagram comparing ROC curves of multiple retinal disease parallel detection models under different domain adaptation strategies.

[0060] Figure 9 It is a flow chart of a device for parallel detection of multiple types of retinal diseases. DETAILED DESCRIPTION

[0061] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0062] Although Faster R-CNN (Fast Region-based Convolutional Neural Network) has performed well as a general object detection framework in natural image recognition, it cannot be directly applied to disease detection in OCT images. The main reasons include: medical images are specific. The unique tissue texture features of OCT images (such as retinal layering and lesion morphology) differ significantly from those of natural images. Disease signs (such as drusen and subretinal fluid) exhibit unique grayscale distributions and spatial relationships in OCT, while general object detection models lack dedicated feature encoding mechanisms for medical images. Collaborative detection of multiple lesions is challenging. OCT images often contain multiple related lesions (for example, AMD patients may have drusen, geographic atrophy, and choroidal neovascularization). However, Faster R-CNN's binary classifier (foreground / background) cannot distinguish specific disease types and lacks the ability to model pathological associations between lesions. Cross-domain adaptability is insufficient. Images generated by different OCT devices (frequency domain / time domain) and scanning protocols vary significantly. The distribution difference between the training set and actual clinical data (domain shift) leads to a sharp decline in model performance on data from new devices.

[0063] Therefore, based on Faster R-CNN, the present invention replaces the binary classifier of Faster R-CNN with an instance-level multi-label classifier, designs a single-slice prediction module, integrates the contextual relationship between adjacent slices, and designs a parallel detection model for multiple types of retinal diseases.

[0064] In addition, considering that traditional parallel detection methods for multiple types of retinal diseases often adopt a single-slice independent analysis mode, that is, lesion detection and classification are performed separately on each OCT slice, but since retinal lesions (such as CNV, retinal detachment, etc.) are mostly distributed continuously in three-dimensional space, relying solely on single-slice analysis will cause many problems: First, it may lead to missed lesions, because the lesions may not be obvious in a single slice, but have obvious characteristics in adjacent slices; second, it will cause boundary misjudgment, and the starting and ending positions of the lesion need to be accurately defined by combining the information of the upper and lower slices, such as the range of subretinal fluid; third, it is susceptible to false positive interference, and isolated noise or artifacts may be misjudged as lesions, while cross-slice consistency analysis can improve the robustness of detection; fourth, it will cause spatial information loss. OCT images are essentially three-dimensional data. Single-slice processing ignores the spatial correlation between adjacent slices, reducing the coherence of lesion detection; fifth, it leads to low diagnostic efficiency. The single-slice independent processing mode requires analysis of each slice, and cannot fully utilize the contextual information between slices, which in turn affects the overall diagnostic efficiency.

[0065] To solve the above problems, refer to Figure 1 As shown, this embodiment 1 provides a method for parallel detection of multiple types of retinal diseases, including the following steps:

[0066] like Figure 2 As shown, Figure 2 This is a structural diagram of the parallel detection model for multiple retinal diseases.

[0067] Step S1: Input each OCT slice image of the patient to be tested into the single slice prediction module of the multi-class retinal disease parallel detection model, and output the coordinates of the lesion area bounding box of each OCT slice image The corresponding probability distribution of multiple disease types is obtained, and the disease type to which each lesion area bounding box belongs is determined. The binary classifier in the Faster R-CNN model is replaced with a multi-label classifier to construct a single-slice prediction module. The output dimension of the multi-label classifier is the total number of retinal disease types.

[0068] Among them, for the i-th OCT slice image, it contains N i The lesion area bounding box, B i is the coordinate of the lesion area bounding box of the i-th OCT slice image, B i,j is the bounding box of the jth lesion area in the i-th OCT slice image, B i,j =(x j,min ,y j,min ,x j,max ,y j,max ), each lesion area bounding box B i,j Corresponding to a multi-disease type probability distribution P i,j =(p i,j,1,p i,j,2 ,…,p i,j,C ), C is the total number of retinal disease types, P i,j B is the bounding box of the lesion area i,j The corresponding probability distribution of multiple disease types, P i,j,c is the probability that the i-th OCT slice image has the c-th retinal disease, P i,j,c =P(y c,i =1|x i ), c is the retinal disease type index, i is the OCT slice image index, j = 1, 2...N i , x j,min y is the lower left corner abscissa / upper left corner abscissa of the jth lesion area bounding box of the i-th OCT slice image, j,min is the lower right corner abscissa / lower left corner abscissa of the jth lesion area bounding box of the i-th OCT slice image, x j,max y is the lower right corner abscissa / upper right corner abscissa of the jth lesion area bounding box of the i-th OCT slice image, j,max is the upper left corner abscissa / upper right corner abscissa of the jth lesion area bounding box in the i-th OCT slice image;

[0069] In this embodiment, preferably, the multi-label classifier includes:

[0070] Multiple parallel fully connected branches, each fully connected branch includes a fully connected layer and a Sigmoid function connected in sequence; wherein the number of fully connected branches is the total number of retinal disease types.

[0071] In this embodiment, specifically, each OCT slice image of the patient to be tested is input into the single slice prediction module of the multi-class retinal disease parallel detection model, and the coordinates of the lesion area bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types are output, including:

[0072] Each OCT slice image is passed through a feature extraction module to extract a feature map of each OCT slice image; in this embodiment, the feature extraction module uses a ResNet-101 network.

[0073] The feature map of each OCT slice image is passed through the Region Proposal Network (RPN) to generate the coordinates of the candidate box of the lesion region in the feature map of each OCT slice image;

[0074] The feature map of each OCT slice image and the coordinates of the candidate box of the lesion area are input into the bounding box regression module (Bounding Box Regression), the coordinates of the candidate box of the lesion area are adjusted, and the coordinates of the bounding box of the lesion area of ​​each OCT slice image are output;

[0075] The coordinates of the lesion region bounding box of each OCT slice image are mapped to the feature map of the OCT slice image, and the lesion feature vector of each lesion region bounding box of each OCT slice image is extracted through the region of interest pooling module (Roi Pooling);

[0076] The lesion feature vector of each lesion area bounding box of each OCT slice image is passed through a multi-label classifier to obtain the probability distribution of multiple disease types corresponding to the lesion area bounding box of each OCT slice image.

[0077] In this embodiment, specifically, the lesion feature vector of each lesion area bounding box of each OCT slice image is passed through a multi-label classifier to obtain the probability distribution of multiple disease types corresponding to the lesion area bounding box of each OCT slice image. The calculation formula for the probability distribution of multiple disease types is:

[0078]

[0079] Among them, P(y c,i =1|x i ) represents the probability that the i-th OCT slice image has the c-th retinal disease, x i is the i-th OCT slice image, φ(x i ) is the lesion feature vector of each lesion area bounding box in the i-th OCT slice image, y c,i Indicates whether the i-th slice has the c-th retinal disease, v c The transpose of v c is the weight vector of the cth fully connected branch, e is a natural constant, C is the total number of retinal disease types, c is the retinal disease type index, and i is the OCT slice image index.

[0080] The present invention uses Faster R-CNN as the backbone network of the single-slice prediction module, and realizes the positioning and disease classification functions of retinal abnormal areas through the integrated design of structured feature extraction, candidate region generation, and bounding box regression. Specifically, by adopting a feature extraction module with a residual connection architecture, the gradient degradation problem in deep network training can be effectively alleviated, thereby improving the feature characterization capability. The feature map of the extracted OCT slice image is input into the candidate region generation network (Region Proposal Network, RPN) to generate a candidate box for the potential abnormal area; then the candidate box coordinates are optimized through the bounding box regression module, and the disease type of the tissue in the candidate box is classified in combination with the multi-label classifier. Through a multi-task collaborative training mechanism, the accurate positioning of abnormal areas and the efficient discrimination of disease types are ensured.

[0081] In this embodiment, specifically, determining the disease type to which each lesion region bounding box belongs includes:

[0082] The disease type with the highest probability in each lesion region bounding box is taken as the disease type to which the lesion region bounding box belongs.

[0083] Step S2: For each lesion region bounding box in each OCT slice image, calculate the intersection over union (IoU) between the lesion region bounding box of the same disease type in the adjacent slices of the slice and the lesion region bounding box, retain the lesion region bounding box with an IoU greater than a preset threshold as the same type of bounding box of the lesion region bounding box; each lesion region bounding box and its same type of bounding box are collectively regarded as a cross-slice lesion region bounding box set; in this embodiment, the preset threshold is 0.5;

[0084] In this embodiment, preferably, the adjacent slices of each OCT slice image include:

[0085] Arrange N OCT slices into a sequence in order, and the entire sequence is in a cyclic state; extract the k slices before and after each OCT slice image in the sequence as the adjacent slices of each OCT slice image; where N is the total number of OCT slice images and k is a hyperparameter.

[0086] If the current slice is at the beginning of the sequence and there are less than k slices before it, then the corresponding number of slices is added from the end of the sequence forward; if the current slice is at the end of the sequence and there are less than k slices after it, then the corresponding number of slices is added from the beginning of the sequence backward; if the current slice is in the middle of the sequence, then the k slices before and after it are directly taken, that is, the N OCT slices are regarded as a first-connected circular sequence. Starting from any slice position, N slices are counted forward and backward in the circular sequence. These slices are the adjacent slices of the OCT slice.

[0087] In this embodiment, specifically, for each lesion region bounding box of each OCT slice image, the intersection-over-union ratio of the lesion region bounding box of the same disease type in the adjacent slices of the slice and the lesion region bounding box is calculated, and the formula is:

[0088]

[0089] Among them, B i,j is the bounding box of the jth lesion area in the i-th OCT slice image, B i′,j′ is the bounding box of the j′th lesion area in the adjacent slice of the i-th OCT slice image, IoU(·) is the intersection over union ratio, Area(B i,j ∩B i′,j′ ) is B i,j With B i′,j′ The intersection area, Area(B i,j∪B i′,j′ ) is B i,j With B i′,j′ The union area of ​​.

[0090] Step S3: based on the bounding box coordinates of the cross-slice lesion region bounding box set corresponding to each lesion region bounding box, each lesion region bounding box is expanded to obtain the coordinates of each target lesion region bounding box;

[0091] In this embodiment, preferably, based on the bounding box coordinates of the cross-slice lesion area bounding box set corresponding to each lesion area bounding box, each lesion area bounding box is expanded to obtain the coordinates of the target lesion area bounding box of each lesion area, including:

[0092] For each lesion region bounding box, the minimum horizontal coordinate in the corresponding set of cross-slice lesion region bounding boxes is used as the horizontal coordinate of the lower left corner and the upper left corner of the lesion region bounding box;

[0093] The maximum horizontal coordinate in the corresponding cross-slice lesion region bounding box set is used as the horizontal coordinate of the lower right corner and the upper right corner of the lesion region bounding box;

[0094] The minimum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the vertical coordinate of the lower left corner and the lower right corner of the lesion region bounding box;

[0095] The maximum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the upper left corner vertical coordinate and the upper right corner vertical coordinate of the lesion region bounding box.

[0096] Step S4: The mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box is used as the target multi-disease type probability distribution of each target lesion area bounding box.

[0097] For the jth lesion area bounding box B in the i-th OCT slice image i,j , whose target multi-disease type probability distribution Defined as the set of cross-slice lesion area bounding boxes corresponding to the lesion area bounding box The mean of the probability distribution of multiple disease types, is the probability that the cth retinal disease exists in the i-th OCT slice image, and the calculation formula is:

[0098]

[0099] in, is the bounding box B of the jth lesion area in the i-th OCT slice image i,j The corresponding set of bounding boxes of the lesion area across slices The total number of elements, (i′, j′) is the bounding box B of the jth lesion area in the i-th OCT slice image i,j The corresponding set of bounding boxes of the lesion area across slices The elements in p represent the index of the same bounding box in the adjacent slice, p i′,j′,c is the probability that the j′th bounding box of the i′th slice in the adjacent slices belongs to the cth disease category.

[0100] Step S5: The coordinates of the bounding boxes of each target lesion area in each OCT slice image and the probability distribution of the target multiple disease types are used as the output of the multi-class retinal disease parallel detection model.

[0101] Considering that if OCT images are directly used to train a parallel detection model for multiple types of retinal diseases, the images produced by different OCT devices (frequency domain / time domain) and scanning protocols are significantly different, and there are distribution differences between the training set and the actual clinical data, the trained model will have poor generalization and its performance will drop sharply on new device data.

[0102] Traditional domain adaptation methods (such as DA-CNN) typically rely on unsupervised or semi-supervised learning, using unlabeled data from the target domain to align feature distributions. However, in the field of ophthalmic medical imaging, a small amount of labeled data from the target domain (such as new OCT devices) is often available, but existing methods fail to fully utilize this labeled information. The domain adaptation process only optimizes feature alignment without explicitly enhancing disease classification performance, resulting in insufficient retinal disease prediction capabilities. Common domain adaptation frameworks (such as DomainAdaptive FasterR-CNN) use a binary classifier (foreground / background) by default, which cannot adapt to the multi-label classification requirements of retinal diseases, resulting in limited detection of multiple diseases.

[0103] like Figure 3 As shown, Figure 3 It is the structural diagram of the domain adaptation strategy BV-DA-CNN.

[0104] To address the above shortcomings, the present invention designs a domain adaptation strategy BV-DA-CNN, which is as follows:

[0105] In this embodiment, preferably, the training method of the multi-type retinal disease parallel detection model is:

[0106] Obtaining a target domain device dataset, the target domain device dataset including: all OCT slice images of different patients, coordinates of a true lesion region bounding box of each OCT slice image, and corresponding true multi-disease type probability distribution;

[0107] All OCT slice images of the patient are input into the multi-class retinal disease parallel detection model, and the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of the target multi-disease type are output;

[0108] A multi-label classification loss is constructed based on the difference between the target multi-disease type probability distribution and the true multi-disease type probability distribution of each lesion area bounding box in all OCT slice images;

[0109] Based on the intersection-over-union ratio of the bounding box of each lesion area in all OCT slice images and the bounding box of the true lesion area, a bounding box tightness constraint loss is constructed;

[0110] In this embodiment, preferably,

[0111]

[0112] Among them, L multi-label is the multi-label classification loss, which is used to measure the difference between the model prediction probability and the true label, y i,j,c is the true probability that the j-th lesion area bounding box of the i-th OCT slice image has the c-th retinal disease, is the probability that the i-th OCT slice image has the c-th retinal disease, is the model prediction value, and w c is the category weight of the cth retinal disease, which is used to balance the sample imbalance problem.

[0113] The bounding box tightness constraint loss is:

[0114]

[0115] Among them, L ins is the bounding box tightness constraint loss, N is the total number of OCT slice images, i is the OCT slice image index, is the lesion area bounding box of the i-th OCT slice image and the actual lesion area bounding box IoU(·) is the intersection-over-union ratio, is the lesion area bounding box of the i-th OCT slice image, is the true lesion area bounding box of the i-th OCT slice image ‖·‖2 is the Euclidean norm.

[0116] The total loss function is:

[0117] L=L multi-label +λL ins ,

[0118] Among them, L is the total loss function, L multi-label is the multi-label classification loss, Lins is the bounding box tightness constraint loss, and λ is the bounding box tightness constraint loss weight.

[0119] Based on multi-label classification loss and bounding box tightness constraint loss, a total loss function is constructed; the total loss function is used to train a parallel detection model for multiple categories of retinal diseases to obtain the target parallel detection model for multiple categories of retinal diseases.

[0120] Existing research often relies on datasets collected by a single device. However, differences in imaging parameters between different OCT scanners can cause image feature distributions to shift, significantly degrading system performance during cross-device testing. For example, some studies use data derived only from specific scanners, which directly leads to significantly reduced model performance when applied across devices. Although some studies have attempted to alleviate this problem through transfer learning, there is currently a lack of effective validation of domain adaptation methods for multi-disease detection.

[0121] The present invention breaks the limitation of single pathological feature extraction by constructing a multi-disease, multi-center OCT dataset, combining the lesion area detection model with a multi-label classifier, and realizes the simultaneous identification and spatial positioning of 14 types of retinal diseases. Specifically, the present invention introduces a domain adaptation algorithm. The present invention directly uses the full amount of labeled data (target domain device dataset) of the target domain device (such as BV1000) for end-to-end fine-tuning, replacing the traditional semi-supervised strategy (DA-CNN), eliminating the noise interference caused by unlabeled data, and effectively reducing the feature distribution differences between different scanners, significantly reducing the dependence on specific equipment during clinical deployment. At the same time, the present invention proposes a cloud-based collaborative computing architecture, which integrates the spatial information of adjacent slices through slice-level parallel reasoning, improves the coherence of cross-slice lesion detection, and reduces the risk of misdiagnosis due to missed detection of local slices.

[0122] In this embodiment, three core data sets were constructed for model training, testing and domain adaptation research. The training and test data sets were collected by two OCT scanners, Optovue and Heidelberg, respectively, covering clinical data from six centers, with a total of 14 types of retinal diseases annotated, including Pigment Epithelium Detachment (PED), Posterior Vitreous Detachmen (PVD), Epiretinal Membrane (ERM), Subretinal Fluid (SRF), Choroidal Neovascularization (CNV), Drusen, Retinoschisis, Cystoid Macular Edema (CME), Hard Exudates (HE), Full-Thickness Macular Hole (FTMH), Retinal Detachment (RD), Inner Segment / Outer Segment Break (Inner Segment / Outer Segment Break) Disruption, IS / OS), Epiretinal Membrane in Non-Macular Areas (ERM (Non-Macular)) and Lamellar Macular Hole (LMH).

[0123] The target domain device dataset is collected by a domestic BV1000 OCT scanner, and the annotated disease categories are expanded to 15, with four new diseases: macular hole (MH), choroidal excavation (CE), choroidal atrophy (CA), and retinal hemorrhage (RH). The remaining 11 categories are consistent with the training and test sets. All OCT slice images are annotated by clinicians, and the annotation content includes the bounding box coordinates of the lesion area and its corresponding disease type, such as Figure 4 As shown, Figure 4 It is the annotation of different categories of retinal diseases, including the bounding box of the real lesion area and its corresponding real multi-disease type probability distribution, Figure 4 (a) in the figure is the pigment epithelial detachment (PED) mark. Figure 4 (b) in the figure is the mark of posterior vitreous detachment (PVD). Figure 4(c) in the figure is the epiretinal membrane (ERM). Figure 4 (d) in the figure is the subretinal fluid (SRF) mark. Figure 4 (e) in the figure is the choroidal neovascularization (CNV) mark. Figure 4 The (f) in the figure is the mark for drusen. Figure 4 (g) in the figure is marked as Retinoschisis. Figure 4 (h) in the figure is the mark of cystoid macular edema (CME). Figure 4 (i) in the figure is hard exudate (HE). Figure 4 (j) in the figure is the mark of full-thickness macular hole (FTMH). Figure 4 (k) in the figure is the retinal detachment (RD) mark. Figure 4 (l) in the figure is the mark of internal segment / external segment fracture (IS / OS). Figure 4 The (m) in the figure is the non-macular epiretinal membrane (ERM (Non-Macular)). Figure 4 The (n) in the figure represents the lamellar macular hole (LMH) annotation. The training and test set annotations adhere to 14 disease categories, while the domain adaptation dataset uses a unified 15-category annotation specification to support cross-device feature alignment research. This dataset covers multiple centers, multiple devices, and multiple disease types, providing a high-quality foundation for robust model training and generalization validation.

[0124] As shown in Table 1, Table 1 is a data diagram of the training and test data sets.

[0125] Table 1

[0126]

[0127] Table 1 is used to show the distribution of datasets in the model training and testing phases, in contrast to the target domain data in Table 2. The target domain device dataset contains 15 categories of retinal diseases, as shown in Table 2. Table 2 is a data schematic table of the target domain device dataset.

[0128] Table 2

[0129]

[0130]

[0131] Table 2 shows the distribution of the dataset in the target domain (i.e., different OCT scanner devices) to verify the generalization ability of the domain adaptation strategy (BV-DA-CNN) in cross-device scenarios. Some disease categories differ from the training and test datasets. To eliminate the impact of annotation heterogeneity, three OCT imaging experts re-annotated the original training dataset and uniformly adjusted it to the 15-category disease classification standard.

[0132] The model was trained using the PyTorch framework, employing the Stochastic Gradient Descent (SGD) optimizer, over 180,000 training iterations on a server equipped with eight GeForce GTX1080Ti GPUs. The initial learning rate was set to 0.0025, and the weight decay coefficient was set to 0.0001 to balance model complexity and generalization. To enhance data diversity, data augmentation strategies such as random flipping, random cropping, and noise addition were introduced during training to effectively mitigate overfitting.

[0133] The model performance evaluation covers two dimensions: positioning and classification: positioning performance is quantified by the intersection over union (IoU) of the predicted bounding box and the true annotation box; classification performance is comprehensively measured by false negative rate (FNR), false positive rate (FPR), accuracy (ACC), precision (PRE), Kappa coefficient and F1 score. In addition, multi-model comparative analysis is performed by drawing the ROC curve (Receiver Operating Characteristic Curve, ROC) and calculating the area under the curve (Area Under Curve, AUC). The definitions of each indicator are as follows: FNR reflects the proportion of missed diagnosis, FPR represents the risk of misdiagnosis, ACC and PRE respectively evaluate the overall diagnostic accuracy and the accuracy of positive sample discrimination, the Kappa coefficient measures the consistency between annotators, the F1 score balances the recall rate and precision, and the ROC curve comprehensively displays the sensitivity and specificity trade-off of the model at different thresholds. The formula is as follows:

[0134]

[0135] Among them, FNR is the false negative rate, FPR is the false positive rate, ACC is the accuracy rate, PRE is the precision rate, F1score is the F1 score, Kappa is the Kappa coefficient, TP is the number of true positive samples, FP is the number of false positive samples, FN is the number of true negative samples, and FN is the number of false negative samples.

[0136] like Figure 5 As shown, Figure 5It is a comparison diagram of the intersection-over-union (IoU) of the target lesion area bounding box predicted by the parallel detection model for multiple types of retinal diseases and the actual target lesion area bounding box. The present invention trains the parallel detection model for retinal disease positioning using a training data set, and verifies its performance on a test data set. The model generates a bounding box for the identified lesion area in the OCT image, and intuitively demonstrates the positioning effect. By calculating the intersection-over-union (IoU) of the predicted bounding box and the actual labeled box, the average IoU of the model on 14 types of retinal diseases reaches 0.44, which is better than the attending physician group (average IoU 0.42) and the resident physician group (average IoU 0.42). Specifically, among the 14 types of diseases, the positioning performance of the parallel detection model for multiple types of retinal diseases is better than or equal to that of the physician group for 9 types of diseases, and the positioning accuracy of the remaining 5 types of diseases is slightly lower than that of at least one physician group. All IoU values ​​are calculated based on the annotations of the chief physician group. The results show that the positioning results of the parallel detection model for multiple types of retinal diseases are closest to the judgment of the chief physician, verifying its technical advantages in lesion area identification.

[0137] like Figure 6 As shown, Figure 6 This is a schematic diagram of the multi-label classification results of retinal diseases using a parallel detection model for multiple retinal diseases. Figure 6 (a) in the figure shows the accuracy result. Figure 6 (b) in the figure shows the precision result. Figure 6 (c) in the figure shows the F1 score result. Figure 6 (d) in the figure is the false positive rate (FPR) result. Figure 6 (e) in the figure is the false negative rate (FNR) result. Figure 6 (f) in the figure is a schematic diagram of the Kappa coefficient results.

[0138] The present invention performs multi-label classification training on the OCT-AI system through a training data set, and verifies its performance on a test data set. Taking the annotations of the chief physician group as a benchmark, the average accuracy of the system reached 93.12%, which is better than the attending physician group (89.36%) and the resident physician group (87.24%). Among the 14 types of diseases, the classification performance of OCT-AI for 11 types of diseases is better than or equal to that of the attending physician group, and only slightly worse in three types of diseases: full-thickness macular hole (FTMH), retinal detachment (RD) and lamellar macular hole (LMH). Specifically, the system has the highest classification accuracy for pigment epithelial detachment (PED) (97.31%), and the lowest accuracy for drusen (87.13%). Other key metrics included an average precision of 82.32%, an F1 score of 0.78, a false positive rate (FPR) of 3.69%, a false negative rate (FNR) of 21.46%, and a Kappa coefficient of 0.76, all significantly exceeding those of the resident group (FNR of 40.79%), validating its technical superiority in multi-label classification tasks. Experimental results demonstrate that the multi-class retinal disease parallel detection model demonstrates diagnostic capabilities approaching or exceeding those of human experts in disease categories with sufficient data, providing a reliable basis for large-scale clinical application.

[0139] Based on the target domain device dataset, four domain adaptation experiments were conducted. Figure 7 As shown, Figure 7 This is a structural comparison diagram of different domain adaptation strategies. FR-CNN and BV-FR-CNN are commonly used methods to solve domain shift. The FR-CNN method uses a re-labeled training dataset to train the Faster R-CNN model and directly applies it to 242,422 test images in the domain adaptation dataset. The DA-CNN method performs semi-supervised domain adaptation based on the Domain Adaptive Faster R-CNN framework, uses unlabeled training images in the domain adaptation dataset to adjust model parameters, and improves model robustness through image-level and instance-level feature alignment and regularization consistency constraints. The BV-FR-CNN method directly uses labeled samples in the domain adaptation dataset to train the Faster R-CNN model. Based on a multi-class retinal disease parallel detection method designed by the present invention, the designed domain adaptation strategy BV-DA-CNN method performs fully supervised domain adaptation based on the FR-CNN pre-trained model, expands the instance-level classifier from binary classification to multi-class classification, removes the consistency regularization constraint in the original framework, and designs a bounding box tightness constraint loss.

[0140] As shown in Table 3, Table 3 shows the experimental results of four domain adaptation experiments.

[0141] Table 3

[0142]

[0143] like Figure 8 As shown, Figure 8 This is a schematic diagram comparing ROC curves of parallel detection models for multiple types of retinal diseases under different domain adaptation strategies. In the domain adaptation experimental study, the present invention constructed a domain adaptation dataset based on the BV1000-OCT scanner. This dataset has a potential data distribution offset problem. The domain adaptation strategy BV-DA-CNN method achieved the best values ​​in terms of false negative rate (FNR), Kappa coefficient and F1 score indicators, and its area under the ROC curve (AUC) was 0.9406, which was significantly better than other methods. Although the traditional FR-CNN method has the best false positive rate (FPR), its FNR, Kappa coefficient and F1 score are significantly inferior to the other three methods, and its AUC is only 0.7068. The overall performance of the DA-CNN method is inferior to that of BV-FR-CNN and BV-DA-CNN. The BV-FR-CNN method performs best in terms of accuracy (ACC) and precision (PRE) indicators, but its Kappa coefficient and F1 score are slightly lower than those of the BV-DA-CNN method. Experimental data show that the BV-DA-CNN method achieves generalization performance on the test set that is comparable to that of the training set, verifying the effectiveness of this domain adaptation method.

[0144] like Figure 9 As shown, Figure 9 This is a flow chart of a device for detecting multiple types of retinal diseases in parallel. Based on the first embodiment, this second embodiment provides a device for detecting multiple types of retinal diseases in parallel, including:

[0145] The terminal OCT scanner is used to collect the patient's retinal OCT images and associated clinical data in real time; the associated clinical data shown includes information such as age, gender, and region;

[0146] The cloud server is connected to the terminal OCT scanner and is used to receive the patient's retinal OCT images and associated clinical data acquired by the terminal OCT scanner. The cloud server processes the received patient OCT images using the deployed multi-class retinal disease parallel detection model to obtain the coordinates of the bounding box of each lesion area in each OCT slice image of the patient and the probability distribution of the target multi-disease type, and generate a preliminary auxiliary diagnosis report.

[0147] The expert team review platform is connected to the cloud server and is used to receive the preliminary auxiliary diagnosis report generated by the cloud server to review the diagnosis results, generate the target auxiliary diagnosis report, and transmit it to the cloud server.

[0148] The diagnostic process of the present invention covers three core links: terminal data collection, cloud-based intelligent analysis, and expert collaborative correction. Specifically, the terminal OCT scanner collects the patient's retinal OCT images and related clinical data (including age, gender, region, etc.) in real time, and uploads the data to the cloud server. The OCT-AI system deployed on the cloud server processes the received OCT images in parallel, and synchronously completes the positioning of abnormal areas and the classification of 14 types of retinal diseases based on the deep learning model to generate a preliminary auxiliary diagnosis report. The report is then pushed to the expert team review platform, and an expert team composed of multidisciplinary physicians reviews the diagnosis results and, if necessary, corrects or supplements the diagnosis opinions to reach a consensus. The final confirmed diagnosis report is transmitted back to the terminal device through the cloud to provide a reference for clinical physicians to formulate treatment plans. Through the deep integration of cloud-based intelligence and manual review, this process has built an efficient and accurate remote diagnosis service system, which is particularly suitable for areas with uneven distribution of medical resources.

[0149] The data annotation process of the present invention adopts a multi-level review mechanism to ensure the accuracy and consistency of annotation. For the test data set, three types of annotation groups (chief physician group, attending physician group and resident physician group) are formed by 9 physicians with different levels of clinical experience, and each group has 1 team leader and 2 members. During the annotation process, each group member first independently completes the preliminary annotation of all OCT images, covering the bounding box positioning of the lesion area and the corresponding disease type classification; then the team leader reviews all the annotation results item by item, corrects the errors and completes the final confirmation. Among them, the annotation results of the chief physician group are used as the true labels of the performance indicators of the calculation system, and the annotation data of the other two groups are used for comparative analysis to evaluate the annotation differences of physicians with different experience levels.

[0150] For the training and domain adaptation datasets, three OCT imaging experts with experience comparable to that of the chief physician team were recruited to carry out the labeling work. After two members independently completed the bounding box position and disease type labeling of all data, the team leader systematically reviewed all the labeling results, corrected the deviations, and completed the final confirmation to form the true labels for model training and domain adaptation. This process effectively reduced the subjective error of labeling through multi-level collaboration and cross-validation mechanisms, and improved the reliability and generalization ability of the dataset. All labeled data contains two parts of information: the bounding box coordinates of the lesion area and the corresponding retinal disease type, providing structured supervision signals for the model's parallel positioning and classification tasks.

[0151] This paper proposes a multi-task deep learning framework that integrates positioning and classification. By designing a parallel detection model for multiple types of retinal diseases, using ResNet-101 as the feature extraction backbone network, combining the candidate region generation network (RPN) with a multi-label classifier, single-slice prediction is achieved, and considering the contextual relationship between different slices, a parallel detection model for multiple types of retinal diseases is constructed. This realizes the synchronous parallel detection of 14 types of retinal diseases and end-to-end abnormal area positioning and disease classification, significantly improving detection efficiency and accuracy, and breaking through the limitations of traditional single disease and few disease detection models. To address the problem of model performance degradation caused by imaging differences between different OCT scanners, a domain adaptive method (Domain Adaptive Faster R-CNN) is introduced. By aligning image-level and instance-level feature distributions and combining multi-label classification module optimization, the model's cross-device generalization capability is enhanced.

[0152] A multi-center, multi-device OCT dataset was constructed (14 disease categories in the training / test set and 15 disease categories in the domain adaptation set). A multi-level expert review mechanism was implemented, with collaborative annotation by the chief physician group, attending physician group, and resident physician group (with the chief physician group's annotation as the gold standard) and cross-validation annotation by three experts on the training and domain adaptation datasets. This reduced subjective differences in annotation and provided data assurance for model robustness. A cloud-based remote diagnosis process was proposed, supporting terminal data collection, cloud-based parallel analysis, and expert collaborative review. Through slice-level parallel reasoning, the spatial correlation information of the whole-eye OCT sequence was integrated to reduce the risk of missed detection in a single slice, thereby enabling efficient screening and remote consultation in primary medical institutions.

[0153] This third embodiment provides a multi-type retinal disease parallel detection system, including:

[0154] A single-slice prediction module is used to input each OCT slice image of the patient to be tested into the single-slice prediction module of the multi-class retinal disease parallel detection model, output the coordinates of the lesion region bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types, and determine the disease type to which each lesion region bounding box belongs. The single-slice prediction module is constructed by replacing the binary classifier in the Faster R-CNN model with a multi-label classifier; the output dimension of the multi-label classifier is the total number of retinal disease types;

[0155] A set construction module is configured to calculate, for each lesion region bounding box in each OCT slice image, the intersection-and-union ratio of the lesion region bounding box with the same disease type in adjacent slices of the slice, retaining the lesion region bounding box with an intersection-and-union ratio greater than a preset threshold as the same type of bounding box of the lesion region bounding box; and combining each lesion region bounding box with its same type of bounding box as a cross-slice lesion region bounding box set;

[0156] an expansion module, configured to expand each lesion region bounding box based on the bounding box coordinates of the set of cross-slice lesion region bounding boxes corresponding to each lesion region bounding box, to obtain the coordinates of each target lesion region bounding box;

[0157] A probability prediction module is used to calculate the mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box as the target multi-disease type probability distribution of each target lesion area bounding box;

[0158] The output module is used to output the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of its target multi-disease type as the output of the multi-class retinal disease parallel detection model.

[0159] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0161] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0163] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for parallel detection of multiple types of retinal diseases, characterized in that: include: Each OCT slice image of the patient to be tested is input into the single-slice prediction module of the multi-class retinal disease parallel detection model. The coordinates of the lesion region bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types are output, and the disease type to which each lesion region bounding box belongs is determined. The binary classifier in the Faster R-CNN model is replaced with a multi-label classifier to construct the single-slice prediction module. The output dimension of the multi-label classifier is the total number of retinal disease types. For each lesion bounding box in each OCT slice image, the intersection-and-union ratio (IoU) of the lesion bounding box with the same disease type in the adjacent slices of the slice is calculated. The lesion bounding boxes with an IoU ratio greater than a preset threshold are retained as the same-type bounding boxes of the lesion bounding box. Each lesion bounding box and its same-type bounding boxes are collectively regarded as a cross-slice lesion bounding box set. Expanding each lesion region bounding box based on the bounding box coordinates of the cross-slice lesion region bounding box set corresponding to each lesion region bounding box to obtain the coordinates of each target lesion region bounding box; The mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box is used as the target multi-disease type probability distribution of each target lesion area bounding box; The coordinates of the bounding boxes of each target lesion area in each OCT slice image and the probability distribution of target multi-disease types are used as the output of the multi-class retinal disease parallel detection model.

2. A method for parallel detection of multiple types of retinal diseases according to claim 1, characterized in that: The adjacent slices of each OCT slice image include: Arrange N OCT slices into a sequence in order, and the entire sequence is in a cyclic state; extract the k slices before and after each OCT slice image in the sequence as the adjacent slices of each OCT slice image; where N is the total number of OCT slice images and k is a hyperparameter.

3. The method for parallel detection of multiple retinal diseases according to claim 1, wherein: Based on the bounding box coordinates of the cross-slice lesion region bounding box set corresponding to each lesion region bounding box, each lesion region bounding box is expanded to obtain the coordinates of the target lesion region bounding box of each lesion region, including: For each lesion region bounding box, the minimum horizontal coordinate in the corresponding set of cross-slice lesion region bounding boxes is used as the horizontal coordinate of the lower left corner and the upper left corner of the lesion region bounding box; The maximum horizontal coordinate in the corresponding cross-slice lesion region bounding box set is used as the horizontal coordinate of the lower right corner and the upper right corner of the lesion region bounding box; The minimum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the vertical coordinate of the lower left corner and the lower right corner of the lesion region bounding box; The maximum vertical coordinate in the corresponding cross-slice lesion region bounding box set is used as the upper left corner vertical coordinate and the upper right corner vertical coordinate of the lesion region bounding box.

4. The method for parallel detection of multiple retinal diseases according to claim 1, wherein: The method includes inputting each OCT slice image of the patient to be tested into the single slice prediction module of the multi-class retinal disease parallel detection model, and outputting the coordinates of the lesion area bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types, including: Each OCT slice image is passed through a feature extraction module to extract a feature map of each OCT slice image; The feature map of each OCT slice image is passed through the candidate region generation network to generate the coordinates of the candidate box of the lesion area in the feature map of each OCT slice image; The feature map of each OCT slice image and the coordinates of the candidate box of the lesion area are input into the bounding box regression module, the coordinates of the candidate box of the lesion area are adjusted, and the coordinates of the bounding box of the lesion area of ​​each OCT slice image are output; The coordinates of the lesion region bounding box of each OCT slice image are mapped to the feature map of the OCT slice image, and the lesion feature vector of each lesion region bounding box of each OCT slice image is extracted through the region of interest pooling module; The lesion feature vector of each lesion area bounding box of each OCT slice image is passed through a multi-label classifier to obtain the probability distribution of multiple disease types corresponding to the lesion area bounding box of each OCT slice image.

5. The method for parallel detection of multiple types of retinal diseases according to claim 4, characterized in that: Multi-label classifiers include: Multiple parallel fully connected branches, each fully connected branch includes a fully connected layer and a Sigmoid function connected in sequence; wherein the number of fully connected branches is the total number of retinal disease types.

6. The method for parallel detection of multiple types of retinal diseases according to claim 5, characterized in that: The lesion feature vectors of each lesion region bounding box of each OCT slice image are passed through a multi-label classifier to obtain the probability distribution of multiple disease types corresponding to the lesion region bounding box of each OCT slice image. The calculation formula for the probability distribution of multiple disease types is: Among them, P(y c,i =1|x i ) represents the probability that the i-th OCT slice image has the c-th retinal disease, x i is the i-th OCT slice image, φ(x i ) is the lesion feature vector of each lesion area bounding box in the i-th OCT slice image, y c,i Indicates whether the i-th slice has the c-th retinal disease, v c The transpose of v c is the weight vector of the cth fully connected branch, e is a natural constant, C is the total number of retinal disease types, c is the retinal disease type index, and i is the OCT slice image index.

7. The method for parallel detection of multiple retinal diseases according to claim 1, characterized in that: The training method for the multi-class retinal disease parallel detection model is: Obtaining a target domain device dataset, the target domain device dataset including: all OCT slice images of different patients, coordinates of a true lesion region bounding box of each OCT slice image, and corresponding true multi-disease type probability distribution; All OCT slice images of the patient are input into the multi-class retinal disease parallel detection model, and the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of the target multi-disease type are output; A multi-label classification loss is constructed based on the difference between the target multi-disease type probability distribution and the true multi-disease type probability distribution of each lesion area bounding box in all OCT slice images; Based on the intersection-over-union ratio of the bounding box of each lesion area in all OCT slice images and the bounding box of the true lesion area, a bounding box tightness constraint loss is constructed; Based on multi-label classification loss and bounding box tightness constraint loss, a total loss function is constructed; the total loss function is used to train a parallel detection model for multiple categories of retinal diseases to obtain the target parallel detection model for multiple categories of retinal diseases.

8. The method for parallel detection of multiple retinal diseases according to claim 7, characterized in that: The bounding box tightness constraint loss is: Among them, L ins is the bounding box tightness constraint loss, N is the total number of OCT slice images, i is the OCT slice image index, is the lesion area bounding box of the i-th OCT slice image and the actual lesion area bounding box IoU(·) is the intersection-over-union ratio, is the lesion area bounding box of the i-th OCT slice image, is the true lesion area bounding box of the i-th OCT slice image ‖·‖2 is the Euclidean norm.

9. The method for parallel detection of multiple retinal diseases according to claim 7, characterized in that: The total loss function is: L=L multi-label +λL ins , Among them, L is the total loss function, L multi-label is the multi-label classification loss, L ins is the bounding box tightness constraint loss, and λ is the bounding box tightness constraint loss weight.

10. A parallel detection system for multiple types of retinal diseases, characterized in that: include: A single-slice prediction module is used to input each OCT slice image of the patient to be tested into the single-slice prediction module of the multi-class retinal disease parallel detection model, output the coordinates of the lesion region bounding box of each OCT slice image and the corresponding probability distribution of multiple disease types, and determine the disease type to which each lesion region bounding box belongs. The single-slice prediction module is constructed by replacing the binary classifier in the Faster R-CNN model with a multi-label classifier; the output dimension of the multi-label classifier is the total number of retinal disease types; A set construction module is configured to calculate, for each lesion region bounding box in each OCT slice image, the intersection-and-union ratio of the lesion region bounding box with the same disease type in adjacent slices of the slice, retaining the lesion region bounding box with an intersection-and-union ratio greater than a preset threshold as the same type of bounding box of the lesion region bounding box; and combining each lesion region bounding box with its same type of bounding box as a cross-slice lesion region bounding box set; an expansion module, configured to expand each lesion region bounding box based on the bounding box coordinates of the set of cross-slice lesion region bounding boxes corresponding to each lesion region bounding box, to obtain the coordinates of each target lesion region bounding box; A probability prediction module is used to calculate the mean of the multi-disease type probability distribution of the set of cross-slice lesion area bounding boxes corresponding to each lesion area bounding box as the target multi-disease type probability distribution of each target lesion area bounding box; The output module is used to output the coordinates of the bounding box of each target lesion area in each OCT slice image and the probability distribution of its target multi-disease type as the output of the multi-class retinal disease parallel detection model.

Images