A multi-expert collaborative toxic detection system and method based on fundus image dissection prior and uncertainty quantification

Abstract

The application belongs to the technical field of deep learning, and discloses a multi-specialist collaborative toxic detection system and method based on fundus image dissection prior and uncertainty quantification, which comprises the following steps: a data acquisition and preprocessing module uses a professional fundus camera to shoot an original fundus image of a person to be tested and performs a series of pretreatments; a multi-specialist feature extraction module performs feature extraction and subsequent processing on the global image, optic disc ROI and macular ROI after preprocessing by the global specialist network, optic disc specialist network and macular specialist network respectively; a multi-specialist feature fusion module fuses the features extracted by the three specialist networks to generate a fusion feature vector; an uncertainty quantification module obtains a classification probability and an uncertainty score based on Bayesian approximation; and a hierarchical decision and report generation module obtains a hierarchical result based on hierarchical decision logic, and generates a final hierarchical report and a report of each specialist according to the hierarchical results of the multi-specialists and the hierarchical result of the fusion branch.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically relating to a multi-expert collaborative toxicology detection system and method based on fundus image anatomical priors and uncertainty quantification. Background Technology

[0002] Drug abuse detection is a crucial issue in the field of public safety. Current technologies primarily rely on biological samples such as urine, blood, and hair, but these have limitations including high invasiveness, short window periods, susceptibility to metabolic interference, and high testing costs. Portable devices based on pupillary reflexes suffer from poor specificity due to interference from ambient light and eye conditions. In contrast, research has shown that the fundus retina contains rich pathological characteristics of drug abuse, providing a theoretical basis for non-invasive drug detection.

[0003] However, directly applying existing general-purpose deep learning classification models for eye diseases to detect drug abuse still faces challenges. Drug-induced fundus changes are anatomically specific and subtle. Existing convolutional neural networks typically use full-image scaling input, which can cause these crucial local micro-features to be masked or lost by background texture, leading to missed detections. Furthermore, existing models are mostly "black box" models, lacking uncertainty quantification capabilities. Even with blurry or irrelevant images, they often output high-confidence classification results, resulting in a high risk of false positives. Therefore, integrating prior anatomical knowledge and employing detection methods that effectively assess predictive risk can help capture subtle fundus features induced by drugs and reduce the risk of false positives in drug use detection. Summary of the Invention

[0004] This invention provides a multi-expert collaborative toxicology detection system and method based on anatomical priors and uncertainty quantification of fundus images, in order to solve the problems of loss of local minor features in fundus images, lack of reliability assessment of model prediction results, and difficulty for a single deep learning network to achieve a balance between global geometric features and local features.

[0005] To achieve the above objectives, the present invention provides the following solution: A multi-expert collaborative toxicology testing system based on fundus image anatomical priors and uncertainty quantification, the system comprising: a data acquisition and preprocessing module, a multi-expert feature extraction module, a multi-expert feature fusion module, an uncertainty quantification module, and a hierarchical decision-making and report generation module; The data acquisition and preprocessing module is used to preprocess the raw fundus image and construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map; The multi-expert feature extraction module is used to construct a multi-expert convolutional neural network model containing three parallel sub-networks based on the multi-source input data stream, and to extract features from the preprocessed global fundus map, optic disc ROI map and macular ROI map; The multi-expert feature fusion module is used to fuse the extracted features to generate a fused feature vector. The uncertainty quantification module is used to obtain the classification probability and uncertainty score based on the fused feature vector using Bayesian approximation. The hierarchical decision and report generation module is used to obtain hierarchical results based on hierarchical decision logic, and generate a final hierarchical report and reports from each expert based on the hierarchical results of multiple experts and the hierarchical results of the fusion branch.

[0006] Preferably, the process of preprocessing the raw fundus image to construct a multi-source input data stream including a global fundus map, an optic disc ROI map, and a macular ROI map includes: Acquire raw fundus images; Preprocessing operations such as effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization are performed on the original fundus image; The lightweight object detection network YoLov5 is used to generate bounding boxes for the optic disc and macula on the preprocessed image. The ROI image is then cropped on the original fundus image after the effective region is extracted based on the bounding box coordinates. The cropped ROI image is subjected to size normalization, adaptive enhancement, noise reduction, and statistical normalization to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map.

[0007] Preferably, the multi-expert convolutional neural network model includes: a global expert network, a visual disc expert network, and a macular expert network; The global expert network is used to receive the global fundus image and extract the retinal vessel distribution, arteriovenous diameter ratio and overall texture features through a deep convolutional network. The optic disc expert network is used to receive optic disc ROI images and extract optic nerve color pallor, disc rim morphology, and optic cup-optic disc ratio features. The macular expert network is used to receive macular ROI images and extract features of macular edema, pigmentary disorders, and microaneurysms.

[0008] Preferably, the process of fusing the extracted features to generate a fused feature vector includes: The extracted features are mapped to a unified channel dimension, the feature sizes are aligned using a spatial broadcasting mechanism, and the interaction weights between local and global features are calculated using a channel attention mechanism, ultimately generating a fused feature vector that includes anatomical priors.

[0009] Preferably, the process of obtaining the classification probability and uncertainty score based on the fused feature vector using Bayesian approximation includes: Construct a fully connected classifier containing a Dropout layer and input the fused feature vector into the classifier; keep the Dropout layer active during the testing phase and perform N random forward propagation operations on the same input sample to obtain N sets of predicted probability distributions; calculate the mean of the N sets of predicted probabilities as the detection result, and calculate the variance of the N sets of predicted probabilities as the uncertainty score of the detection result.

[0010] This invention also provides a multi-expert collaborative toxicology detection method based on fundus image anatomical prior and uncertainty quantification. The method is implemented through the aforementioned system and includes: The raw fundus images are preprocessed to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map; Based on the multi-source input data stream, a multi-expert convolutional neural network model containing three parallel sub-networks is constructed to extract features from the preprocessed global fundus map, optic disc ROI map, and macular ROI map. The extracted features are fused to generate a fused feature vector; Based on the fused feature vectors, the classification probability and uncertainty score are obtained using Bayesian approximation. The classification results are obtained based on the classification logic, and the final classification report and the reports of each expert are generated based on the classification results of multiple experts and the classification results of the fusion branch.

[0011] Preferably, the method for preprocessing the raw fundus image to construct a multi-source input data stream including a global fundus map, an optic disc ROI map, and a macular ROI map includes: Acquire raw fundus images; Preprocessing operations such as effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization are performed on the original fundus image; The lightweight object detection network YoLov5 is used to generate bounding boxes for the optic disc and macula on the preprocessed image. The ROI image is then cropped on the original fundus image after the effective region is extracted based on the bounding box coordinates. The cropped ROI image is subjected to size normalization, adaptive enhancement, noise reduction, and statistical normalization to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map.

[0012] Preferably, the multi-expert convolutional neural network model includes: a global expert network, a visual disc expert network, and a macular expert network; The global expert network is used to receive the global fundus image and extract the retinal vessel distribution, arteriovenous diameter ratio and overall texture features through a deep convolutional network. The optic disc expert network is used to receive optic disc ROI images and extract optic nerve color pallor, disc rim morphology, and optic cup-optic disc ratio features. The macular expert network is used to receive macular ROI images and extract features of macular edema, pigmentary disorders, and microaneurysms.

[0013] Preferably, methods for fusing the extracted features to generate a fused feature vector include: The extracted features are mapped to a unified channel dimension, the feature sizes are aligned using a spatial broadcasting mechanism, and the interaction weights between local and global features are calculated using a channel attention mechanism, ultimately generating a fused feature vector that includes anatomical priors.

[0014] Preferred methods for obtaining classification probabilities and uncertainty scores based on fused feature vectors and using Bayesian approximation include: Construct a fully connected classifier containing a Dropout layer and input the fused feature vector into the classifier; keep the Dropout layer active during the testing phase and perform N random forward propagation operations on the same input sample to obtain N sets of predicted probability distributions; calculate the mean of the N sets of predicted probabilities as the detection result, and calculate the variance of the N sets of predicted probabilities as the uncertainty score of the detection result.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention avoids the loss of image details and reduces the rate of missed diagnoses by introducing a local expert network for the optic disc and macula. By introducing uncertainty quantification, the system can automatically identify abnormal samples with poor image quality, atypical lesion features, or unknown types, and mark them as "doubtful" and divert them to the manual review process, reducing the risk of misjudgment. The multi-expert architecture enables the system to generate structured reports that clearly indicate which part of the region is dominant in positive results. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a block diagram of a multi-expert collaborative toxicology detection system based on fundus image anatomical priors and uncertainty quantification, according to an embodiment of the present invention. Figure 2 This is a block diagram of the overall structure of the model according to an embodiment of the present invention; Figure 3 This is a structural block diagram of the feature fusion module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of uncertainty quantification and decision-making logic in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Example 1 like Figure 1 As shown, the multi-expert collaborative toxicology detection system based on fundus image anatomical prior and uncertainty quantification provided by the present invention comprises six components: a data acquisition and preprocessing module, a multi-expert feature extraction module, a multi-expert feature fusion module, an uncertainty quantification module, and a hierarchical decision and report generation module. The data acquisition and preprocessing module extracts the effective regions of the original fundus image, performs size normalization, adaptive enhancement, noise reduction, and statistical normalization. It uses a pre-trained object detection network to locate the optic disc and macular regions in the preprocessed fundus image, and crops the optic disc ROI image and macular ROI image from the original image based on the localization coordinates. Then, it performs size normalization, adaptive enhancement, noise reduction, and statistical normalization on them to construct a multi-source input data stream containing the global fundus image, optic disc ROI image, and macular ROI image. The multi-expert feature extraction module, based on the multi-source input data stream, constructs a multi-expert convolutional neural network model containing three parallel sub-networks. These sub-networks include: a global expert network that receives a global fundus image and extracts retinal vessel distribution, arteriovenous diameter ratio, and overall texture features through a deep convolutional network; an optic disc expert network that receives an optic disc ROI image and extracts optic nerve pallor, disc rim morphology, and optic cup-optic disc ratio features; and a macular expert network that receives a macular ROI image and extracts features of macular edema, pigmentary disorders, and microaneurysms. The multi-expert feature fusion module inputs the feature maps output by the three expert networks into the feature fusion module. The feature fusion module first maps the features of each branch to a unified channel dimension, aligns the feature size using a spatial broadcasting mechanism, and calculates the interaction weights between local and global features through a channel attention mechanism, finally generating a fused feature vector containing anatomical priors. The uncertainty quantification module constructs a fully connected classifier containing a Dropout layer and inputs the fused feature vector into the classifier; during the testing phase, the Dropout layer is kept active, and N random forward propagations are performed on the same input sample to obtain N sets of predicted probability distributions; the mean of the N sets of predicted probabilities is calculated as the drug detection result, and the variance of the N sets of predicted probabilities is calculated as the uncertainty score of the detection result; The grading and report generation module, based on preset uncertainty and classification thresholds, categorizes test results into three levels: "high confidence positive," "high confidence negative," and "low confidence uncertain," and outputs a diagnostic report. The feature fusion branch outputs a grading result, and the three expert branches also output their respective grading results. The final grading result report is generated by combining the grading results from the fusion branch and the three expert branches.

[0021] The data acquisition and preprocessing module is used to acquire raw fundus images and address issues such as inconsistent resolution, uneven illumination, background noise interference, and cropping and preprocessing of optic disc and macular region of interest (ROI) images from different fundus cameras. It performs effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization on the raw fundus images. Using the lightweight object detection network YoLov5, it generates bounding boxes for the optic disc and macular regions on the preprocessed images. Based on the bounding box coordinates, it crops the ROI images from the extracted raw fundus images, and then performs size normalization, adaptive enhancement, noise reduction, and statistical normalization on the cropped ROI images. Finally, this module sends the preprocessed raw fundus images, preprocessed optic disc ROI images, and preprocessed macular ROI images to the multi-expert feature extraction module. Specifically, the data preprocessing of the raw fundus images includes the following steps: (1) Effective region extraction: The red, green and blue channels of the original color fundus image are weighted pixel by pixel according to preset weights and converted into a grayscale image. And using a preset brightness threshold For grayscale images Perform binarization segmentation to generate a binarized mask M indicating the effective eye region. : Subsequently, the mask M is calculated. The largest connected component with a pixel value of 1 is identified, and its minimum bounding rectangle coordinates are obtained. Based on these coordinates, the effective region is cropped on the original color fundus image. A connected component analysis algorithm is applied to extract the largest connected region in the binarized mask. The minimum value of the horizontal coordinate is obtained by traversing and calculating the extreme values ​​of the horizontal and vertical coordinates of all non-zero pixels within this largest connected region. With the maximum value and the minimum value of the ordinate. With the maximum value This generates the coordinates of the minimum bounding rectangle enclosing the eye region. Finally, the coordinates are used to perform a cropping operation on the original color fundus image to remove invalid black background areas. (2) Size normalization: The cropped image is uniformly scaled to a preset size using a bicubic interpolation algorithm. Pixel standard input size; the algorithm utilizes the surrounding area of ​​the point to be sampled. A smooth surface is constructed by weighting 16 pixels in the neighborhood, which effectively suppresses aliasing noise during scaling while adjusting the image resolution, and preserves the continuous edges of the retinal vascular network to the greatest extent. (3) Adaptive Enhancement: The image is converted to the Lab color space, and an adaptive histogram equalization algorithm is applied only to the luminance component, with a grid size of [value missing]. to Keeping the chromaticity components unchanged, the processed luminance components are merged with the original chromaticity components, and then converted back to RGB space; (4) Noise reduction: A Gaussian filter is used to smooth the enhanced image in order to suppress high-frequency noise amplified by histogram equalization; (5) Statistical normalization: Z-Score normalization is applied to the image to accelerate the convergence of the convolutional neural network. This method maps the image data to an approximate standard normal distribution with a mean of 0 and a variance of 1. This not only eliminates the dimensional differences between different channels, but also effectively avoids the deep activation functions of the network from getting stuck in the gradient saturation region, thereby significantly accelerating the parameter convergence of the multi-expert convolutional neural network and improving the generalization ability of the model. Specifically, the cropping and preprocessing of the optic disc and macula ROI images includes the following steps: (1) ROI Image Cropping: For the preprocessed fundus image, the predicted bounding boxes of the optic disc and macula are extracted using the pre-trained lightweight object detection network YOLOv5, and the center point coordinates of the bounding boxes in the preprocessed low-resolution image are obtained. and the width of the bounding box With height Then, the size scaling factor between the original high-resolution fundus image and the preprocessed low-resolution image is calculated. The specific calculation method is: horizontal scaling factor. Vertical scaling factor ,in and These represent the width and height of the image, respectively. and These represent the original high-resolution fundus image and the preprocessed low-resolution image, respectively. Then, the bounding box coordinates from the low-resolution image are mapped back to the original high-resolution fundus image space. This mapping process involves multiplying the coordinates by the width and height: High-resolution center point coordinates: High-resolution width: ,high: Finally, cropping is performed on the original high-resolution image based on the mapped coordinates. To prevent anatomical distortion caused by subsequent image scaling, cropping preferentially adopts a square cropping strategy with the lesion center as the origin, i.e., using... Choose as the geometric center and The maximum value in the image is used as the side length of the square cropping frame. Slices are then applied to the original high-resolution data to obtain lossless ROI images of the optic disc and macula. (2) Size normalization: The cropped ROI image is uniformly adjusted to size using a bicubic interpolation algorithm. ; (3) Adaptive Enhancement: The size-normalized ROI image is then converted to the Lab color space, and an adaptive histogram equalization algorithm is applied only to the luminance component, with a grid size of [missing value]. to Keeping the chromaticity components unchanged, the processed luminance components are merged with the original chromaticity components, and then converted back to RGB space; (4) Noise reduction: A Gaussian filter is used to smooth the enhanced ROI image in order to suppress high-frequency noise amplified by histogram equalization; (5) Statistical normalization: The Z-Score normalization method is applied to the image to accelerate the convergence of the convolutional neural network; The multi-expert feature extraction module constructs a three-branch convolutional neural network architecture to extract features from the preprocessed raw fundus image, the preprocessed optic disc ROI image, and the preprocessed macular ROI image input from the data acquisition and preprocessing modules, respectively. Specifically, such as Figure 2As shown, the multi-expert feature extraction module comprises three expert networks: the global expert network uses a convolutional neural network as its backbone to extract features from the global fundus image, with a global average pooling layer, a Dropout layer, and a fully connected classification layer at the end of the convolutional neural network; the optic disc expert network and the macular expert network use convolutional neural networks with fewer parameters as their backbones to extract features from the ROI image, each with a global average pooling layer, a Dropout layer, and a fully connected classification layer at the end. Specifically: (1) Global Expert Network: A ResNet-50 deep residual network is used as the feature extraction backbone of the global expert network, with an input size of [missing information]. The entire fundus image is used. This branch is mainly responsible for extracting the overall course of retinal vessels, the characteristics of arteriovenous crossing compression, and widespread retinal ischemia. Its specific network structure and feature extraction process are as follows: Phase 1: Shallow Feature Extraction. The input image first undergoes a... The convolutional layer (with a stride of 2) and The max pooling layer is used to obtain feature maps for extracting basic shallow features such as edges and colors from fundus images.

[0022] The second stage: deep semantic feature extraction. Subsequently, the feature maps from the first stage are sequentially passed through four residual block groups. To address the vanishing gradient problem in deep networks, these residual blocks employ a cross-layer identity mapping (Skip-Connection) mechanism. Let the first... The input of each residual block is After weighting Two convolution operations After that, with input Perform element-wise addition, and finally output the result using the ReLU activation function. Its core mathematical formula is expressed as: Through this residual mechanism, the network can continuously deepen its understanding of the global field of view, thereby capturing deep semantic features representing the entire retinal blood vessels (at this point, the output feature map size is: ).

[0023] The third stage: Feature dimensionality reduction and pooling. The deep feature maps output from the backbone network's ends are input into a global average pooling layer. This layer averages the feature maps across each channel in the spatial dimension, compressing the three-dimensional feature map into a one-dimensional feature vector. Let the feature map be... The pixel value of each channel , This represents the number of pixels in the vertical direction of the feature map. If the number of pixels is in the horizontal direction, then the global average pooling output of this channel is... The calculation formula is: Phase Four: Regularization and Uncertainty Inference Preparation. The obtained feature vectors... The input is fed into the Dropout layer. In this layer, neurons are fed with probability... (0.3 to 0.5) are randomly set to zero. Let the mask matrix be... Then the output feature vector The calculation formula is: This operation is used to prevent model overfitting during training; if kept on during inference, it is used for subsequent uncertainty quantification.

[0024] Fifth stage: Classification probability output. Feature vectors after the Dropout layer. Finally, the input is fed into a fully connected classification layer. The fully connected layer uses a weight matrix... and bias A linear transformation is performed, and then the output is mapped to the probability distribution of each class using the Softmax function. Let a certain class... The linear output is Then the final classification probability of this category. The calculation formula is: in, This represents the total number of categories. Ultimately, the network outputs the probability of drug use classification for the global field of view. In this system, Because the categories are only two: drug users and non-drug users.

[0025] (2) Visual disc expert network: The MobileNetV3 lightweight convolutional neural network is used as the feature extraction backbone of the visual disc expert network, with an input size of [insert size here]. The optic disc ROI image. This branch focuses on analyzing subtle local features such as the paleness of the optic disc's color and the shape of its rim. Its specific network structure and feature extraction process are as follows: Phase 1: Depthwise Separable Convolution. The core of the network consists of multiple inverse residual structures. To reduce computational redundancy for local image analysis, the backbone network uses depthwise separable convolution instead of standard convolution, extracting high-frequency texture and color gradient features of the view disk with minimal parameter count. Depthwise separable convolution decomposes standard convolution into two steps: depthwise convolution and pointwise convolution. Let the input feature tensor be... ( For space dimensions, (where is the number of input channels), and the depthwise convolution kernel size is... Depthwise convolution performs spatial filtering independently on each channel, outputting a feature map. In spatial location ,aisle The calculation formula is: in, The depthwise convolution weight matrix is... This represents the relative spatial coordinate offset within the convolution kernel, used to traverse the entire local filtering window. Subsequently, the following is employed... Pointwise convolution weights Linearly combine the outputs of all channels to generate a product with... The final output characteristics of each channel : in, This represents the spatial pixel coordinate index on the output feature map; The channel index of the input feature map; The feature tensor representing the output of the deep convolutional layer is used as the input for pointwise convolution. Phase Two: Inverse Residual Feature Extraction. The inverse residual structure is first extracted through... Convolution performs channel upscaling; after performing depthwise convolution, it is then passed through... Convolutional dimensionality reduction output, along with the introduction of the h-swish activation function with nonlinear characteristics, significantly enhances the model's ability to represent subtle color differences in the visual disc.

[0026] The third stage: Feature dimensionality reduction and pooling. The feature maps output from the backbone network's terminals are input into a global average pooling layer. This operation compresses the three-dimensional feature map into a one-dimensional feature vector by calculating the arithmetic mean of the spatial dimensions. It effectively integrates the global color and shape statistical features of the viewing area. The specific operation is consistent with the third stage of the global expert.

[0027] Phase Four: Regularization and Preparation for Local Uncertainty Inference. The extracted feature vectors... Input random inactivation rate A Dropout layer of (0.3 to 0.5) generates mask features. (in This operation is used to suppress overfitting during model training and is forced to remain on during inference to support the quantification of local uncertainties in the visual disc.

[0028] Fifth stage: Classification probability output. Finally, the masked features are processed through a fully connected classification layer, and the independent classification probability of local abnormal lesions in the optic disc is output using the Softmax activation function. The specific calculations are consistent with the fifth stage of the global expert analysis.

[0029] (3) Macular Expert Network: The MobileNetV3 lightweight convolutional neural network is used as the feature extraction backbone of the macular expert network, with an input size of [missing information]. The branch focuses on capturing macular ROI images. It specifically targets macular edema, pigmentary disturbances, and talc particle deposition caused by long-term intravenous drug use. Its specific network structure and feature extraction process are as follows: Phase 1: Depthwise Separable Convolution. Similar to the optic disc expert network structure, the core of this backbone network is an inverse residual structure containing depthwise separable convolution. Since talc crystals in fundus images often appear as isolated, tiny, highly reflective dots with a diameter of only a few pixels, traditional standard convolution tends to smooth out these details during downsampling. Depthwise separable convolution decomposes the standard convolution. Let the input macular feature tensor be... ( For space dimensions, (where is the number of input channels), and the depthwise convolution kernel size is... Depthwise convolution performs spatial filtering independently on each channel, outputting a feature map. In spatial location ,aisle The calculation formula is: This step enables precise extraction of fine-grained crystalline texture within each channel of the macular region without mixing channel information. Subsequently, using weights... of Pointwise convolution is used to fuse cross-channel features and generate features. : The second stage: Inverse residual feature extraction. With the "dimensionality increase followed by dimensionality reduction" mechanism of the inverse residual structure, the network can extract the pathological features of "talc crystals" or "microaneurysms" from the complex macular background texture with extremely low parameter quantity and computational cost.

[0030] The third stage: Feature dimensionality reduction and pooling. The deep macular feature map output from the backbone network is input into a global average pooling layer. By calculating the arithmetic mean in the spatial dimension, the three-dimensional feature map is compressed into a one-dimensional feature vector. This involves the global statistical analysis and integration of high-frequency lesion characteristics in the macular region. The specific procedures are consistent with the third stage of the global expert analysis.

[0031] Phase Four: Regularization and Preparation for Local Uncertainty Inference. The extracted feature vectors... Input random inactivation rate A Dropout layer of (0.3 to 0.5) generates mask features. (in This operation is used to suppress overfitting during model training and is forced to remain on during inference to support the quantification of local uncertainties in the macula.

[0032] Fifth stage: Classification probability output. Finally, the masked features are processed through a fully connected classification layer, and the independent classification probability of local macular abnormalities is output using the Softmax activation function. The specific calculations are consistent with the fifth stage of the global expert analysis.

[0033] The multi-expert feature extraction module feeds the feature map output from the second stage of the global expert network and the feature vectors output from the third stage of the visual disc and macular expert networks into the multi-expert feature fusion module.

[0034] The multi-expert feature fusion module processes the feature maps and feature vectors input from the multi-expert feature extraction module. The process involves multiple modules, including convolution, spatial broadcasting, feature concatenation, and channel attention, to output a fused feature vector. Specific processing steps include: dimensionality unification: using... Convolutional layers map global features to a graph Visual disc feature vector and macular feature vector Projected to the same dimension Size unification: A broadcast mechanism is used to spatially align the feature vectors of the visual disc and macular experts with the feature map of the global expert; Feature concatenation: The three projected feature maps are concatenated along the channel dimension to obtain combined features. Attention weighting: combining features The input is fed into the channel attention module, which learns the dependencies between channels through its fully connected layers, generating weight coefficients for... Perform recalibration.

[0035] Specifically, such as Figure 3 As shown, the feature fusion module includes the following modules: (1) Convolutional layer: The one-dimensional feature vectors output by the global average pooling layers of the visual disc expert and macular expert, and the three-dimensional feature map output by the end of the global expert backbone network, are respectively processed by... The convolutional layers align their channel counts to a preset dimension of 256. (2) Spatial broadcasting: For the channel-aligned visual disk and macula feature vectors, the spatial broadcasting mechanism is used to repeatedly copy them in the spatial dimension and expand them to the same spatial size as the global feature map.

[0036] (3) Feature stitching: Stack and stitch three feature maps of uniform size and channel along the channel dimension to generate a combined feature tensor. At this point, different channel segments of the tensor contain feature information of the global structure, the local area of ​​the visual disk, and the local area of ​​the macula, respectively. (4) Channel attention module: the concatenated combined feature tensor The data is fed into the channel attention module (SE-Block) to adaptively learn and assign weights to different anatomical region features, specifically including the following sub-steps: Squeeze: First of all Perform a global average pooling operation along the spatial dimension to compress it into a single... The channel descriptor vector is used to obtain the channel energy distribution in the global view.

[0037] Excitation: The channel descriptor vector is input into a structure consisting of two fully connected layers, and then a sigmoid activation function is applied to output a weight vector with the same size as the number of channels. (size is) (The value is between 0 and 1). This weight coefficient vector The relative importance of global, optic disc, and macular features in current image diagnostics was precisely quantified.

[0038] Scale: Using the weight coefficient vector For the original combined feature tensor Channel-level element-wise multiplication is performed to generate a recalibrated fused feature map. Finally, the fused feature map is passed through a global average pooling layer to output the final fused feature vector for use by subsequent classifiers.

[0039] The feature fusion module feeds the output fused feature vector to the subsequent Dropout layer and fully connected classification layer.

[0040] To overcome the limitations of point estimation in traditional deep neural networks due to the single deterministic forward propagation, and to assess the reliability of the features output by the multi-expert feature extraction and fusion modules, this invention designs a dedicated uncertainty quantification module. The specific implementation logic of this module is as follows: (1) Construction of random auxiliary classification head: At the end of each independent expert network and at the end of the feature fusion module, a "random auxiliary classification head" is constructed. This classification head contains a Dropout layer and a fully connected classification layer connected in sequence.

[0041] (2) Monte Carlo Random Inference: Unlike traditional deep learning inference models, this module forces all Dropout layers in the classification heads to remain active during the inference phase after model deployment. For each input fundus image sample, the system controls the four classification heads to execute simultaneously. Second-rate( Random forward propagation.

[0042] (3) Decoupling calculation of probability and uncertainty: Assume that in a certain classification head, the first The probability of the next forward transmission being classified as "positive for drug use" is: Then the final predicted probability for The average of the results: At the same time, calculate the variance of the predicted distribution. As the uncertainty score of the model: (4) Multi-granularity risk output: Through the above mechanism, this module not only outputs the global comprehensive prediction probability and uncertainty score based on fusion features, but also simultaneously calculates the independent anomaly probability and uncertainty score of the global geometry, optic disc region, and macular region. This decoupled design enables the system to accurately trace which specific anatomical region caused the low confidence of the overall detection, thereby providing multi-dimensional data support and consistency verification benchmark for the subsequent hierarchical decision-making and report generation modules.

[0043] Quantitative evaluation and consistency verification output module: setting uncertainty threshold and feature activation threshold =0.5, these two thresholds are used to evaluate the reliability of the model's output data and output structured auxiliary analysis data; specifically, this module includes two sub-modules: data reliability early warning and multi-branch feature consistency verification. (1) Data reliability early warning logic: such as Figure 4 As shown, the system does not output qualitative diagnostic conclusions, but rather based on predicted probabilities. and variance Provide quantitative data status assessment: Scenario A (Data Unreliable): If ,regardless Regardless of the value, the system classifies the current sample as "low-reliability data." This indicates that the input image may contain blur, artifacts, or belong to a distribution not seen by the model. In this case, the system automatically triggers a data warning and outputs a message suggesting "re-acquire the image or transfer it for manual review."

[0044] Scenario B (Data is reliable and features are significant): If and The system determines that the data extraction process is reliable and that the target's abnormal features respond strongly. The system outputs a quantitative indicator of high feature matching degree and records the corresponding probability value.

[0045] Case C (Data is reliable and features are not activated): If and The system determines that the data extraction is reliable and no significant abnormal features of the target are detected, and directly outputs the basic probability value.

[0046] (2) Multi-branch feature consistency verification and data output: In order to assist professionals in the final manual interpretation, the system performs spatial consistency verification on the output results of the feature fusion branch and the single expert branch: First level: The system directly outputs the probability of the feature fusion branch. and variance As an auxiliary reference benchmark at the whole map level.

[0047] The second step: The system compares the fusion branch result with the feature activation states of the three independent expert branches. If the fusion branch... If the predicted probability of a single expert branch exceeds the feature activation threshold even though it has not been exceeded, the system determines that a "local feature space conflict" has occurred. At this point, the system forcibly triggers a "local feature conflict warning," using the anatomical attributes of the independent branch for spatial source tracing, directly outputting the specific anatomical label causing the conflict, along with the probabilities of each branch. and uncertainty The numerical values ​​are packaged into structured data output and handed over to the operator for final manual review and adjudication.

[0048] Example 2 This invention also provides a multi-expert collaborative toxicology detection method based on fundus image anatomical prior and uncertainty quantification. The method is implemented using the system described in Example 1, and includes: The raw fundus images are preprocessed to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map; Based on the multi-source input data stream, a multi-expert convolutional neural network model containing three parallel sub-networks is constructed to extract features from the preprocessed global fundus map, optic disc ROI map, and macular ROI map. The extracted features are fused to generate a fused feature vector; Based on the fused feature vectors, the classification probability and uncertainty score are obtained using Bayesian approximation. The classification results are obtained based on the classification logic, and the final classification report and the reports of each expert are generated based on the classification results of multiple experts and the classification results of the fusion branch.

[0049] In this embodiment, the method for preprocessing the raw fundus image to construct a multi-source input data stream including a global fundus map, an optic disc ROI map, and a macular ROI map includes: Acquire raw fundus images; Preprocessing operations such as effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization are performed on the original fundus image; The lightweight object detection network YoLov5 is used to generate bounding boxes for the optic disc and macula on the preprocessed image. The ROI image is then cropped on the original fundus image after the effective region is extracted based on the bounding box coordinates. The cropped ROI image is subjected to size normalization, adaptive enhancement, noise reduction, and statistical normalization to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map.

[0050] In this embodiment, the multi-expert convolutional neural network model includes: a global expert network, a visual disc expert network, and a macular expert network; A global expert network is used to receive global fundus images and extract retinal vessel distribution, arteriovenous diameter ratio, and overall texture features through a deep convolutional network. The optic disc expert network is used to receive optic disc ROI images and extract optic nerve color pallor, disc rim morphology, and optic cup-optic disc ratio features. The Macular Expert Network is used to receive macular ROI images and extract features of macular edema, pigmentary disorders, and microaneurysms.

[0051] In this embodiment, the method for fusing the extracted features to generate a fused feature vector includes: The extracted features are mapped to a unified channel dimension, the feature sizes are aligned using a spatial broadcasting mechanism, and the interaction weights between local and global features are calculated using a channel attention mechanism, ultimately generating a fused feature vector that includes anatomical priors.

[0052] In this embodiment, the method for obtaining the classification probability and uncertainty score based on the fused feature vector using Bayesian approximation includes: Construct a fully connected classifier containing a Dropout layer and input the fused feature vector into the classifier; keep the Dropout layer active during the testing phase and perform N random forward propagation operations on the same input sample to obtain N sets of predicted probability distributions; calculate the mean of the N sets of predicted probabilities as the detection result, and calculate the variance of the N sets of predicted probabilities as the uncertainty score of the detection result.

[0053] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A multi-expert collaborative toxicology detection system based on anatomical priors and uncertainty quantification of fundus images, characterized in that, The system includes: a data acquisition and preprocessing module, a multi-expert feature extraction module, a multi-expert feature fusion module, an uncertainty quantification module, and a hierarchical decision-making and report generation module; The data acquisition and preprocessing module is used to preprocess the raw fundus image and construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map; The multi-expert feature extraction module is used to construct a multi-expert convolutional neural network model containing three parallel sub-networks based on the multi-source input data stream, and to extract features from the preprocessed global fundus map, optic disc ROI map and macular ROI map; The multi-expert feature fusion module is used to fuse the extracted features to generate a fused feature vector. The uncertainty quantification module is used to obtain the classification probability and uncertainty score based on the fused feature vector using Bayesian approximation. The hierarchical decision and report generation module is used to obtain hierarchical results based on hierarchical decision logic, and generate a final hierarchical report and reports from each expert based on the hierarchical results of multiple experts and the hierarchical results of the fusion branch.

2. The system according to claim 1, characterized in that, The process of preprocessing raw fundus images to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map includes: Acquire raw fundus images; Preprocessing operations such as effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization are performed on the original fundus image; The lightweight object detection network YoLov5 is used to generate bounding boxes for the optic disc and macula on the preprocessed image. The ROI image is then cropped on the original fundus image after the effective region is extracted based on the bounding box coordinates. The cropped ROI image is subjected to size normalization, adaptive enhancement, noise reduction, and statistical normalization to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map.

3. The system according to claim 1, characterized in that, The multi-expert convolutional neural network model includes: a global expert network, a visual disc expert network, and a macular expert network; The global expert network is used to receive the global fundus image and extract the retinal vessel distribution, arteriovenous diameter ratio and overall texture features through a deep convolutional network. The optic disc expert network is used to receive optic disc ROI images and extract optic nerve color pallor, disc rim morphology, and optic cup-optic disc ratio features. The macular expert network is used to receive macular ROI images and extract features of macular edema, pigmentary disorders, and microaneurysms.

4. The system according to claim 1, characterized in that, The process of fusing the extracted features to generate a fused feature vector includes: The extracted features are mapped to a unified channel dimension, the feature sizes are aligned using a spatial broadcasting mechanism, and the interaction weights between local and global features are calculated using a channel attention mechanism, ultimately generating a fused feature vector that includes anatomical priors.

5. The system according to claim 1, characterized in that, The process of obtaining the classification probability and uncertainty score based on the fused feature vector and using Bayesian approximation includes: Construct a fully connected classifier containing a Dropout layer and input the fused feature vector into the classifier; keep the Dropout layer active during the testing phase and perform N random forward propagation operations on the same input sample to obtain N sets of predicted probability distributions; calculate the mean of the N sets of predicted probabilities as the detection result, and calculate the variance of the N sets of predicted probabilities as the uncertainty score of the detection result.

6. A multi-expert collaborative toxicology detection method based on fundus image anatomical prior and uncertainty quantification, wherein the method is implemented by the system described in any one of claims 1-5, characterized in that, The method includes: The raw fundus images are preprocessed to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map; Based on the multi-source input data stream, a multi-expert convolutional neural network model containing three parallel sub-networks is constructed to extract features from the preprocessed global fundus map, optic disc ROI map, and macular ROI map. The extracted features are fused to generate a fused feature vector; Based on the fused feature vectors, the classification probability and uncertainty score are obtained using Bayesian approximation. The classification results are obtained based on the classification logic, and the final classification report and the reports of each expert are generated based on the classification results of multiple experts and the classification results of the fusion branch.

7. The method according to claim 6, characterized in that, Methods for preprocessing raw fundus images to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map include: Acquire raw fundus images; Preprocessing operations such as effective region extraction, size normalization, adaptive enhancement, noise reduction, and statistical normalization are performed on the original fundus image; The lightweight object detection network YoLov5 is used to generate bounding boxes for the optic disc and macula on the preprocessed image. The ROI image is then cropped on the original fundus image after the effective region is extracted based on the bounding box coordinates. The cropped ROI image is subjected to size normalization, adaptive enhancement, noise reduction, and statistical normalization to construct a multi-source input data stream containing a global fundus map, an optic disc ROI map, and a macular ROI map.

8. The method according to claim 6, characterized in that, The multi-expert convolutional neural network model includes: a global expert network, a visual disc expert network, and a macular expert network; The global expert network is used to receive the global fundus image and extract the retinal vessel distribution, arteriovenous diameter ratio and overall texture features through a deep convolutional network. The optic disc expert network is used to receive optic disc ROI images and extract optic nerve color pallor, disc rim morphology, and optic cup-optic disc ratio features. The macular expert network is used to receive macular ROI images and extract features of macular edema, pigmentary disorders, and microaneurysms.

9. The method according to claim 6, characterized in that, Methods for fusing extracted features to generate a fused feature vector include: The extracted features are mapped to a unified channel dimension, the feature sizes are aligned using a spatial broadcasting mechanism, and the interaction weights between local and global features are calculated using a channel attention mechanism, ultimately generating a fused feature vector that includes anatomical priors.

10. The method according to claim 6, characterized in that, Methods for obtaining classification probabilities and uncertainty scores based on fused feature vectors and using Bayesian approximation include: Construct a fully connected classifier containing a Dropout layer and input the fused feature vector into the classifier; keep the Dropout layer active during the testing phase and perform N random forward propagation operations on the same input sample to obtain N sets of predicted probability distributions; calculate the mean of the N sets of predicted probabilities as the detection result, and calculate the variance of the N sets of predicted probabilities as the uncertainty score of the detection result.

Images