An AI multi-modal fusion-based gynecological microecological detection system

Abstract

The application discloses a gynecological micro-ecological detection system based on AI multi-modal fusion, and belongs to the technical field of gynecological micro-ecological detection. The system comprises a data acquisition module, a standardization module and a micro-ecological output unit. The data acquisition module acquires immunofluorescence signals, gram staining morphological characteristics and dry chemical indexes. The data acquisition module is connected to the standardization module. The standardization module comprises a pre-processing unit for pre-processing collected data. The standardization module is connected with a data classification unit for data classification. The data classification unit is connected to a heterologous data normalization processing unit. The gynecological micro-ecological detection system based on AI multi-modal fusion can realize accurate evaluation of gynecological micro-ecology, construct a dynamic model of microbial community parameters, realize inflammation process quantification, generate a visual micro-ecological atlas through three-dimensional space mapping, and output inflammation types and overall micro-ecological evaluation.

Description

Technical Field

[0001] This invention specifically relates to a gynecological microecological detection system based on AI multimodal fusion, belonging to the field of gynecological microecological detection technology. Background Technology

[0002] The vaginal microecology evaluation system comprises two parts: morphological detection and functional detection. Gram staining of vaginal secretion specimens using conventional optical microscopy is a crucial method for morphological detection. For example, Chinese Patent Publication No. CN112966645B discloses an intelligent detection and classification counting method for multiple types of bacilli in gynecological microecology. This method can effectively improve the detection rate of different types of bacilli and reduce the false negative rate, thus increasing the efficiency of generating biological specimen reports. However, existing gynecological microecology detection methods rely on single technical means (such as Gram staining microscopy or dry chemical test strips), which have the following limitations:

[0003] In the field of gynecological microecological testing, single testing methods have limitations and cannot comprehensively and accurately reflect the true state of the vaginal microecology, resulting in the following problems:

[0004] Limited data dimensions: It is impossible to simultaneously acquire microbial morphological characteristics (such as the ratio of Gram-positive to Gram-negative bacteria), molecular-level signals (such as fluorescently labeled antigens), and metabolites (such as hydrogen peroxide concentration).

[0005] Inconsistent dimensions: Data such as fluorescence signal intensity (unit: RFU), bacterial density (unit: CFU / mL), and pH value (dimensionless) are difficult to directly integrate and analyze;

[0006] Limitations of static assessment: The gynecological microecological status is a dynamic process. Currently, most gynecological microecological test data are static analyses. Semi-quantitative methods such as Nugent score and AV score only reflect the status at a certain point in time and cannot capture the dynamic changes of the microbial community.

[0007] Insufficient visualization: The lack of intuitive presentation of the spatial distribution characteristics of the microecology affects the efficiency of clinical decision-making. Summary of the Invention

[0008] To address the aforementioned issues, this invention proposes a gynecological microecological detection system based on AI multimodal fusion. This system enables precise assessment of the gynecological microecology, constructs a dynamic model of microbial community parameters to quantify the inflammatory process, and generates a visualized microecological map through three-dimensional spatial mapping, outputting the inflammation type and overall microecological evaluation.

[0009] The gynecological microecological detection system based on AI multimodal fusion of the present invention includes:

[0010] The data acquisition module acquires immunofluorescence signals, Gram staining morphological characteristics, and dry chemical indicators. Immunofluorescence signals are obtained using immunofluorescence methods, such as detecting the fluorescence signal intensity of mycoplasma / chlamydia (0-1000 RFU) using 488 nm excitation light and 525 nm emission light. Gram staining morphological characteristics are obtained using Gram staining methods, such as counting Gram-positive bacteria (e.g., lactobacilli) and Gram-negative bacteria (e.g., Gardnerella vaginalis) under a microscope (1000× oil immersion) and calculating the ratio (accuracy 0.01). Dry chemical indicators are obtained using dry chemical enzymatic methods, such as detecting pH value (3.5-5.5, resolution 0.01), sialidase activity (0-10 U / mL), and hydrogen peroxide (0-50 μmol / L, detection limit 0.1 μmol / L).

[0011] The standardization module includes a preprocessing unit for preprocessing the collected data and a data classification unit connected to the preprocessing unit for data classification. The data classification unit is connected to a heterogeneous data normalization processing unit, which performs normalization processing on the collected data. The heterogeneous data normalization processing unit is connected to a data storage unit. After acquiring the collected data, the data acquisition module sends it to the preprocessing unit for preprocessing. After preprocessing, the preprocessed data is classified by the data classification unit, or it can be directly sent to the data classification unit for classification without preprocessing. After classification, the heterogeneous data normalization processing unit sends the corresponding classified data to the corresponding normalization model. After processing, the original data and normalized data are sent to the data storage unit.

[0012] The microecological output unit is communicatively connected to the data storage unit; the microecological output unit includes a diagnostic result output unit, a dynamic analysis module for constructing a time series model of microbial community parameters, and a visualization module for generating a three-dimensional microecological map.

[0013] Furthermore, the data classification unit classifies the data acquired by the data acquisition module, assigning a unique correlation factor to each category of data. This correlation factor is connected to a factor library. The classified data includes percentage data, linear data, and non-linear data. The heterogeneous data normalization processing unit includes:

[0014] The first normalization processing model for accessing percentage data is: normalizing the percentage to [0,1].

[0015] The second normalized processing model for linear data:

[0016] ;

[0017] in, For the normalization result of linear data, This refers to linear data acquired by the data acquisition module. for Minimum value corresponding to the data type for Maximum value corresponding to the data type;

[0018] The third normalization processing model for nonlinear data uses the Sigmoid function mapping. ;

[0019] in, For the normalization result of nonlinear data, For the slope parameter, This refers to a specific nonlinear data acquired by the data acquisition module. for The midpoint value of the corresponding data type;

[0020] After completing the single-dimensional normalization, the heterogeneous data normalization processing unit assigns different weights to the data collected by the data acquisition module based on its contribution to the micro-ecological assessment, resulting in a comprehensive normalized value, as follows:

[0021] Multi-dimensional weighted fusion model for comprehensive normalized value calculation:

[0022] , ;

[0023] in, This represents the normalized value of the immunofluorescence signal. These are the normalized values ​​for Gram staining morphological characteristics. Normalized values ​​for dry chemical indicators For immunofluorescence signal weights, Weights for orchid staining morphological features. The weights are for dry chemical indicators; among which, , and Weights are determined through principal component analysis (PCA), such as It is 0.4. It is 0.3. It is 0.3.

[0024] Furthermore, the preprocessing unit includes:

[0025] Immunofluorescence signal preprocessing: An adaptive threshold valve is used to remove background noise from the immunofluorescence signal. The threshold is the mean plus 2 times the standard deviation, while retaining the effective signal region.

[0026] Gram staining morphological features preprocessing: The Gram staining morphological features were converted to grayscale, then edge detection was performed using the Canny operator, and then the U-Net model was used to extract the cell contours from the edge detection data.

[0027] Dry chemical index preprocessing: removing outliers and data that are outside the linear range.

[0028] Furthermore, the dynamic analysis module is specifically as follows:

[0029] The dynamic analysis module acquires dynamic data of a specific object within a selected stage through a data storage unit. This dynamic data includes the community density corresponding to each time series. CFU / mL (obtained by serial dilution plate counting method); Colony diversity index , For the number of bacterial species, For the first The percentage of specific bacteria; the percentage of dominant bacteria, such as lactobacilli, in the total bacteria (0-100%), and the calculation of the inflammation risk index, which ranges from 0-10, as detailed below:

[0030] ;

[0031] in, , and The weights are optimized through principal component analysis to reflect the contribution of each parameter to inflammation, and can be trained and optimized through clinical data. As community density weight, =0.5; As the weight of the colony diversity index, =0.3; Weighting based on the proportion of dominant bacteria. =0.2; Community density, Taking the logarithm of the community density linearizes the exponential growth. As a colony diversity index, To transform diversity into a measure of deviation from a healthy state; The percentage of dominant bacteria, The percentage of non-dominant bacteria indicates an abnormal bacterial community structure.

[0032] Based on dynamic data and inflammation risk index, an LSTM neural network is used to construct a time series model. The input is the dynamic data under the time series, and the output is the inflammation risk index.

[0033] Community density (unit: CFU / mL), measured value CFU / mL; therefore, the community density term is obtained: Taking the logarithm (log) linearizes the exponentially growing data, preventing high-density values ​​from dominating the results.

[0034] The calculation process for the community density term is as follows: High-density microbial community ( The CFU / mL count may indicate excessive pathogen proliferation and is positively correlated with inflammatory risk; after logarithmic transformation, for every 10-fold increase in density, this contribution increases by 0.5 points.

[0035] The colony diversity index is 4.2, with an actual measured value of 4.2; in the formula, To convert diversity into a measure of deviation from a healthy state, since healthy vaginas typically have high diversity; the calculation process is: 5 - 4.2 = 0.8; 0.3 * 0.8 = 0.24; decreased diversity (e.g., H(t) = 4.2) may reflect a flora imbalance (e.g., reduced lactobacilli, increased anaerobic bacteria), which is associated with bacterial vaginosis; for every 1 unit decrease in diversity, the risk increases by 0.3 points;

[0036] Percentage of dominant bacteria: ;like The percentage of lactobacilli was 0.2% (measured value). The percentage of "non-lactobacteria" reflects an abnormal flora structure; the calculation process is as follows: 1 - 0.2 = 0.8; 0.2 * 0.8 = 0.16; lactobacilli are the dominant bacteria in vaginal health, and a decrease in their proportion (such as 20%) indicates flora imbalance, which may be accompanied by pathogen colonization; for every 10% decrease in lactobacilli in this item, the risk increases by 0.02 points.

[0037] ;

[0038] Risk classification: 4.4 points is considered mild inflammation (threshold setting: ≥6 points is severe, 3-5 points is mild).

[0039] Furthermore, the visualization module includes a cross-modal feature fusion network, the output of which is connected to the input of the visualization processing module;

[0040] The modality feature fusion network includes three sets of input branches, specifically:

[0041] Branch 1: 3D-ResNet processing of fluorescent antigen distribution (128×128×64 voxels);

[0042] Branch 2: CNN+LSTM processing of Gram staining time series images;

[0043] Branch 3: Fully connected networks process enzyme metabolism indicators;

[0044] Fusion layer: After feature concatenation, the features are weighted by a self-attention mechanism to output a 256-dimensional fusion vector;

[0045] After acquiring the output data of the fusion layer, the visualization processing module generates a visualization output through a three-dimensional microecological map; specifically, it includes red areas: pathogen enrichment areas; blue areas: lactobacillus dominant areas; and green flow trajectory: dynamic changes in H2O2 concentration.

[0046] Furthermore, the visualization module includes feature extraction and three-dimensional spatial mapping, as detailed below:

[0047] The feature extraction is as follows:

[0048] Fluorescence image: The antigen distribution region was segmented using a U-Net network, and the centroid coordinates (x1, y1) were extracted.

[0049] Gram-stained images: Bacterial cell morphology features (length, width, texture) were extracted using ResNet50.

[0050] Metabolites: Sialidase activity is converted into three-dimensional z-axis coordinates: z = activity / 10, and the activity is normalized to [0,1].

[0051] After the feature extraction output data is obtained, a coordinate system is constructed through three-dimensional spatial mapping:

[0052] X-axis: Abscissa of fluorescence image (0-1024 pixels);

[0053] Y-axis: Vertical axis of Gram staining morphology image (0-1024 pixels);

[0054] Z-axis: Normalized value of metabolite activity (0-1);

[0055] A three-dimensional surface model was generated using the Marching Cubes algorithm to visualize the distribution and metabolic activity of the microbial community.

[0056] Furthermore, the diagnostic result output unit is connected to the diagnostic result output library, which contains first correlation data between the comprehensive normalized value and the overall microecological evaluation; and correlation data between each inflammation type and the threshold range of each single-dimensional normalized value. The diagnostic result output unit obtains the single-dimensional normalized value and the comprehensive normalized value, retrieves the diagnostic result output library according to the input data, and outputs the overall microecological evaluation and inflammation type.

[0057] Furthermore, the overall microecological evaluation includes overall cleanliness, Nugent score, microbial density, proportion of lactobacilli, and proportion of other miscellaneous bacteria.

[0058] Compared with existing technologies, the gynecological microecological detection system based on AI multimodal fusion of this invention can achieve accurate assessment of gynecological microecology: It establishes a unified data standard for heterogeneous data such as immunofluorescence, Gram staining, and dry chemical enzymatic methods; develops a multi-parameter normalization algorithm; and constructs a dynamic model of microbial community parameters to quantify the inflammatory process. Furthermore, it generates a visualized microecological map through three-dimensional spatial mapping and outputs the inflammation type and overall microecological evaluation. Specifically, by constructing a unified representation framework for multimodal data, it breaks down the barriers between data, allowing different modalities to complement and verify each other, thereby more comprehensively and accurately representing the state of the gynecological microecology. By analyzing the biological characteristics and pathological changes of gynecological microecological samples, an intelligent identification and diagnostic model based on multimodal data is constructed. Deep learning algorithms are used for pattern recognition and classification to achieve accurate diagnosis of gynecological microecological status, particularly improving the accuracy of identifying complex cases such as mixed infections and early lesions. Finally, a dynamic analysis model for gynecological microecological testing data is established. By analyzing testing data at different time points and treatment stages, dynamic change patterns in the data are mined, providing clinicians with more targeted treatment plans and prognostic assessments. This transforms the service from simple disease diagnosis to a full-process service encompassing disease prevention, treatment monitoring, and prognostic assessment. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the overall structure of the gynecological microecological detection system based on AI multimodal fusion of the present invention.

[0060] Figure 2 This is a schematic diagram of the single-dimensional normalization and comprehensive normalization workflow of the present invention.

[0061] Figure 3 This is a schematic diagram of the workflow of the pre-processing unit of the present invention.

[0062] Figure 4 This is a schematic diagram of the workflow of the dynamic analysis module of the present invention.

[0063] Figure 5 This is a schematic diagram of the workflow of the visualization module of the present invention.

[0064] Figure 6 This is another schematic diagram of the workflow of the visualization module of the present invention. Detailed Implementation

[0065] like Figures 1 to 6 The gynecological microecological detection system based on AI multimodal fusion shown includes:

[0066] The data acquisition module acquires immunofluorescence signals, Gram staining morphological characteristics, and dry chemical indicators. Immunofluorescence signals are obtained using immunofluorescence methods, such as detecting the fluorescence signal intensity of mycoplasma / chlamydia (0-1000 RFU) using 488 nm excitation light and 525 nm emission light. Gram staining morphological characteristics are obtained using Gram staining methods, such as counting Gram-positive bacteria (e.g., lactobacilli) and Gram-negative bacteria (e.g., Gardnerella vaginalis) under a microscope (1000× oil immersion) and calculating the ratio (accuracy 0.01). Dry chemical indicators are obtained using dry chemical enzymatic methods, such as detecting pH value (3.5-5.5, resolution 0.01), sialidase activity (0-10 U / mL), and hydrogen peroxide (0-50 μmol / L, detection limit 0.1 μmol / L).

[0067] The standardization module includes a preprocessing unit for preprocessing the collected data and a data classification unit connected to the preprocessing unit for data classification. The data classification unit is connected to a heterogeneous data normalization processing unit, which performs normalization processing on the collected data. The heterogeneous data normalization processing unit is connected to a data storage unit. After acquiring the collected data, the data acquisition module sends it to the preprocessing unit for preprocessing. After preprocessing, the preprocessed data is classified by the data classification unit, or it can be directly sent to the data classification unit for classification without preprocessing. After classification, the heterogeneous data normalization processing unit sends the corresponding classified data to the corresponding normalization model. After processing, the original data and normalized data are sent to the data storage unit.

[0068] The microecological output unit is communicatively connected to the data storage unit; the microecological output unit includes a diagnostic result output unit, a dynamic analysis module for constructing a time series model of microbial community parameters, and a visualization module for generating a three-dimensional microecological map.

[0069] The data classification unit classifies the data acquired by the data acquisition module, assigning a unique correlation factor to each category of data. This correlation factor is connected to a factor library. The categorized data includes percentage data, linear data, and non-linear data. The heterogeneous data normalization processing unit includes:

[0070] The first normalization processing model for accessing percentage data is: normalizing the percentage to [0,1].

[0071] The second normalized processing model for linear data:

[0072] ;

[0073] in, For the normalization result of linear data, This refers to linear data acquired by the data acquisition module. for Minimum value corresponding to the data type for Maximum value corresponding to the data type;

[0074] The third normalization processing model for nonlinear data uses the Sigmoid function mapping. ;

[0075] in, For the normalization result of nonlinear data, For the slope parameter, This refers to a specific nonlinear data acquired by the data acquisition module. for The midpoint value of the corresponding data type;

[0076] After completing the single-dimensional normalization, the heterogeneous data normalization processing unit assigns different weights to the data collected by the data acquisition module based on its contribution to the micro-ecological assessment, resulting in a comprehensive normalized value, as follows:

[0077] Multi-dimensional weighted fusion model for comprehensive normalized value calculation:

[0078] , ;

[0079] in, This represents the normalized value of the immunofluorescence signal. These are the normalized values ​​for Gram staining morphological characteristics. Normalized values ​​for dry chemical indicators For immunofluorescence signal weights, Weights for orchid staining morphological features. The weights are for dry chemical indicators; among which, , and Weights are determined through principal component analysis (PCA), such as It is 0.4. It is 0.3. It is 0.3.

[0080] The preprocessing unit includes:

[0081] Immunofluorescence signal preprocessing: An adaptive threshold valve is used to remove background noise from the immunofluorescence signal. The threshold is the mean plus 2 times the standard deviation, while retaining the effective signal region.

[0082] Gram staining morphological features preprocessing: The Gram staining morphological features were converted to grayscale, then edge detection was performed using the Canny operator, and then the U-Net model was used to extract the cell contours from the edge detection data.

[0083] Dry chemical index preprocessing: removing outliers and data that are outside the linear range.

[0084] The dynamic analysis module is as follows:

[0085] The dynamic analysis module acquires dynamic data of a specific object within a selected stage through a data storage unit. This dynamic data includes the community density corresponding to each time series. CFU / mL (obtained by serial dilution plate counting method); Colony diversity index , For the number of bacterial species, For the first The percentage of specific bacteria; the percentage of dominant bacteria, such as lactobacilli, in the total bacteria (0-100%), and the calculation of the inflammation risk index, which ranges from 0-10, as detailed below:

[0086] ;

[0087] in, , and The weights are optimized through principal component analysis to reflect the contribution of each parameter to inflammation, and can be trained and optimized through clinical data. As community density weight, =0.5; As the weight of the colony diversity index, =0.3; Weighting based on the proportion of dominant bacteria. =0.2; Community density, Taking the logarithm of the community density linearizes the exponential growth. As a colony diversity index, To transform diversity into a measure of deviation from a healthy state; The percentage of dominant bacteria, The percentage of non-dominant bacteria indicates an abnormal bacterial community structure.

[0088] Based on dynamic data and inflammation risk index, an LSTM neural network is used to construct a time series model. The input is the dynamic data under the time series, and the output is the inflammation risk index.

[0089] Community density (unit: CFU / mL), measured value CFU / mL; therefore, the community density term is obtained: Taking the logarithm (log) linearizes the exponentially growing data, preventing high-density values ​​from dominating the results.

[0090] The calculation process for the community density term is as follows: High-density microbial community ( The CFU / mL count may indicate excessive pathogen proliferation and is positively correlated with inflammatory risk; after logarithmic transformation, for every 10-fold increase in density, this contribution increases by 0.5 points.

[0091] The colony diversity index is 4.2, with an actual measured value of 4.2; in the formula, To convert diversity into a measure of deviation from a healthy state, since healthy vaginas typically have high diversity; the calculation process is: 5 - 4.2 = 0.8; 0.3 * 0.8 = 0.24; decreased diversity (e.g., H(t) = 4.2) may reflect a flora imbalance (e.g., reduced lactobacilli, increased anaerobic bacteria), which is associated with bacterial vaginosis; for every 1 unit decrease in diversity, the risk increases by 0.3 points;

[0092] Percentage of dominant bacteria: ;like The percentage of lactobacilli was 0.2% (measured value). The percentage of "non-lactobacteria" reflects an abnormal flora structure; the calculation process is as follows: 1 - 0.2 = 0.8; 0.2 * 0.8 = 0.16; lactobacilli are the dominant bacteria in vaginal health, and a decrease in their proportion (such as 20%) indicates flora imbalance, which may be accompanied by pathogen colonization; for every 10% decrease in lactobacilli in this item, the risk increases by 0.02 points.

[0093] ;

[0094] Risk classification: 4.4 points is considered mild inflammation (threshold setting: ≥6 points is severe, 3-5 points is mild).

[0095] The visualization module includes a cross-modal feature fusion network, and the output of the cross-modal feature fusion network is connected to the input of the visualization processing module.

[0096] The modality feature fusion network includes three sets of input branches, specifically:

[0097] Branch 1: 3D-ResNet processing of fluorescent antigen distribution (128×128×64 voxels);

[0098] Branch 2: CNN+LSTM processing of Gram staining time series images;

[0099] Branch 3: Fully connected networks process enzyme metabolism indicators;

[0100] Fusion layer: After feature concatenation, the features are weighted by a self-attention mechanism to output a 256-dimensional fusion vector;

[0101] After acquiring the output data of the fusion layer, the visualization processing module generates a visualization output through a three-dimensional microecological map; specifically, it includes red areas: pathogen enrichment areas; blue areas: lactobacillus dominant areas; and green flow trajectory: dynamic changes in H2O2 concentration.

[0102] I. The specific network architecture and processing flow of the cross-modal feature fusion network are as follows:

[0103] 1.1 Multimodal input branch:

[0104] Branch 1 (3D-ResNet path): Input specifications: 3D fluorescent antigen distribution data of 128×128×64 voxels; Network structure: 5 layers of 3D residual blocks, each layer contains: 3D convolution kernel (3×3×3); Batch normalization LeakyReLU (α=0.01); Output: 64-dimensional feature vector, capturing spatial distribution patterns;

[0105] Branch 2 (CNN+LSTM path): Input specifications: Gram-stained temporal image (128×128×T, T=15 time points); Processing flow: 2D-CNN extracts spatial features (VGG-style architecture); after temporal expansion, it is input into a bidirectional LSTM, and the attention mechanism weights the features at each time point; Output: 128-dimensional temporal feature vector;

[0106] Branch 3 (Metabolic Indicator Path): Input: 12-dimensional enzyme metabolic indicators (including ATPase, LDH, etc.); Network structure: 3 fully connected layers (256→128→64); Feature enhancement: Activated using exponential linear units (ELU);

[0107] 1.2 Cross-modal fusion layer:

[0108] Feature splicing: 64 + 128 + 64 = 256-dimensional joint features;

[0109] Self-attention mechanism: Query-key projection dimension: 64; Multi-head attention (4 heads); Position encoding: 3D relative position encoding; Feature weighting: Adjusting the contribution of each modality through a sigmoid gating mechanism;

[0110] II. Generation of 3D Micro-ecological Maps:

[0111] 2.1 Spatial Mapping Algorithm:

[0112] Coordinate transformation: Map the feature vectors to a 3D mesh (64×64×32);

[0113] Probability field generation: Pathogen distribution: Sigmoid(W_p·F); Microbial community composition: Softmax(W_b·F); Metabolic dynamics: LSTM decoder;

[0114] 2.2 Dynamic Visualization Specifications:

[0115] Table 1, Color Coding Standard Table:

[0116]

[0117] Color coding is performed according to the color coding standard table, and rendering parameters are processed accordingly:

[0118] Volume rendering: Ray casting algorithm;

[0119] Dynamic trajectory: B-spline curve interpolation;

[0120] Transparency control: α = 0.7 × (signal strength / max).

[0121] The visualization module includes feature extraction and 3D spatial mapping, as detailed below:

[0122] The feature extraction is as follows:

[0123] Fluorescence image: The antigen distribution region was segmented using a U-Net network, and the centroid coordinates (x1, y1) were extracted.

[0124] Gram-stained images: Bacterial cell morphology features (length, width, texture) were extracted using ResNet50.

[0125] Metabolites: Sialidase activity is converted into three-dimensional z-axis coordinates: z = activity / 10, and the activity is normalized to [0,1].

[0126] After the feature extraction output data is obtained, a coordinate system is constructed through three-dimensional spatial mapping:

[0127] X-axis: Abscissa of fluorescence image (0-1024 pixels);

[0128] Y-axis: Vertical axis of Gram staining morphology image (0-1024 pixels);

[0129] Z-axis: Normalized value of metabolite activity (0-1);

[0130] A three-dimensional surface model was generated using the Marching Cubes algorithm to visualize the distribution and metabolic activity of the microbial community. For example, during the construction of the three-dimensional map, the distribution coordinates of the fluorescent antigen were (x=320, y=450); the sialidase activity was 2.5 U / mL → z=0.25; a three-dimensional point cloud map was generated, and high-risk areas were marked in red (z>0.5).

[0131] The diagnostic result output unit is connected to the diagnostic result output library, which contains first correlation data between the comprehensive normalized value and the overall microecological evaluation; and correlation data between each inflammation type and the threshold range of each single-dimensional normalized value. The diagnostic result output unit obtains the single-dimensional normalized value and the comprehensive normalized value, retrieves the diagnostic result output library according to the input data, and outputs the overall microecological evaluation and inflammation type.

[0132] The overall evaluation of the microbial ecosystem includes overall cleanliness, Nugent score, microbial density, proportion of lactobacilli and other miscellaneous bacteria.

[0133] Example:

[0134] The gynecological microecological detection system based on AI multimodal fusion of this invention addresses the significant differences in data types and dimensions generated by different detection methods (e.g., fluorescence signal intensity of 0-1000 RFU, pH value of 3.5-5.5, and G+ / G- bacteria ratio of 0-100%). These differences need to be mapped to a unified dimension (e.g., the [0,1] interval) to enable subsequent data fusion and correlation analysis. After normalization, the data retains its original distribution characteristics (e.g., nonlinear relationships), and the consistency error of the normalization results for different batches of data is <5%.

[0135] I. Data Classification and Feature Extraction:

[0136] The raw data is divided into three categories, and key features are extracted:

[0137] Immunofluorescence data: fluorescence signal intensity (e.g., mycoplasma / chlamydia labeled fluorescence, unit RFU), spatial distribution coordinates of fluorescently labeled antigen (x, y pixel values, spatial resolution 1 μm); Preprocessing: background noise removal (using adaptive thresholding method, threshold = mean + 2 times standard deviation), retaining effective signal regions (signal intensity > 50 RFU);

[0138] Gram staining data: Morphological features: number of Gram-positive bacteria, number of Gram-negative bacteria (counted by image segmentation algorithm, classification accuracy ≥95%), calculation of Gram-positive / Gram-negative bacteria ratio (accuracy 0.01); Preprocessing: image grayscale conversion → edge detection (Canny operator) → bacterial cell contour extraction (U-Net model segmentation);

[0139] Dry chemical enzymatic method data: Biochemical indicators: pH value (resolution 0.01), hydrogen peroxide concentration (unit μmol / L, detection limit 0.1 μmol / L), sialidase activity (unit U / mL, detection limit 0.01 U / mL); Pretreatment: removal of outliers (using the 3σ rule to remove data outside the linear range, such as samples with pH > 5.5 or < 3.5);

[0140] II. Data Validation Before Standardization:

[0141] Before classifying the three types of data, a normality test is performed (Shapiro-Wilk test, p>0.05 is considered to conform to a normal distribution). Non-normal data need to be logarithmically transformed (e.g., bacterial density: log10(CFU / mL)).

[0142] III. Multi-parameter normalization:

[0143] The weighted fusion normalization algorithm is adopted and is performed in two steps, as follows:

[0144] 3.1 Single-dimensional normalization:

[0145] Perform linear or nonlinear normalization on each data type separately, mapping it to the [0,1] interval:

[0146] Linear normalization (applicable to normally distributed data):

[0147] ;

[0148] For example, pH value (normal range 3.5-5.5); if the measured pH is 4.5, then It is 4.5. It is 3.5. It is 5.5; normalized value It is 0.5;

[0149] For example: Immunofluorescence assay data (normal range 0-1000 RFU), if the measured value is 580 RFU, then... It is 580. For 1000, =0; normalized value It is 0.58;

[0150] Nonlinear normalization (suitable for skewed data):

[0151] ;

[0152] Normalization of fluorescence signal intensity (0-1000 RFU) using the Sigmoid function enhances the discrimination of weak signals: for example, when fluorescence signal intensity = 800 RFU, It is 0.01. It is 800. It is 500; of which Value and The value is obtained by retrieving the factor library through the association factor and calculating it. ≈0.95;

[0153] 3.2 Multi-dimensional weighted fusion:

[0154] Based on the contribution of data to the micro-ecological assessment, different weights are assigned, and the single-dimensional normalization results are merged into a comprehensive normalized value:

[0155] Weight coefficient determination: Weights were calculated using the analytic hierarchy process (AHP) through training with clinical data (1000 samples).

[0156] Immunofluorescence data: weighting =0.4 (including key information on pathogen quantification);

[0157] Gram staining data: weighting =0.3 (reflects the structure of the bacterial community);

[0158] Dry chemical enzymatic data: weighting =0.3 (reflects the metabolic environment);

[0159] Multi-dimensional weighted fusion model for comprehensive normalized value calculation:

[0160] , ;

[0161] in, This represents the normalized value of the immunofluorescence signal. These are the normalized values ​​for Gram staining morphological characteristics. Normalized values ​​for dry chemical indicators;

[0162] Taking a sample of a patient with bacterial vaginosis as an example, the normalization process is as follows:

[0163] 1. Raw data:

[0164] Fluorescence signal intensity = 650 RFU (Mycoplasma positive);

[0165] Gram staining: G+ bacteria = 20, G- bacteria = 80 → G+ / G- ratio = 0.25;

[0166] Dry chemical enzymatic method: pH=4.8, hydrogen peroxide concentration=5μmol / L.

[0167] 2. Single-dimensional normalization:

[0168] Fluorescence signal (Sigmoid normalized): ≈0.82;

[0169] G+ / G- ratio (linearly normalized, normal range 0.5-2.0, outliers 0.25 are treated as lower limit): = 0 (because 0.25 < 0.5, it is mapped to 0);

[0170] Biochemical indicators (weighted average normalized: pH normalized = 0.65, hydrogen peroxide normalized = 0.3); =0.65×0.5+0.3×0.5=0.475);

[0171] Weighted fusion =0.4*0.82+0.3*0+0.3*0.475=0.328+0+0.1425=0.4705 (The normalized result is 0.47, indicating an imbalance in the microecology).

[0172] The diagnostic result output unit acquires single-dimensional normalized data and weighted fusion data, and links to the diagnostic result output library to output an overall evaluation of the microbiome and the type of inflammation. It performs detection through combinations of various single-dimensional normalized data to output the type of inflammation, such as healthy, mild VVC, severe VVC, BV, TV, AV, CV, mixed infection (VVC+BV), and mixed infection (VVC+BV+TV+AV). Through the combination of single-dimensional normalized data and weighted fusion data, it can output an overall evaluation of the microbiome, including the quantification of the overall cleanliness of the microbiome, Nugent score, microbial density, proportion of lactobacilli, and proportion of other miscellaneous bacteria.

[0173] The above embodiments are merely preferred embodiments of the present invention. Therefore, all equivalent changes or modifications made to the structure, features and principles described in the claims of the present invention are included within the scope of the present invention.

Claims

1. A gynecological microecological detection system based on AI multimodal fusion, characterized in that: include: The data acquisition module acquires immunofluorescence signals, Gram staining morphological characteristics, and dry chemical indicators. A standardization module is provided, wherein the data acquisition module is connected to the standardization module, and the standardization module includes a pre-processing unit for preprocessing the acquired data. And a data classification unit connected to the pre-processing unit for data classification, the data classification unit is connected to the heterogeneous data normalization processing unit, the heterogeneous data normalization processing unit completes the normalization processing of collected data, and the heterogeneous data normalization processing unit is connected to the data storage unit. The microecological output unit is communicatively connected to the data storage unit; the microecological output unit includes a diagnostic result output unit, a dynamic analysis module for constructing a time series model of microbial community parameters, and a visualization module for generating a three-dimensional microecological map.

2. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The data classification unit classifies the data acquired by the data acquisition module, assigning a unique correlation factor to each category. This correlation factor is connected to a factor library. The categorized data includes percentage data, linear data, and non-linear data. The heterogeneous data normalization processing unit includes… The first normalization processing model for accessing percentage data is: normalizing the percentage to [0,1]. The second normalized processing model for linear data: ； in, For the normalization result of linear data, This refers to linear data acquired by the data acquisition module. for Minimum value corresponding to the data type for Maximum value corresponding to the data type; The third normalization processing model for nonlinear data uses the Sigmoid function mapping. ； in, For the normalization result of nonlinear data, For the slope parameter, This refers to a specific nonlinear data acquired by the data acquisition module. for The midpoint value of the corresponding data type; After completing the single-dimensional normalization, the heterogeneous data normalization processing unit assigns different weights to the data collected by the data acquisition module based on its contribution to the micro-ecological assessment, resulting in a comprehensive normalized value, as follows: Multi-dimensional weighted fusion model for comprehensive normalized value calculation: ； in, This represents the normalized value of the immunofluorescence signal. These are the normalized values ​​for Gram staining morphological characteristics. Normalized values ​​for dry chemical indicators For immunofluorescence signal weights, Weights for Gram staining morphological features. The weights are for dry chemical indicators.

3. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The preprocessing unit includes: Immunofluorescence signal preprocessing: An adaptive threshold valve is used to remove background noise from the immunofluorescence signal. The threshold is the mean plus 2 times the standard deviation, while retaining the effective signal region. Gram staining morphological features preprocessing: The Gram staining morphological features were converted to grayscale, then edge detection was performed using the Canny operator, and then the U-Net model was used to extract the cell contours from the edge detection data. Dry chemical index preprocessing: removing outliers and data that are outside the linear range.

4. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The dynamic analysis module is as follows: The dynamic analysis module acquires dynamic data of a specific object within a selected stage through the data storage unit. This dynamic data includes community density, colony diversity index, and dominant bacteria percentage for each time series. It also calculates an inflammation risk index, ranging from 0 to 10, as detailed below: ； in, As community density weight, =0.5; As the weight of the colony diversity index, =0.3; Weighting based on the proportion of dominant bacteria. =0.2; Community density, Taking the logarithm of the community density linearizes the exponential growth. As a colony diversity index, To transform diversity into a measure of deviation from a healthy state; The percentage of dominant bacteria, The percentage of non-dominant bacteria indicates an abnormal bacterial community structure. Based on dynamic data and inflammation risk index, an LSTM neural network is used to construct a time series model. The input is the dynamic data under the time series, and the output is the inflammation risk index.

5. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The visualization module includes a cross-modal feature fusion network, and the output of the cross-modal feature fusion network is connected to the input of the visualization processing module. The modality feature fusion network includes three sets of input branches, specifically: Branch 1: 3D-ResNet processing of fluorescent antigen distribution data. Input specifications: 3D fluorescent antigen distribution data of 128×128×64 voxels; Network structure: 5 layers of 3D residual blocks, each layer contains: 3D convolution kernel; Batch normalization LeakyReLU (α=0.01); Output: 64-dimensional feature vector; Branch 2: CNN+LSTM processing of Gram-stained temporal images; Input specifications: Gram-stained temporal images (128×128×T, T=15 time points); Processing flow: 2D-CNN extracts spatial features; after temporal expansion, the images are input into a bidirectional LSTM, and the features at each time point are weighted by an attention mechanism; Output: 128-dimensional temporal feature vector; Branch 3: Fully connected network for processing enzyme metabolism indicators; Input: 12-dimensional enzyme metabolism indicators; Network structure: 3-layer fully connected (256→128→64); Feature enhancement: Exponential linear unit activation; Fusion layer: After feature concatenation, the features are weighted by a self-attention mechanism to output a 256-dimensional fusion vector; After acquiring the output data from the fusion layer, the visualization processing module generates a visualization output through a 3D micro-ecological map. The specific generation of the 3D micro-ecological map is as follows: Spatial mapping: Coordinate transformation: Map the feature vectors to a 3D mesh (64×64×32); Probability field generation: Pathogen distribution: Sigmoid(W_p·F); Microbial community composition: Softmax(W_b·F); Metabolic dynamics: LSTM decoder; Dynamic visualization specifications: Based on set threshold conditions, determine the pathogen enrichment zone, lactobacillus dominant zone, and dynamic changes in H2O2 concentration corresponding to the 3D grid; determine the color coding standard table, and perform color coding and rendering parameter processing to complete the visualization output, specifically including red area: pathogen enrichment zone; blue area: lactobacillus dominant zone; green flow trajectory: dynamic changes in H2O2 concentration.

6. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The visualization module includes feature extraction and 3D spatial mapping, as detailed below: The feature extraction is as follows: Fluorescence image: The antigen distribution region was segmented using a U-Net network, and the centroid coordinates (x1, y1) were extracted. Gram-stained images: Bacterial cell morphology features (length, width, texture) were extracted using ResNet50. Metabolites: Sialidase activity is converted into three-dimensional z-axis coordinates: z = activity / 10, and the activity is normalized to [0,1]. After the feature extraction output data is obtained, a coordinate system is constructed through three-dimensional spatial mapping: X-axis: Abscissa of fluorescence image (0-1024 pixels); Y-axis: Vertical axis of Gram staining morphology image (0-1024 pixels); Z-axis: Normalized value of metabolite activity (0-1); A three-dimensional surface model was generated using the Marching Cubes algorithm to visualize the distribution and metabolic activity of the microbial community.

7. The gynecological microecological detection system based on AI multimodal fusion according to claim 1, characterized in that: The diagnostic result output unit is connected to the diagnostic result output library, which contains first correlation data between the comprehensive normalized value and the overall microecological evaluation; and correlation data between each inflammation type and the threshold range of each single-dimensional normalized value. The diagnostic result output unit obtains the single-dimensional normalized value and the comprehensive normalized value, retrieves the diagnostic result output library according to the input data, and outputs the overall microecological evaluation and inflammation type.

8. The gynecological microecological detection system based on AI multimodal fusion according to claim 7, characterized in that: The overall evaluation of the microecology includes overall cleanliness, Nugent score, microbial density, proportion of lactobacilli and other miscellaneous bacteria.

Images