Stone composition multi-modal prediction method and system based on urine analysis

Abstract

The application provides a stone composition multi-modal prediction method and system based on urine analysis, and belongs to the technical field of medical detection and artificial intelligence. The method comprises the following steps: acquiring a multi-modal data set of urine of a stone patient; identifying a crystal type; generating a metabolic feature matrix; extracting features from the crystal type identification result to generate a crystal feature vector; performing standardization processing on the metabolic feature matrix and the crystal feature vector to obtain a fusion feature matrix; inputting the fusion feature matrix into an extreme gradient boosting tree model to output an initial probability of a stone composition; inputting the initial probability of the stone composition into a bidirectional long short-term memory network model to output a probability distribution of the stone composition, and completing stone composition prediction. The application breaks through the limitations of traditional technologies in terms of accuracy, efficiency, cost, applicability and the like, and provides a feasible solution for early diagnosis, personalized treatment and popularization of primary medical treatment of urinary stones.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical detection and artificial intelligence, in particular to a stone composition multi-modal prediction method and system based on urine analysis. BACKGROUND

[0002] The composition analysis of urinary calculi is the core basis for developing personalized treatment plans, but the existing technology has significant limitations:

[0003] (1) Invasive detection bottleneck: gold standard methods such as infrared spectroscopy and X-ray diffraction require surgical or extracorporeal lithotripsy to obtain stone samples, which has poor patient compliance and cannot achieve early prediction.

[0004] (2) Defects in urine analysis technology:

[0005] Single dimension limitation: traditional urine detection only focuses on metabolite concentration (such as calcium and oxalic acid), without combining crystal morphology information, resulting in a prediction accuracy of less than 70% for mixed calculi;

[0006] High dependence on equipment: high-performance liquid chromatography (HPLC) and other detection methods require professional laboratory environment and are difficult to popularize in primary medical institutions;

[0007] Lag in morphological analysis: manual interpretation of crystal morphology by optical microscope is highly subjective and cannot identify microcrystals less than 10 microns.

[0008] (3) Machine learning model deficiency: existing models are mostly based on a single data source (such as metabolites or images), do not fully utilize the correlation of multi-modal data, have weak generalization ability, and are difficult to adapt to complex clinical scenarios. SUMMARY

[0009] The present application provides a stone composition multi-modal prediction method and system based on urine analysis to solve one or more of the above problems.

[0010] To achieve the above purpose, the present application adopts the following technical solutions:

[0011] The stone composition multi-modal prediction method based on urine analysis comprises:

[0012] Obtaining a multi-modal data set of urine from stone patients, the multi-modal data set including metabolic detection data and crystal images;

[0013] Using an improved Inception-ResNet-v2 network to process the crystal images and identify the crystal types;

[0014] Extracting features from the metabolic detection data and generating a metabolic feature matrix using principal component analysis combined with partial least squares discriminant analysis;

[0015] Feature extraction is performed on the crystal type recognition result, morphological features and texture features are extracted, and a crystal feature vector is generated;

[0016] The metabolic feature matrix and the crystal feature vector are standardized to obtain a fusion feature matrix;

[0017] The fusion feature matrix is input into an extreme gradient boosting tree model, and an initial probability of stone composition is output;

[0018] The initial probability of stone composition is input into a bidirectional long short-term memory network model, and after feature fusion through an attention mechanism, a probability distribution of stone composition is output, and stone composition prediction is completed.

[0019] In the specification, the improvements of the improved Inception-ResNet-v2 network include: integrating a channel attention module and a residual connection, the calculation formula of the residual connection is that the output feature is equal to the sum of the input feature and the convolution operation result, and the channel attention module enhances the key channel feature through global average pooling, fully connected layer mapping and feature weighting.

[0020] In the specification, the training process of the improved Inception-ResNet-v2 network includes: taking the preprocessed crystal image as input, using a cross-entropy loss function to calculate the difference between the predicted value and the true crystal type label; During training, first freeze the network backbone parameters, only train the channel attention module weights, unfreeze all parameters after 20 iterations, and continue to iterate for 30 rounds; The initial learning rate is 1e-4, which is attenuated to 0.5 times of the previous round every 10 rounds, until the crystal type recognition accuracy of the model on the validation set does not improve for 5 consecutive rounds, then stop training.

[0021] In the specification, the specific operation of the principal component analysis combined with the partial least squares discriminant analysis method includes: first, standardizing the metabolic detection data by Z-score, then calculating the covariance matrix of the standardized data, obtaining the principal components through eigenvalue decomposition, and selecting the first 100 principal components with a cumulative contribution rate of ≥95% as the principal component analysis features; The principal component analysis features and the stone type label are input into a partial least squares discriminant analysis model, 28-dimensional latent variables are extracted by maximizing the covariance of the features and the label, and finally the 100-dimensional principal component analysis features and the 28-dimensional partial least squares discriminant analysis latent variables are spliced to generate a 128-dimensional metabolic feature matrix.

[0022] In the specification, the construction parameters of the gray level co-occurrence matrix when extracting texture features include: based on the crystal region gray scale image, setting the pixel distance to 1 and the angle to 0° / 45° / 90° / 135°, calculating the gray level co-occurrence matrix under each angle, and then extracting 4 types of texture parameters including contrast, energy, entropy and correlation, taking the mean value of the same type of parameters of the 4 angles to obtain 192-dimensional texture features.

[0023] In the specification, the training process of the extreme gradient boosting tree model and the bidirectional long short-term memory network model includes: 1200 clinical samples verified by infrared spectroscopy are used as the training set, and the training set and the test set are divided in a ratio of 7:3; the crystal image is enhanced by random rotation of ±15° and scaling of 0.8-1.2 times, and the metabolite concentration data is enhanced by adding Gaussian noise; the hyperparameters of the extreme gradient boosting tree model include 100 trees, a maximum depth of 6, and a learning rate of 0.1, the hidden layer dimension of the bidirectional long short-term memory network model is 64, the Adam optimizer is used, and the learning rate is 1e-4.

[0024] In the specification, the training process of the extreme gradient boosting tree model further includes: a cross-entropy loss function is used to measure the deviation of the initial probability from the true stone composition label, the optimal split point is selected by a greedy algorithm, the split criterion is to maximize the information gain, the gradient and Hessian and of the left and right sub-trees need to be considered when calculating the information gain, and a regularization term is introduced to control the complexity of the tree to avoid overfitting of the model; the prediction accuracy of the training set and the validation set is calculated after each iteration in the training process, and the model parameters with the highest validation set accuracy are used as the final parameters.

[0025] In the specification, the attention mechanism of the bidirectional long short-term memory network model includes: first, the attention score of the output time step hidden state is calculated, the score is obtained by inputting the hidden state into a fully connected layer with ReLU activation, and then normalized by a Sigmoid function; the hidden state of the time step is weighted and summed according to the attention score to obtain a feature vector with fused attention; the feature vector is input into a fully connected layer, and the final probability distribution of the stone composition is output by combining a Softmax function.

[0026] In the specification, the extraction process of the morphological features includes: first, identify individual crystal regions by binarizing the image, use the contour detection algorithm to obtain the edge coordinates of each crystal, calculate the area (total number of pixels inside the edge), perimeter (total number of edge pixels), longest axis (maximum distance between two edge points), and shortest axis (maximum distance between two edge points perpendicular to the longest axis) of the crystal based on the coordinates, then derive the aspect ratio (longest axis / shortest axis) and circularity (4π×area / perimeter²) based on the above parameters, and finally take the mean of the above parameters for all crystals to obtain 64-dimensional morphological features (1 class of features for each parameter, a total of 8 classes of parameters, 8 dimensions for each class, a total of 64 dimensions).

[0027] The stone composition multi-modal prediction system based on urine analysis applies any one of the stone composition multi-modal prediction methods based on urine analysis described above, and the stone composition multi-modal prediction system based on urine analysis includes:

[0028] The data acquisition module is configured to acquire a multi-modal data set of the urinary calculus patient, and the multi-modal data set comprises metabolic detection data and a crystallization image.

[0029] The crystallization type identification module is configured to process the crystallization image by using an improved Inception-ResNet-v2 network, and identify the crystallization type.

[0030] The feature matrix construction module is configured to extract features from the metabolic detection data, and generate a metabolic feature matrix by using principal component analysis combined with partial least squares discriminant analysis.

[0031] The feature vector construction module is configured to extract features from the crystallization type identification result, extract morphological features and texture features, and generate a crystallization feature vector.

[0032] The fusion module is configured to perform standardization processing on the metabolic feature matrix and the crystallization feature vector, and obtain a fusion feature matrix.

[0033] The initial prediction module is configured to input the fusion feature matrix into an extreme gradient boosting tree model, and output an initial probability of the calculus composition.

[0034] The final prediction module is configured to input the initial probability of the calculus composition into a bidirectional long short-term memory network model, fuse features through an attention mechanism, output a probability distribution of the calculus composition, and complete the calculus composition prediction.

[0035] In summary, the present application has at least the following advantages:

[0036] Significantly improve the prediction accuracy: by fusing the urine metabolomics and crystallization morphology two-dimensional data, the problem of insufficient prediction ability of traditional single data source is solved. Among them, the calcium oxalate calculus recognition accuracy is 91%, the mixed calculus prediction accuracy is improved to 88% (the traditional single dimension detection mixed calculus accuracy is less than 70%); in the crystallization type identification, the calcium oxalate crystallization recognition accuracy is 99%, the uric acid crystallization is 98%, and the mixed calculus prediction F1 value reaches 0.92, which provides a reliable basis for clinical accurate diagnosis.

[0037] Realize non-invasive detection: without the need to obtain calculus samples through surgery or extracorporeal lithotripsy, the detection can be completed only by collecting morning urine, which avoids the pain and risk brought to patients by invasive operation, significantly improves the patient compliance, and is especially suitable for early screening of calculus, monitoring of high-risk groups and the like.

[0038] Precise capture of microcrystalline features: using a 530nm LED light source, 300 times magnification polarized light microscopic imaging, combined with an improved Inception-ResNet-v2 network (integrating SEBlock and residual connection), can identify microcrystals <10μm, solve the defects that traditional optical microscope manual interpretation cannot identify microcrystals and is highly subjective, provide more morphological basis for stone composition prediction.

[0039] Feature extraction is accurate and reduces redundant information: 128-dimensional metabolic features (including 12-dimensional core indicators such as ion activity product IAP) are extracted by PCA+PLS-DA, combined with 256-dimensional crystalline features (including morphological parameters such as roundness, texture parameters such as gray level co-occurrence matrix contrast), which not only retains the key information related to stone formation, but also reduces redundancy through dimension reduction, providing high discriminative input for multi-modal fusion.

[0040] Strong model generalization ability, suitable for complex clinical scenarios: based on 1200 multi-center (top three hospitals) samples, through data augmentation methods such as crystal image rotation (±15°), scaling (0.8-1.2 times) and metabolic data Gaussian noise disturbance, the model's adaptability to stone composition spectrum of different regions and different populations is improved; support online updating of the model, which can dynamically match the changes of clinical data.

[0041] In summary, the present application breaks through the limitations of traditional technology in accuracy, efficiency, cost, and applicability, and provides a feasible solution for early diagnosis, personalized treatment, and popularization of primary medical care of urinary stones. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A schematic diagram of the stone composition multi-modal prediction method based on urine analysis involved in the present application.

[0043] Figure 2 A schematic diagram of the method flow involved in the present application.

[0044] Figure 3 A schematic diagram of the improved Inception-ResNet-v2 network architecture involved in the present application.

[0045] Figure 4 A schematic diagram of the cascade model training process involving random forests involved in the present application.

[0046] Figure 5 A schematic diagram of the improved Inception-ResNet-v2 network architecture involved in the present application. DETAILED DESCRIPTION

[0047] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0048] As Figure 1 and Figure 2 shown, the present embodiment provides a stone composition multi-modal prediction method based on urine analysis, comprising:

[0049] S1: Urine sample collection and pretreatment

[0050] To ensure the accuracy and repeatability of the test results, the sample collection must strictly follow the standardized process. The specific operation is as follows:

[0051] 1. Sample collection timing: Select the first urine (morning urine) of the subject in the morning on an empty stomach, at this time the urine is concentrated after night, the metabolite concentration and crystalline content are more stable, and the physiological state related to the formation of stones in the body can be more accurately reflected.

[0052] 2. Collection volume and container: Use a sterile disposable centrifuge tube (50mL size, pre-labeled with sample number, collection time, and subject information) to collect 5-10mL of urine, avoiding sample residue or contamination.

[0053] 3. Immediate processing requirements: Send to the laboratory for processing within 2 hours after collection; If it cannot be detected in time, it needs to be frozen in a -80℃ ultra-low temperature refrigerator (the longest storage time is not more than 72 hours), to avoid metabolite degradation or crystalline morphology change.

[0054] 4. Thawing treatment: Take out the frozen sample before detection, and let it stand at room temperature for 30 minutes for natural thawing, then vortex it with a vortex mixer (speed 3000rpm) for 1 minute to ensure uniformity of the sample without sediment.

[0055] The core role of this step is to provide high-quality, standardized initial samples for subsequent detection, reducing errors caused by improper collection timing and storage methods. Output: Pretreated urine sample (for S2 centrifugal separation).

[0056] S2: Centrifugal separation of urine sample

[0057] Urine is composed of liquid components (supernatant) and solid particles (urine sediment), of which the supernatant contains metabolites and the urine sediment contains potential crystals, which need to be physically separated by centrifugal separation. The specific operation is as follows:

[0058] 1. Centrifugal parameter setting: Transfer the S1 pretreated urine sample to a 15mL centrifuge tube (to avoid overflow), place it in a desktop high-speed centrifuge (model: Eppendorf 5810R), set the centrifugal temperature to 4℃ (to prevent metabolite denaturation), the speed to 400xg (about 1200rpm), and the centrifugal time to 5 minutes (to ensure complete separation).

[0059] 2. Separation operation: After centrifugation, the centrifuge tube was carefully removed, and the sample was found to be divided into two layers: the upper layer was a light yellow transparent liquid (urine supernatant, about 80% of the total amount), and the lower layer was a grayish white precipitate (urine sediment, about 20% of the total amount).

[0060] 3. Sample transfer: 4 mL of the upper supernatant was taken with a 1 mL pipette (range 0.1-1 mL, accuracy ±1 μL) and transferred to a new 15 mL centrifuge tube (labeled "supernatant + sample number"), and the lower urine sediment was retained in the original tube (labeled "urine sediment + sample number") to avoid cross contamination caused by touching the precipitate during pipetting.

[0061] This step achieves preliminary separation of metabolites and crystals through physical separation, providing independent data sources for subsequent "metabolomics detection" and "crystal morphology analysis", and is the basis for multi-modal analysis. Output: urine supernatant (for enzyme digestion and purification in S3), urine sediment (for crystal image acquisition in S5).

[0062] S3: Enzymatic digestion and purification of urine supernatant

[0063] The urine supernatant contains interfering substances such as proteins and lipids, and some metabolites exist in a bound state (e.g., combined with glucuronide), which need to be released as free metabolites through enzymatic digestion, and then purified to remove interference. The specific operation is as follows:

[0064] 1. Enzymatic reaction (release of free metabolites):

[0065] Take 1 mL of urine supernatant into a 10 mL glass tube, add 10 μL of β-glucuronidase (source: Sigma-Aldrich G7770, concentration 10 U / μL, enzyme activity unit definition: 1 U is the amount of enzyme that catalyzes the hydrolysis of 1 μmol glucuronide at 37°C for 1 minute), and mix gently with a pipette for 3 times.

[0066] Place the glass tube in a constant temperature water bath (accuracy ±0.5°C), set the temperature to 37°C (simulate human physiological environment), incubate for 60 minutes; every 15 minutes, take out the glass tube and vortex it with a vortex instrument (speed 2000 rpm) for 10 seconds to ensure that the enzyme and substrate are in full contact.

[0067] The core function of enzymatic digestion is to convert bound-state metabolites (such as oxalate glucuronide) into free-state, avoiding underestimation of concentration due to differences in metabolite form during detection.

[0068] 2. Solid phase extraction purification (remove interfering substances):

[0069] Activation of extraction column: Take the Oasis HLB solid-phase extraction column (500 mg / 3 mL, Waters Corporation, filled with hydrophilic lipophilic balanced polymer), add 5 mL of methanol (chromatographically pure) and 5 mL of ultrapure water (18.2 MΩ·cm) in turn, control the flow rate to be 1 drop / second (about 0.5 mL / min), and ensure that the filler is fully swollen.

[0070] Sample loading: Transfer the sample after enzymatic hydrolysis to the extraction column, and let it flow naturally (without applying pressure), so that the metabolites are adsorbed by the filler, and the macromolecular impurities such as proteins and lipids are discarded with the effluent.

[0071] Washing: Add 5 mL of 5% methanol aqueous solution (volume ratio, methanol is chromatographically pure) to wash the extraction column, remove the polar impurities that are not firmly adsorbed, and discard the effluent.

[0072] Elution: Add 5 mL of methanol (chromatographically pure) for elution, and collect the eluate into a 10 mL glass tube (at this stage, the metabolites are eluted into methanol).

[0073] 3. Resuspension (adapt to the detection system):

[0074] Put the eluate into a nitrogen blowing dryer (temperature 37°C, nitrogen pressure 0.1 MPa), and blow it dry to near dryness (retain about 50 μL to avoid loss of metabolites).

[0075] Add 100 μL of 0.1% formic acid water (volume ratio, formic acid is mass spectrometry grade), and resuspend it by blowing it 10 times with a pipette gun, then pass it through a 0.22 μm organic phase filter membrane (to remove small particles), to obtain the purified sample, which is transferred to a 2 mL sample bottle.

[0076] This step significantly improves the specificity and sensitivity of metabolite detection through enzymatic hydrolysis and purification, and lays the foundation for the HPLC-MS / MS detection of S4. Output: Purified urine supernatant sample (for metabolite detection of S4).

[0077] S4: Metabolite HPLC-MS / MS detection

[0078] Quantitative detection of 8 core metabolites related to the formation of urinary stones (such as oxalic acid, which is the precursor of calcium oxalate stones, and uric acid, which is the core component of uric acid stones) was performed using high performance liquid chromatography-tandem mass spectrometry (HPLC-MS / MS) technology. The specific operation is as follows:

[0079] 1. Detection system and parameter settings:

[0080] Instrument: UPLC-TQ-SYNAPT G2 system (Waters Corporation), consisting of ultra-high performance liquid chromatography (UPLC) and triple quadrupole-time-of-flight mass spectrometry (TQ-SYNAPT).

[0081] Chromatographic conditions:

[0082] Chromatographic column: ACQUITY UPLC BEH C18 (2.1 x 150 mm, 1.7 μm), which has good retention for both polar and non-polar metabolites;

[0083] Column temperature: 40°C (to reduce column efficiency loss);

[0084] Mobile phase: A phase is 0.1% formic acid water (to enhance ionization efficiency), B phase is methanol (chromatographic pure);

[0085] Gradient elution program (to ensure complete separation of 8 metabolites):

[0086] Time (min) A phase ratio (%) B phase ratio (%) Flow rate (mL / min) 0-5 95 5 0.2 5-15 5 95 0.2 15-18 5 95 0.2 18-19 95 5 0.2 19-25 95 5 0.2

[0087] Mass spectrometry conditions:

[0088] Ion source: electrospray ion source (ESI+), suitable for ionization of polar metabolites;

[0089] Capillary voltage: 3.0 kV, cone hole voltage: 30 V (to optimize ion transmission efficiency);

[0090] Desolvation gas: nitrogen, temperature 350°C, flow rate 800 L / h (to remove solvent residues);

[0091] Detection mode: multiple reaction monitoring (MRM), to improve specificity by selecting specific parent ion-daughter ion pairs, MRM parameters for 8 metabolites are as follows (all optimized by pre-experiment):

[0092] Metabolite Parent ion (m / z) Sub ion (m / z) Collision energy (eV) Retention time (min) Oxalic acid 89.0 45.0 15 3.2 Calcium ion 40.1 40.1 0 2.1 Uric acid 168.0 124.0 20 8.5 Citric acid 191.0 111.0 18 5.7 Magnesium ion 24.3 24.3 0 2.3 Phosphate 97.0 49.0 25 4.1 Potassium ion 39.1 39.1 0 2.0 Sodium ion 23.0 23.0 0 1.8

[0093] 2. Concentration quantitative calculation:

[0094] Standard preparation: 8 metabolites (purity ≥98%) were weighed and dissolved in 0.1% formic acid water to prepare 6 concentration gradient mixed standard solutions (range covering 0.5-2 times of the common concentration in clinic, intermediate value uniformly increasing, including both normal concentration of healthy population and abnormal concentration (such as high oxalic acid, high uric acid) that may occur in stone patients, providing complete reference for standard curve drawing and sample concentration quantification of HPLC-MS / MS detection), specific concentrations are as follows:

[0095] Metabolite Concentration gradient (mmol / L) Clinical reference range (mmol / L) Gradient setting logic explanation Oxalic acid 0.05,0.1,0.5,1.0,5.0,10.0 0.1-5.0 (normal range of oxalic acid in urine) Minimum value = 0.1 x 0.5 = 0.05, maximum value = 5.0 x 2 = 10.0 Calcium ion 0.1,0.5,1.0,2.0,5.0,10.0 0.2-5.0 (normal range of calcium ion in urine) Minimum value = 0.2 x 0.5 = 0.1, maximum value = 5.0 x 2 = 10.0 Uric acid 0.1,0.5,1.0,2.0,5.0,10.0 0.2-5.0 (normal range of uric acid in urine) Minimum value = 0.2 x 0.5 = 0.1, maximum value = 5.0 x 2 = 10.0 Citric acid 0.15,0.3,1.0,2.0,3.0,6.0 0.3-3.0 (normal range of citric acid in urine) Minimum value = 0.3 x 0.5 = 0.15, maximum value = 3.0 x 2 = 6.0 Magnesium ion 0.1,0.2,0.5,1.0,1.5,2.0 0.2-1.0 (normal range of magnesium ion in urine) Minimum value = 0.2 x 0.5 = 0.1, maximum value = 1.0 x 2 = 2.0 Phosphate 0.75,1.5,3.0,4.5,6.0,12.0 1.5-6.0 (normal range of phosphate in urine) Minimum value = 1.5 x 0.5 = 0.75, maximum value = 6.0 x 2 = 12.0 Potassium ion 10,20,40,60,100,200 20-100 (common range of potassium ion in urine) Minimum value = 20 x 0.5 = 10, maximum value = 100 x 2 = 200 Sodium ion 20,40,100,150,220,440 40-220 (common range of sodium ion in urine) Minimum value = 40 x 0.5 = 20, maximum value = 220 x 2 = 440

[0096] Standard curve drawing: HPLC-MS / MS detection was performed on 6 concentrations of standard, with concentration as abscissa (x) and corresponding peak area as ordinate (y), least squares method was used to fit the standard curve, the formula is:

[0097] (1)

[0098] where b is the slope of the curve (sensitivity indicator), a is the intercept (system error indicator), and the correlation coefficient should be ≥ 0.999 (to ensure good linearity).

[0099] Sample concentration calculation: the peak area of the sample (A) is substituted into the standard curve to back-calculate the metabolite concentration (C):

[0100] (2)

[0101] where Cx is the concentration of the xth metabolite in the sample (unit: mmol / L), is the peak area of the sample, and a and b are the intercept and slope of the standard curve for the metabolite.

[0102] This step realizes the quantification of 8 core metabolites by high-precision HPLC-MS / MS technology, providing reliable metabolomics data for subsequent feature extraction, which is the first key dimension of multi-modal prediction.

[0103] Output: Concentration matrix of 8 metabolites to correspond to the concentrations of oxalic acid, calcium ions, uric acid, citric acid, magnesium ions, phosphate ions, potassium ions, and sodium ions, respectively.

[0104] S5: Urinary sediment crystal image acquisition

[0105] The urinary sediment may contain tiny crystals consistent with the composition of the stone (such as calcium oxalate crystals, which are a precursor of calcium oxalate stones), and the morphological characteristics (such as shape, size, and texture) of the crystals can be captured by microscopy imaging, providing morphological basis for stone composition prediction. The specific operation is as follows:

[0106] 1. Urinary sediment suspension preparation:

[0107] Add 200 μL of sterile normal saline (0.9% NaCl solution, to avoid crystal dissolution or deformation) to the urinary sediment obtained in S2 (about 1 mL), blow 20 times with a pipette, and then vortex for 30 seconds at a speed of 2000 rpm using a vortex instrument to prepare a uniform urinary sediment suspension, ensuring uniform dispersion of the crystals.

[0108] 2. Tablet preparation:

[0109] ​​​10 μL suspension was taken with a 10 μL pipette and slowly added to the center of the glass slide (76×26 mm, thickness 0.17 mm) to avoid air bubbles; then a cover glass (24×24 mm) was picked up with tweezers and slowly covered on the droplet at a 45° angle to evenly spread the liquid (covering an area of about 1 cm²), and placed for 5 minutes to stabilize the liquid.

[0110] 3. Microscopic imaging parameters:

[0111] Microscope: Olympus BX53 polarized light microscope (equipped with polarizer and compensator to enhance the contrast between crystals and background);

[0112] Objective lens: 300x magnification (10x eyepiece + 30x objective lens, numerical aperture 0.75), which can clearly distinguish crystals ≥1 μm;

[0113] Light source: 530 nm monochromatic LED light source (to reduce stray light interference), light intensity set to 5000 lux;

[0114] Camera: 12 million pixel CMOS camera (resolution 1024×1024 pixels, frame rate 30 fps), exposure time 50 ms (to avoid overexposure).

[0115] 4. Image acquisition strategy:

[0116] To avoid bias caused by uneven samples, the "grid sampling method" was used: the cover glass covered area was divided into a 5×6 grid (a total of 30 areas), and one image was taken for each area to ensure that the center and edge areas of the sample were covered. After acquisition, the images were saved in TIFF format (uncompressed, preserving the original information), with the file name format "sample number_region number.tif".

[0117] This step captures the morphological features of crystals in urine sediment through a standardized imaging process, providing high-quality image data for S6 crystal identification, and is the second key dimension of multi-modal prediction. Output: 30 microscopic images of urine sediment crystals (TIFF format, 1024×1024 pixels, for S6 crystal identification).

[0118] S6: Crystal identification using improved Inception-ResNet-v2 network

[0119] The morphological features of crystals (such as calcium oxalate crystals in octahedral shape and uric acid crystals in rhombic shape) are directly related to the composition of the stone, and need to be automatically identified through a deep learning network. This scheme uses an improved Inception-ResNet-v2 network to improve recognition accuracy by integrating a channel attention module, and the process of the improved Inception-ResNet-v2 network architecture is as follows: Figure 3The specific operation is as follows:

[0120] 1. Image preprocessing (enhance feature recognition):

[0121] Gaussian denoising: There may be sensor noise in 30 original images, which needs to be processed by Gaussian filtering, and the kernel function is:

[0122] ; (3)

[0123] In the formula, (x, y) is the pixel coordinate, = 1.0 (the optimal value verified by experiment), which removes high-frequency noise by convolution with the image, while preserving the crystal edge features.

[0124] Binary segmentation: The maximum inter-class variance method (Otsu algorithm) is used to distinguish crystals from the background, and the threshold T is calculated to maximize the inter-class variance between the foreground (crystals) and the background:

[0125] ; (4)

[0126] In the formula, , are the pixel proportions of the background and the foreground, , are the average gray values of the background and the foreground, and the final binary image is:

[0127] ; (5)

[0128] 2. Improved network structure design:

[0129] The original Inception-ResNet-v2 network has insufficient recognition accuracy for small crystals (<10 μm), and this scheme improves performance through two improvements:

[0130] Residual connection: Add a residual connection after each convolutional module to alleviate the gradient vanishing problem of deep networks, and the formula is:

[0131] ; (6)

[0132] In the formula, is the module input feature, is the convolution operation within the module (including BatchNorm and ReLU activation), is the module output feature, which ensures that shallow features can be passed to deep layers.

[0133] Channel attention module (SEBlock): The key features of crystals (such as edge texture) may be distributed in a small number of channels, and SEBlock is used to enhance the weights of these channels:

[0134] 1. Global average pooling: compress the feature map (H, W are the height and width of the feature map, C is the number of channels) into channel descriptors :

[0135] ; (7)

[0136] where, is the global statistical value of the c-th channel, reflecting the overall feature intensity of the channel.

[0137] 2. Channel weight learning: learn the importance of channels through a fully connected layer, the formula is:

[0138] ; (8)

[0139] where, , is the fully connected layer weight (r=16 is the compression ratio to reduce the amount of calculation), is the ReLU activation function, is the Sigmoid function, [0, 1] is the attention weight of the c-th channel (the larger the value, the more important the channel).

[0140] 3. Feature weighting: multiply the weight with the original feature map to enhance the key channel features:

[0141] ; (9)

[0142] 3. Crystal type identification and result integration:

[0143] Network input: preprocessed binary image (1024x1024x3);

[0144] Feature extraction: extract deep features (including shape and texture information) through 4 improved Inception-ResNet modules;

[0145] Classification output: output the probability of 8 crystal types through a fully connected layer and a softmax function:

[0146] ; (10)

[0147] where, is the output score of the k-th crystal type, is the recognition probability of this crystal type, and the 8 crystal types are: calcium oxalate monohydrate, calcium oxalate dihydrate, uric acid, calcium phosphate, cystine, magnesium ammonium phosphate, calcium carbonate, and silica.

[0148] Result integration: take the arithmetic mean of the recognition results of 30 images to get the final crystallization type probability matrix of the sample:

[0149] ; (11)

[0150] This step realizes the automatic recognition of crystallization type through an improved deep learning network, solves the problems of subjectivity and low efficiency of manual interpretation, and provides morphological probability data for subsequent feature extraction. Output: probability matrix of 8 crystallization types Crystallization feature extraction for S8.

[0151] S7: Metabolic feature extraction (PCA+PLS-DA)

[0152] The 8 kinds of metabolite concentration data obtained in S4 have low dimensions and may have collinearity (such as high correlation between calcium and oxalic acid concentrations), which need to be converted into high-dimensional and low-redundancy metabolic features through feature extraction to enhance the discrimination ability of subsequent models. The specific operation is as follows:

[0153] 1. Data standardization (eliminate dimension effect):

[0154] The concentration units and ranges of different metabolites are quite different (such as oxalic acid concentration in mmol / L, while calcium ion in mmol / L but with different numerical ranges), which need to be standardized by Z-score:

[0155] ; (12)

[0156] In the formula, is the standardized value of the jth metabolite, is the original concentration obtained in S4, is the mean value of the metabolite in the training set (based on 1200 samples), is the standard deviation of the metabolite in the training set. The mean value of the standardized data is 0 and the standard deviation is 1, which ensures the comparability of different metabolite features.

[0157] 2. Principal component analysis (PCA) dimension reduction:

[0158] PCA maps high-dimensional data to low-dimensional space through linear transformation, retaining the main variation information of the data, with the following steps:

[0159] Calculate the covariance matrix: reflect the correlation between metabolites, the formula is:

[0160] ; (13)

[0161] In the formula, n=1200 is the number of training samples, is the standardized training set metabolic data. Eigenvalue decomposition: solve

[0162] , get eigenvector matrix (Principal component loadings, reflecting the contribution of metabolites to principal components) and eigenvalue matrix , , indicating the variance contribution of principal components. Principal component selection: calculate cumulative contribution rate

[0163] , select the top k=100 principal components with cumulative contribution rate ≥95% (extended to 100 dimensions by kernel trick, enhanced non-linear representation), generate 100-dimensional PCA features:

[0164] ; (14)

[0165] where, is the normalized metabolic data of the current sample, is the extended top 100 principal component loading matrix.

[0166] 3. Partial least squares discriminant analysis (PLS-DA) enhances discriminability:

[0167] PCA only retains data variation and does not consider the relevance to stone type, PLS-DA extracts more discriminative features by maximizing the correlation between features and labels:

[0168] Define label matrix: stone type label in training set (8 categories, one-hot encoding: 1 for a certain category, 0 for others).

[0169] Latent variable extraction: iteratively solve the covariance between feature latent variables t and label latent variables u to maximize , extract 28-dimensional latent variables (experimentally verified that 28-dimensional is optimal for discrimination).

[0170] Feature integration: combine 100-dimensional PCA features and 28-dimensional PLS-DA features to generate 128-dimensional metabolic feature matrix, which contains 12-dimensional core clinical features (such as oxalic acid concentration, calcium ion concentration, ion activity product , etc., directly related to stone formation risk).

[0171] This step converts the original metabolic data into high-dimensional, high-discriminability features through PCA dimension reduction and PLS-DA enhancement, providing a metabolic omics feature basis for subsequent multi-modal fusion. Output: 128-dimensional metabolic feature matrix (feature standardization for S9).

[0172] S8: Crystallization feature extraction

[0173] The crystal type probability obtained in S6 needs to be further converted into a feature vector in combination with morphological details (such as size, texture) of the crystal to capture more abundant morphological information. The specific operation is as follows:

[0174] 1. Morphological feature extraction (64 dimensions):

[0175] Based on the binary image of S6, 64 morphological parameters are calculated for each crystal region, including:

[0176] Size features: area A (number of pixels x pixel area), perimeter P (number of edge pixels x pixel edge length), longest axis L, shortest axis W;

[0177] Shape features: aspect ratio , circularity (circularity = 1 for a perfect circle), elongation , compactness .

[0178] Distribution features: crystal density (number of crystals per unit area), average distance (mean of center-to-center distance between crystals), etc.

[0179] Take the average of all crystal parameters to obtain a 64-dimensional morphological feature vector .

[0180] 2. Texture feature extraction (192 dimensions):

[0181] The texture of the crystal (such as surface smoothness) is related to the composition (such as cystine crystal surface roughness), and 192-dimensional texture features are extracted through a gray level co-occurrence matrix (GLCM):

[0182] GLCM construction: within the crystal region, calculate the probability of gray values i and j appearing simultaneously at a distance d = 1 pixel and an angle = 0° / 45° / 90° / 135°. .

[0183] Texture parameter calculation (48 parameters are extracted for each angle):

[0184] Contrast: (reflects local gray level variation);

[0185] Energy: (reflects texture uniformity);

[0186] Entropy: (reflects texture complexity);

[0187] Correlation: (reflects gray level linear correlation).

[0188] Take the average of the parameters of the four angles to get a 192-dimensional texture feature vector .

[0189] 3. Feature integration:

[0190] Combine morphological features and texture features to generate a 256-dimensional crystalline feature matrix:

[0191] ; (15)

[0192] This step converts the visual information of the crystal into a quantifiable feature vector by extracting morphological and texture features, providing a morphological feature basis for multi-modal fusion. Output: 256-dimensional crystalline feature matrix (Feature standardization for S9).

[0193] S9: Feature standardization processing

[0194] Metabolic features of S7 and crystalline features of S8 have different dimensions and distribution ranges (such as morphological features may be in pixels, and texture features are dimensionless values), and need to be standardized to ensure that the model learns fairly for different features. The specific operation is as follows:

[0195] Min-Max standardization (mapping feature values to the [0, 1] interval) is used, and the formula is:

[0196] ; (16)

[0197] where F is the original feature value, and are the minimum and maximum values of the feature in the training set (based on 1200 samples). After standardization, metabolic features and crystalline features have the same data range, avoiding the model being dominated by features with large numerical ranges.

[0198] The standardized metabolic features and crystalline features are concatenated by column to generate a 384-dimensional fusion feature matrix:

[0199] ; (17)

[0200] This step eliminates the dimensional differences between features by standardization, providing a fair basis for subsequent XGBoost model feature importance evaluation and weight learning. Output: 384-dimensional fusion feature matrix (XGBoost initial prediction for S10).

[0201] S10: XGBoost layer initial prediction

[0202] XGBoost is an ensemble learning algorithm based on decision trees, which is good at handling high-dimensional features and capturing non-linear relationships. It is used to make preliminary predictions on the fused features and output the initial probabilities of stone composition. The specific operation is as follows:

[0203] 1. Model construction:

[0204] Input: 384-dimensional fused features ;

[0205] Output: Initial probabilities of 8 stone compositions (8 stone compositions: calcium oxalate stones, uric acid stones, calcium phosphate stones, cystine stones, magnesium ammonium phosphate stones, calcium carbonate stones, silica stones, mixed stones)

[0206] Objective function: contains loss term and regularization term, balances prediction accuracy and model complexity:

[0207] ; (18)

[0208] where, is the set of t decision trees, is the cross-entropy loss (measures the difference between the predicted value and the true label ), is the regularization term (T is the number of leaves of the kth tree, is the leaf weight, , is the regularization coefficient to prevent overfitting).

[0209] Hyperparameter settings (optimized by grid search): number of trees t=100, maximum depth 6, learning rate 0.1, =0.1, =1.

[0210] 2. Initial prediction process:

[0211] Input fused features , through the integration of 100 decision trees, output the initial probability:

[0212] ; (19)

[0213] ; (20)

[0214] where, is the prediction score of the ith tree for the kth stone, is the initial probability of the kth stone.

[0215] This step preliminarily fuses the metabolic and crystallization characteristics through the strong nonlinear fitting ability of XGBoost, providing a basic probability distribution for the fine fusion of BiLSTM in the subsequent step. Output: initial probability of 8 stone components BiLSTM final prediction for S11.

[0216] S11: BiLSTM layer final prediction

[0217] The initial prediction of XGBoost does not consider the time sequence association between features (such as the dynamic matching relationship between metabolite concentration and crystallization type), and BiLSTM (Bidirectional Long Short-Term Memory Network) realizes more fine multi-modal fusion by capturing sequence dependency and combining attention mechanism. The specific operation is as follows:

[0218] 1. Model construction:

[0219] Input: XGBoost initial probability According to the order of stone type, it is regarded as a sequence of length T=8

[0220] ;

[0221] Bidirectional LSTM layer: contains forward LSTM (captures the dependence from class 1 to class 8) and backward LSTM (captures the dependence from class 8 to class 1), hidden layer dimension 64, and the hidden state calculation of each time step is as follows:

[0222] Forward LSTM: (t=1,...,8);

[0223] Backward LSTM: (t=8,...,1);

[0224] Merge hidden state: (concatenate forward and backward hidden states)

[0225] Attention mechanism: the importance of the probability of different stone types to the final prediction is different (such as the probability of calcium oxalate crystallization is more important to the prediction of calcium oxalate stone), and the weight is assigned through the attention mechanism:

[0226] ; (21)

[0227] In the formula, is the attention score of the t-th time step, , is a learnable parameter, is a bias, is the weight of the t-th time step (sum is 1).

[0228] The fused features are:

[0229] (22)

[0230] Output layer: Fully connected layer (weights , bias ) combined with softmax function, output final probability:

[0231] (23)

[0232] 2. Final prediction process:

[0233] Input initial probability sequence , capture temporal association through BiLSTM and weighted by attention mechanism, output final probability distribution of 8 stone components .

[0234] This step solves the problem of insufficient feature association capture by XGBoost through the temporal modeling capability of BiLSTM and the weighted fusion of attention mechanism, and improves the prediction accuracy of mixed stones (containing multiple components). Output: Probability distribution of 8 stone components (used for the result output of S16).

[0235] Training data set construction:

[0236] Model training needs to rely on large-scale labeled data to ensure generalization ability. The specific operation is as follows:

[0237] 1. Sample source and quantity: collect 1200 clinical samples from hospitals (can be multiple), covering 8 stone types (150 cases for each type), sample inclusion criteria:

[0238] Obtain stone samples by surgery or extracorporeal lithotripsy, and confirm the composition by infrared spectroscopy (gold standard);

[0239] Collect morning urine samples within 24 hours before surgery, and do not use drugs that affect metabolism (such as diuretics).

[0240] 2. Data content: each sample contains:

[0241] 8 metabolite concentrations of urine supernatant (S4 detection results);

[0242] 30 urine sediment crystal images (S5 collection results);

[0243] Stone type label verified by infrared spectroscopy (one-hot encoding).

[0244] 3. Dataset division: randomly divide into training set (840 cases, for model parameter learning) and test set (360 cases, for model evaluation) in the ratio of 7:3, ensuring consistent stone type distribution in the two groups of samples (P>0.05 by chi-square test).

[0245] The constructed labeled dataset provides the "input-output" mapping relationship for model training and is the basic guarantee of model performance.

[0246] Output: training set (840 cases) and test set (360 cases) (for model training and evaluation of S13-S14).

[0247] Training data augmentation:

[0248] Limited sample size of the training set may lead to model overfitting (good performance in the training set and poor performance in the test set), which requires expanding sample diversity through data augmentation. The specific operation is as follows:

[0249] 1. Crystallization image enhancement: geometric transformation is performed on 30 crystallization images of the training set to generate 4 times the original number of images:

[0250] Random rotation: rotate around the center by ±15°, ±30° (step 15°), simulate the influence of microscope stage rotation;

[0251] Random scaling: scale by 0.8, 0.9, 1.1, 1.2 times, simulate the change of imaging distance;

[0252] Random brightness adjustment: adjust within ±10% of the original brightness, simulate the fluctuation of light source intensity.

[0253] 2. Metabolic data enhancement: Gaussian noise is added to the metabolite concentration of the training set (avoid changing the data distribution):

[0254] ; (24)

[0255] wherein, is the enhanced concentration, is Gaussian noise (mean 0, standard deviation 5% of the original concentration), ensuring that the enhanced data is still within the clinically reasonable range.

[0256] After enhancement, the sample size of the training set is expanded from 840 cases to 3360 cases (840x4), significantly improving the generalization ability of the model.

[0257] Output: enhanced training set (3360 cases) (for model training of S14).

[0258] Cascade model training:

[0259] The cascade model (XGBoost + BiLSTM) needs to optimize the parameters through the training set to ensure the prediction accuracy. The specific operation is as follows:

[0260] 1. XGBoost training:

[0261] Input: 384-dimensional fusion features of enhanced training set and labels ;

[0262] Optimization process: greedy algorithm is used to iteratively construct decision trees, and each tree selects the optimal split point by maximizing information gain:

[0263] ; (25)

[0264] where, , are the gradient and Hessian sum of the left subtree (L is the sample set of the left subtree), , are the corresponding values of the right subtree.

[0265] Early stopping strategy: calculate the test set accuracy after each round of training, and stop training if there is no improvement for 5 consecutive rounds to avoid overfitting.

[0266] 2. BiLSTM training:

[0267] Input: initial probability output by XGBoost and labels ;

[0268] Optimizer: Adam (adaptive matrix estimation), learning rate 1e-4, batch size 32, and iteration 50 rounds;

[0269] Loss function: cross-entropy loss ;

[0270] Early stopping strategy: same as XGBoost, using F1 score (balanced precision and recall) of the test set as the indicator.

[0271] 3. Model evaluation:

[0272] Evaluate the performance on the test set, core indicators: calcium oxalate stone recognition accuracy: 91%; mixed stone recognition accuracy: 88%; average F1 score: 0.90.

[0273] Save the model parameters with the best performance (XGBoost tree structure, BiLSTM weight matrix, etc.).

[0274] Optimize the model parameters through training to enable the cascade model to accurately learn the mapping relationship between metabolic and crystallization features and stone composition.

[0275] Output: The trained XGBoost-BiLSTM cascade model (for prediction in S10-S11).

[0276] Experimental verification:

[0277] I. Experimental purpose

[0278] To verify the accuracy, efficiency, and practicality of the urinary stone composition prediction method based on the integration of urine metabolomics and crystalline morphology analysis, specifically including:

[0279] 1. Core technical indicators (such as stone type recognition accuracy, mixed stone prediction accuracy);

[0280] 2. Model generalization ability and clinical applicability.

[0281] II. Experimental objects and sample sources

[0282] 1. Sample size and source: 1200 clinical samples were collected from three-level hospitals, including 720 males and 480 females, aged 25-70 years (mean 45.6±12.3 years).

[0283] 2. Sample type: All samples were obtained by surgery or extracorporeal lithotripsy, and the composition was determined by infrared spectroscopy (gold standard), covering 8 stone types:

[0284] Single-component stones: 300 cases of calcium oxalate stones, 200 cases of uric acid stones, 150 cases of calcium phosphate stones, 100 cases of cystine stones, 100 cases of magnesium ammonium phosphate stones, 50 cases of calcium carbonate stones, and 50 cases of silica stones;

[0285] Mixed stones: 250 cases of stones containing 2 or more components (such as calcium oxalate + uric acid, calcium phosphate + magnesium ammonium phosphate, etc.).

[0286] III. Experimental methods and steps

[0287] The experimental process is as follows:

[0288] (I) Sample pretreatment (corresponding to steps S1-S2)

[0289] 1. Collect 5mL of morning urine from all samples, centrifuge at 400xg for 5 minutes at 4°C, and separate the urine supernatant and urinary sediment;

[0290] 2. The supernatant is used for metabolomics detection, and the urinary sediment is used for crystalline morphology analysis.

[0291] (II) Metabolomics detection (corresponding to steps S3-S4)

[0292] 1. Enzymatic hydrolysis and purification: Take 1 mL of supernatant, add 10 μL of β-glucuronidase (10 U / μL), and incubate at 37°C for 60 minutes; purify by Oasis HLB solid phase extraction column (500 mg / 3 mL), and elute with nitrogen blowing, then re-dissolve to 100 μL.

[0293] 2. HPLC-MS / MS detection:

[0294] Instrument: UPLC-TQ-SYNAPT G2 system, chromatographic column: ACQUITY UPLC BEH C18 (2.1 x 150 mm, 1.7 μm);

[0295] Mobile phase: A phase (0.1% formic acid water), B phase (methanol), gradient elution program: 0-5 min 5%B→5-15 min 95%B→15-18 min 95%B→18-19 min 5%B→19-25 min 5%B, flow rate 0.2 mL / min;

[0296] Detect 8 core metabolites (oxalic acid, calcium, uric acid, citric acid, magnesium, phosphorus, potassium, sodium), and record the concentration data.

[0297] (Three) Crystal morphology analysis (corresponding to steps S5-S6)

[0298] 1. Crystal image acquisition:

[0299] Urine sediment is suspended in 200 μL of normal saline, and 10 μL is taken for tablet preparation;

[0300] Use a polarized light microscope (300x magnification) with a 530 nm LED light source to take 30 images (covering different areas).

[0301] 2. Crystal recognition:

[0302] After denoising the image with Gaussian filtering, input the improved Inception-ResNet-v2 network (containing SEBlock and residual connection);

[0303] Output the recognition probability of 8 crystal types (calcium oxalate monohydrate, calcium oxalate dihydrate, etc.).

[0304] (Four) Feature extraction and fusion (corresponding to steps S7-S11)

[0305] 1. Feature extraction:

[0306] Metabolic features: After Z-score standardization of the concentrations of 8 metabolites, use PCA+PLS-DA to extract a 128-dimensional metabolic feature matrix (containing 12-dimensional core indicators such as ion activity product);

[0307] Crystalline features: 64-dimensional morphological features (aspect ratio, circularity, etc.) and 192-dimensional texture features (contrast, energy, etc. from gray level co-occurrence matrix) are extracted from images and merged into a 256-dimensional vector.

[0308] 2. Standardization and fusion modeling:

[0309] Features are Min-Max normalized (mapped to the [0, 1] interval) and merged into 384-dimensional fusion features.

[0310] Input XGBoost model outputs initial probabilities, then fuse them through BiLSTM model (hidden layer dimension 64, Adam optimizer) to output probability distribution of 8 stone components.

[0311] (Five) Model training and data augmentation

[0312] 1. Dataset division: 1200 samples are divided into training set (840) and test set (360) according to 7:3;

[0313] 2. Data augmentation:

[0314] Crystal images: randomly rotate ±15°, scale 0.8-1.2 times;

[0315] Metabolic data: add Gaussian noise (standard deviation is 5% of original concentration);

[0316] 3. Model training: XGBoost hyperparameters (tree number 100, maximum depth 6), BiLSTM iteration 50 rounds, learning rate 1e-4.

[0317] Four, experimental results and analysis

[0318] (I) Prediction accuracy verification (test set 360 samples)

[0319] Stone type Recognition accuracy (%) Traditional single method accuracy (%) Improvement range (%) Calcium oxalate stone 91 78 (only metabolite detection) 13 Uric acid stone 90 75 (only crystallization detection) 15 Calcium phosphate stone 89 72 17 Cystine stone 87 68 19 Struvite stone 88 70 18 Calcium carbonate stone 85 65 20 Silica stone 86 66 20 Mixed stone 88 69 (traditional infrared spectroscopy) 19 Average accuracy 88.5 70.4 18.1

[0320] F1 value verification: mixed stone prediction F1 value reaches 0.92 (traditional method 0.75), indicating that the method has better ability to distinguish complex component stones.

[0321] (II) Microcrystalline recognition ability

[0322] For <10 μm microcrystalline (traditional microscope manual reading missed rate >40%), this method realizes through improved Inception-ResNet-v2 network:

[0323] Calcium oxalate microcrystalline recognition rate: 96% (manual reading 58%);

[0324] Uric acid microcrystalline recognition rate: 95% (manual reading 55%).

[0325] (III) Model generalization ability (100 new samples for verification)

[0326] Different hospital sample adaptability: for another 100 samples from 2 hospitals, the average accuracy is 87.3% (fluctuation ≤3%), indicating that the model has strong generalization ability for regional difference samples;

[0327] Data enhancement effect: after enhancement, the accuracy of the model on rare stone types (such as silica stones) is improved by 12% (from 74% to 86%).

[0328] (IV) Clinical application value

[0329] Individualized recommendation effectiveness: for 80 high-risk patients (calcium oxalate stone probability >60%), after adjusting the diet for 3 months according to the doctor's recommendation, the average urine oxalate concentration decreased by 32%, and the crystallization detection rate decreased by 40%.

[0330] Doctors can give appropriate recommendations according to actual conditions, such as:

[0331] High risk of calcium oxalate stones (probability >50%): reduce the intake of spinach, almonds and other oxalates, and drink ≥2000mL of water per day;

[0332] High risk of uric acid stones: low purine diet (avoid animal offal), alkalinize urine (oral sodium bicarbonate).

[0333] Experimental data show that the method of the present application is superior to traditional methods in stone composition prediction accuracy (average improvement of 18.1%), microcrystal identification (missed detection rate reduction >40%), detection efficiency (time consumption reduction of 99%) and cost (reduction of 67%), and has strong generalization ability, simple operation, and fully meets the needs of primary medical care and clinical detection.

[0334] In some embodiments, random forest (RF) is introduced on the basis of XGBoost-BiLSTM cascade model to form a collaborative architecture of "random forest-feature weighting-XGBoost-probability fusion-BiLSTM". Random forest is mainly used for feature importance screening, and the output feature weight and the initial probability of XGBoost are fused by weighting, and then input into BiLSTM for final prediction (the cascade model training process containing random forest is shown in Figure 4 ), realizing the interaction of the three:

[0335] 1. Random forest evaluates the importance of metabolic / crystallization features, and the weight is used to enhance the feature input of XGBoost;

[0336] 2. The prediction probabilities of Random Forest and XGBoost are fused and weighted as the sequence input of BiLSTM, which improves the ability to capture time series correlation.

[0337] Construction and training of Random Forest (RF)

[0338] 3.1 Model construction

[0339] Random Forest is composed of T CART decision trees, and the output of each tree (t = 1, 2,..., T) is the class vote of the stone composition, and the final probability is the proportion of the vote: ;

[0340] In the formula, is the probability of Random Forest predicting the kth type of stone (k = 1,..., 8), is an indicator function (1 if the condition is met, otherwise 0), and T = 200 (the number of trees is optimized by cross-validation).

[0341] 3.2 Feature importance calculation

[0342] Random Forest evaluates feature importance through "out-of-bag data (OOB) error": for the ith feature, randomly permute the feature values in the OOB data, and calculate the error increment before and after permutation. The larger the increment, the higher the importance:

[0343] ; In the formula, is the importance score of the ith feature, is the original OOB error, is the OOB error after feature i is replaced.

[0344] 3.3 Training process

[0345] Training set: 1200 samples are shared with XGBoost / BiLSTM (70% training set, 30% test set), and the label is the stone type verified by infrared spectrum;

[0346] Data augmentation: same as before (crystallization image rotation, metabolic data noise addition);

[0347] Hyperparameters: 200 trees, maximum depth 8, minimum split sample size 5, feature sampling rate 0.8 (each tree randomly uses 80% of the features).

[0348] Interactive fusion of Random Forest and XGBoost

[0349] 4.1 Feature weighted enhancement (RF→XGBoost)

[0350] Feature importance output by Random Forest Softmax normalization to get weights , the fusion feature After weighting, input XGBoost to enhance the impact of key features:

[0351] ; In the formula, is the weighted fusion feature, ensuring that XGBoost pays more attention to high importance features selected by random forest (such as oxalic acid concentration, calcium oxalate crystallization roundness, etc.).

[0352] 4.2 Probability fusion (RF + XGBoost)

[0353] Random forest prediction probability Initial probability of XGBoost Through the weight Fusion, balance the advantages of both (RF has strong anti-overfitting ability, XGBoost has good nonlinear fitting):

[0354] ; In the formula, = 0.3 (optimized on the validation set to maximize F1 value through grid search, search range 0.1-0.7, step 0.1), As the input sequence of BiLSTM.

[0355] Interaction between BiLSTM and fusion probability (final prediction)

[0356] 5.1 BiLSTM input sequence construction

[0357] Fuse probability According to the stone type order (calcium oxalate → uric acid →... → mixed stones), construct a sequence of length T = 8 , input the hidden layer of BiLSTM:

[0358] ;

[0359] In the formula, is the forward LSTM hidden state, is the backward LSTM hidden state, and the dimension after splicing is 128 (64+64).

[0360] 5.2 Attention mechanism weighting

[0361] Assign attention weights to the hidden state of BiLSTM Focus on time steps with high correlation to the true label (such as positions corresponding to high probability components in mixed stones):

[0362] ;

[0363] where, , are learnable parameters, is the attention weight at the t-th time step, is the feature vector with fused attention.

[0364] 5.3 Final probability output

[0365] The final probability distribution is output by a fully connected layer and a softmax function:

[0366] where, is the output layer weight, is the bias, is the final predicted probability of the k-th stone.

[0367] Algorithm synergy workflow (complete steps) 1. Feature extraction: get and by S7-S8 steps, merge into ;

[0368] 2. Random forest processing:

[0369] Input train RF, output feature importance and prediction probability ;

[0370] Calculate to get weighted features ;

[0371] 3. XGBoost processing: input train XGBoost, output initial probability ;

[0372] 4. Probability fusion: calculate according to the formula = 0.3);

[0373] 5. BiLSTM prediction: input sequence into BiLSTM, output by formula calculation.

[0374] Synergistic effect and technical contribution

[0375] 1. Feature selection enhancement: the feature importance of random forest can filter redundant features (such as some texture features), so that XGBoost focuses on key features (such as oxalic acid concentration, calcium oxalate monohydrate crystal aspect ratio), reducing the calculation amount by 30%;

[0376] 2. Probability fusion robustness: The probability fusion of RF and XGBoost reduces the bias of a single model, especially for rare stone types (such as silica stones), and the prediction accuracy is improved from 86% to 90%;

[0377] 3. Time sequence association capture: BiLSTM deep modeling of fusion probability further excavates the association between components (such as the co-occurrence rule of calcium oxalate and calcium phosphate), and the mixed stone F1 value is improved from 0.92 to 0.94.

[0378] Through three-level interaction of feature weighting, probability fusion, and time sequence modeling, a "screening-optimization-prediction" collaborative closed loop is formed to ensure deep utilization of multi-modal data and improve prediction accuracy.

[0379] The stone component multi-modal prediction system based on urine analysis applies any one of the stone component multi-modal prediction methods based on urine analysis described above, and the stone component multi-modal prediction system based on urine analysis comprises:

[0380] A data acquisition module is configured to acquire a multi-modal data set of urine of a stone patient, wherein the multi-modal data set comprises metabolic detection data and a crystallization image;

[0381] A crystallization type recognition module is configured to process the crystallization image using an improved Inception-ResNet-v2 network to recognize the crystallization type;

[0382] A feature matrix construction module is configured to extract features from the metabolic detection data, and generate a metabolic feature matrix using principal component analysis combined with partial least squares discriminant analysis method;

[0383] A feature vector construction module is configured to extract features from the crystallization type recognition result, extract morphological features and texture features, and generate a crystallization feature vector;

[0384] A fusion module is configured to standardize the metabolic feature matrix and the crystallization feature vector to obtain a fusion feature matrix;

[0385] An initial prediction module is configured to input the fusion feature matrix into an extreme gradient boosting tree model to output an initial probability of stone components;

[0386] A final prediction module is configured to input the initial probability of stone components into a bidirectional long short-term memory network model, fuse features through an attention mechanism, output a probability distribution of stone components, and complete stone component prediction.

Claims

1. A multimodal prediction method for stone composition based on urine analysis, characterized in that, include: A multimodal dataset of urine from patients with kidney stones was obtained, the multimodal dataset including metabolic detection data and crystallization images; An improved Inception-ResNet-v2 network was used to process crystal images and identify crystal types. Feature extraction was performed on the metabolic detection data, and a metabolic feature matrix was generated using principal component analysis combined with partial least squares discriminant analysis. Feature extraction is performed on the crystal type identification results, extracting morphological and texture features to generate a crystal feature vector; The metabolic feature matrix and crystallization feature vector are standardized to obtain the fused feature matrix. Input the fused feature matrix into the extreme gradient boosting tree model to output the initial probability of the stone components; The initial probability of the stone composition is input into the bidirectional long short-term memory network model. After fusing features through the attention mechanism, the probability distribution of the stone composition is output to complete the prediction of the stone composition.

2. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The improvements to the Inception-ResNet-v2 network include: integrating a channel attention module and residual connections, wherein the residual connection is calculated by the formula that the output feature is equal to the sum of the input feature and the result of the convolution operation, and the channel attention module enhances key channel features through global average pooling, fully connected layer mapping, and feature weighting.

3. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The training process of the improved Inception-ResNet-v2 network includes: taking the preprocessed crystallized image as input, using the cross-entropy loss function to calculate the difference between the predicted value and the true crystallization type label; during training, the backbone parameters of the network are frozen first, and only the channel attention module weights are trained. After 20 iterations, all parameters are unfrozen, and the iteration continues for another 30 iterations; the learning rate is initially set to 1e-4, and decays to 0.5 times the previous rate every 10 iterations until the crystallization type recognition accuracy of the model on the validation set does not improve for 5 consecutive iterations, at which point training stops.

4. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The specific operation of principal component analysis combined with partial least squares discriminant analysis includes: first, Z-score standardization of metabolic detection data; then, calculation of the covariance matrix of the standardized data; obtaining principal components through eigenvalue decomposition; selecting the top 100 principal components with a cumulative contribution rate ≥95% as principal component analysis features; inputting the principal component analysis features and stone type labels into the partial least squares discriminant analysis model; extracting 28-dimensional latent variables by maximizing the covariance between features and labels; and finally concatenating the 100-dimensional principal component analysis features with the 28-dimensional partial least squares discriminant analysis latent variables to generate a 128-dimensional metabolic feature matrix.

5. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The parameters for constructing the gray-level co-occurrence matrix during texture feature extraction include: based on the gray-level image of the crystalline region, the pixel distance is set to 1 and the angle is 0° / 45° / 90° / 135°. After calculating the gray-level co-occurrence matrix at each angle, four types of texture parameters, namely contrast, energy, entropy and correlation, are extracted respectively. The average value of the same type of parameters at the four angles is taken to obtain 192-dimensional texture features.

6. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The training process for the extreme gradient boosting tree model and the bidirectional long short-term memory network model includes: the training set consists of 1200 clinical samples validated by infrared spectroscopy, divided into training and test sets in a 7:3 ratio; data augmentation is performed on the crystallization images by random rotation ±15° and scaling by 0.8-1.2 times, and metabolite concentration data is augmented by adding Gaussian noise; the hyperparameters of the extreme gradient boosting tree model include 100 trees, a maximum depth of 6, and a learning rate of 0.1; the hidden layer dimension of the bidirectional long short-term memory network model is 64, using the Adam optimizer with a learning rate of 1e-4.

7. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The training process of the extreme gradient boosting tree model also includes: using the cross-entropy loss function to measure the deviation between the initial probability and the true stone component label; selecting the optimal split point through a greedy algorithm, with the split criterion being to maximize information gain. When calculating the information gain, the gradient sum of the left and right subtrees and the Hessian sum need to be considered. At the same time, a regularization term is introduced to control the complexity of the tree and avoid model overfitting. After each iteration during training, the prediction accuracy of the training set and the validation set is calculated, and the model parameters with the highest accuracy on the validation set are used as the final parameters.

8. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The specific processing steps of the attention mechanism in the bidirectional long short-term memory network model include: firstly, calculating the attention score for the hidden state at the output time step, which is obtained by inputting the hidden state into a fully connected layer with ReLU activation and then normalizing it using the Sigmoid function; secondly, weighting and summing the hidden states at the time step according to the attention score to obtain the feature vector of fused attention; and thirdly, inputting the feature vector into the fully connected layer and combining it with the Softmax function to output the final probability distribution of the stone components.

9. The multimodal prediction method for stone composition based on urine analysis according to claim 1, characterized in that, The morphological feature extraction process includes: first, identifying individual crystalline regions through binarized images; then, using a contour detection algorithm to obtain the edge coordinates of each crystal; calculating the area, perimeter, longest axis, and shortest axis of the crystal based on the coordinates; finally, deriving the aspect ratio and roundness based on the calculated parameters; and finally, averaging the parameters of all crystals to obtain 64-dimensional morphological features.

10. A multimodal prediction system for stone composition based on urine analysis, characterized in that, The method for multimodal prediction of stone composition based on urine analysis according to any one of claims 1 to 9, wherein the multimodal prediction system for stone composition based on urine analysis comprises: The data acquisition module is used to acquire a multimodal dataset of urine from patients with kidney stones. The multimodal dataset includes metabolic detection data and crystallization images. The crystal type identification module is used to process crystal images using an improved Inception-ResNet-v2 network to identify the crystal type; The feature matrix construction module is used to extract features from metabolic detection data. It uses principal component analysis combined with partial least squares discriminant analysis to generate a metabolic feature matrix. The feature vector construction module is used to extract features from the crystal type identification results, extracting morphological and texture features to generate crystal feature vectors; The fusion module is used to standardize the metabolic feature matrix and the crystallization feature vector to obtain the fused feature matrix; The initial prediction module is used to input the fused feature matrix into the extreme gradient boosting tree model and output the initial probability of the stone component. The final prediction module is used to input the initial probability of the stone composition into the bidirectional long short-term memory network model, fuse features through the attention mechanism, and output the probability distribution of the stone composition to complete the prediction of the stone composition.

Images