Method and device for automatic analysis of urine samples combined with microscope hyperspectral imaging technology

By combining microscopic hyperspectral imaging technology and machine learning algorithms, a urine sample formation object detection model is constructed, which solves the problem of time-consuming, labor-intensive and insufficient accuracy in urine formation analysis, and realizes automatic analysis and accuracy of urine samples Identify.

CN118068030BActive Publication Date: 2024-12-06EAST CHINA NORMAL UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410219590.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-28
Publication Date
2024-12-06
Estimated Expiration
2044-02-28

AI Technical Summary

Technical Problem

The existing urine formation analysis methods rely on microscopic observation and manual recognition, which are time-consuming, labor-intensive, subjective, and insufficient accuracy, especially the poor interpretability of the recognition model based on deep learning and sensitivity to crystallographic morphology changes. Sex affects accuracy.

Method used

Combining microscopic hyperspectral imaging technology and machine learning algorithms, through the spectral and spatial feature analysis of microscopic hyperspectral images, a urine sample formation target detection model is built to realize automatic analysis of urine samples, and improve the accuracy and reliability of the analysis. .

Benefits of technology

It realizes automated analysis of urine samples, improves the accuracy and reliability of urine analysis, provides more comprehensive and objective patient health information, and makes up for the shortcomings of traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118068030B_ABST
    Figure CN118068030B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and device for automatic analysis of urine samples in combination with microscope hyperspectral imaging technology, the method comprising: collecting urine samples; performing microscopic hyperspectral full-film imaging of the urine samples to obtain microscopic hyperspectral images; constructing a urine sample shaped component target detection model, inputting the microscopic hyperspectral image into the urine sample shaped component target detection model, and obtaining the category and coordinate frame of the urine sample shaped component, wherein the urine sample shaped component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images. The present invention can automatically realize urine sample dry chemical analysis and urine sample shaped component microscopic analysis, and assists in component classification and identification in combination with the spectral characteristics of urine shaped components. Compared with the prior art, it has the advantages of rich information, high precision and strong robustness.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of urine formed component analysis, and in particular to a method and device for automatically analyzing urine samples in combination with microscope hyperspectral imaging technology. Background Art

[0002] The analysis of urine formed components (urine sediment) has important diagnostic value in clinical laboratories. By observing the particles, cells, crystals and other components in urine sediment, doctors can obtain important information about the health of the patient's urinary system. This analysis is essential for the diagnosis and monitoring of diseases such as kidney disease, urinary tract infection, and stone formation. However, at present, the types of crystals in urine sediment are usually observed by microscope and manually identified, relying on the experience and professional knowledge of laboratory technicians, which are time-consuming, labor-intensive, highly subjective, and inaccurate. With the development of computer-aided diagnosis technology, the urine sediment crystal identification method based on microscopic images has brought new opportunities for urine sediment identification. In the patent with publication number CN201910615378, Harbin Engineering University disclosed a urine sediment image recognition system and method based on deep learning; in the patent with publication number CN202211073150, a method for identifying urine red blood cells, white blood cells and calcium oxalate crystals based on deep learning is disclosed, and the classification accuracy rate reaches 89.1%. However, the current urine sediment crystal recognition method of color images relies on a large amount of manually annotated data, and the recognition model based on deep learning is a black box model with poor interpretability; and the crystal morphology varies greatly, affecting the accuracy of the deep learning method. Therefore, the present invention provides a method and system for automatic analysis of urine samples combined with microscope hyperspectral imaging technology. Summary of the invention

[0003] The purpose of the present invention is to provide an automatic urine sample analysis method and device combined with microscope hyperspectral imaging technology, which can realize automatic sampling, collection and analysis of urine samples, and combine machine learning algorithms to analyze spectral characteristics and spatial characteristics, which can make up for the shortcomings of traditional urine sediment analysis methods and improve the accuracy and reliability of urine analysis by introducing information from spectral dimensions.

[0004] To achieve the above object, the present invention provides a method for automatic analysis of urine samples in combination with microscope hyperspectral imaging technology, which is characterized by comprising:

[0005] Collect urine samples;

[0006] Performing microscopic hyperspectral whole-piece imaging on the urine sample to obtain a microscopic hyperspectral image;

[0007] A urine sample shaped component target detection model is constructed, and the microscopic hyperspectral image is input into the urine sample shaped component target detection model to obtain the category and coordinate frame of the urine sample shaped component, wherein the urine sample shaped component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images.

[0008] Optionally, after collecting the urine sample, the process also includes: quantitatively dropping the sample on a urine analysis test strip to obtain a dry chemical analysis result.

[0009] Optionally, performing microscopic hyperspectral whole-piece imaging on the urine sample, and obtaining a microscopic hyperspectral image includes:

[0010] Dropping the urine sample into a counting cell;

[0011] Dividing the imaging range in the counting pool into a plurality of regions of equal size, and presetting equally spaced points as pre-focusing points;

[0012] Performing automatic focusing according to the pre-focus point, obtaining a focusing result, and performing focal plane modeling based on the focusing result;

[0013] Obtaining the coordinate positions of a plurality of areas of the same size, and obtaining the focus distance according to the focal plane;

[0014] A collection task sequence is generated in a zigzag shape according to the coordinate position and the focus distance to obtain the microscopic hyperspectral image.

[0015] Optionally, constructing the urine sample shaped component target detection model includes: constructing the urine sample shaped component target detection model by combining the hyperspectral image space, a backbone encoder of spectral features, a Transformer encoder for target detection regression prediction, a transformer decoder, an object query, and a fronthaul network FFN, wherein the backbone encoder is constructed by replacing the first layer conv3-3 of Resnet50 with conv30-3 and adding a batchnorm layer.

[0016] Optionally, training the urine sample tangible component target detection model by using annotated microscopic hyperspectral urine sample images includes:

[0017] Performing data enhancement based on annotated microscopic hyperspectral urine sample images to obtain a training set, wherein the data enhancement includes random inversion, rotation, and cut and paste data enhancement methods;

[0018] Training the urine sample tangible component target detection model using the training set;

[0019] The loss function used in training is:

[0020] L=L CE +L box

[0021] Among them, L CE To supervise the prediction of the cross entropy loss of the formed classification category, L box The correct loss for supervised regression bounding box prediction.

[0022] On the other hand, it also provides a urine sample automatic analysis device combined with microscope hyperspectral imaging technology, including: a urine sample collection device, a microscopic hyperspectral imaging device and a control terminal;

[0023] The urine sample collection device is used to collect urine samples;

[0024] The microscopic hyperspectral imaging device is used to perform microscopic hyperspectral full-piece imaging on the urine sample to obtain a microscopic hyperspectral image;

[0025] The control terminal is used to construct a urine sample shaped component target detection model, input the microscopic hyperspectral image into the urine sample shaped component target detection model, and obtain automatic microscopic analysis results, wherein the urine sample shaped component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images, and the automatic microscopic analysis results include the category and coordinate frame of the urine sample shaped component.

[0026] Optionally, the device further comprises a dry chemical analysis device, which is used to quantitatively drop samples on urine analysis test strips to obtain dry chemical analysis results.

[0027] Optionally, the urine sample collection device includes: a urine sample test tube, a urine sample test tube rack, a sample delivery module, and a sample suction module;

[0028] The urine sample tube is used to collect the urine sample and record and match the patient information and test results through the attached information barcode label;

[0029] The urine sample test tube rack is used to place a plurality of urine sample test tubes;

[0030] The sample delivery module is used to deliver the urine in the urine sample tube into the sample suction module, scan the information barcode label, and transmit the scanned information to the control terminal;

[0031] The sample suction module is used to suck the urine sample in the urine sample test tube and titrate the counting cell and the test paper.

[0032] Optionally, the control terminal includes a communication module, an interaction module, a urine formed component analysis module and an imaging control module;

[0033] The communication module is used to communicate with the microscopic hyperspectral imaging device and the dry chemical analysis device;

[0034] The interaction module is used for the user to interact with the urine sample collection device, the microscopic hyperspectral imaging device and the dry chemical analysis device, and integrate the dry chemical analysis results and the automatic microscopic analysis results;

[0035] The urine formed component analysis module is used to identify and classify the microscopic hyperspectral image to obtain the category and coordinate frame of the urine sample formed components;

[0036] The imaging control module is used to perform full-film imaging on the sample area in the counting pool to obtain the microscopic hyperspectral image.

[0037] Optionally, the sample suction module includes a sample suction needle, a moving mechanism, a cleaning tank, a first conduit and a second conduit;

[0038] The sample aspirating needle is used to aspirate the urine sample in the urine sample test tube;

[0039] The first conduit is used to quantitatively drip samples on the urine analysis test strip under the action of a syringe pump;

[0040] The second conduit is used to quantitatively drip samples into the counting pool under the action of a syringe pump;

[0041] The moving mechanism is used to control the sample suction needle or the counting pool to move to the cleaning pool for cleaning and drying;

[0042] The cleaning pool is used to clean the sample suction needle.

[0043] The beneficial effects of the present invention are as follows: the present invention can automatically realize dry chemical analysis of urine samples and microscopic analysis of formed components of urine samples, and assist in component classification and identification by combining the spectral characteristics of formed components of urine. Compared with the prior art, the present invention has the advantages of rich information, high precision and strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1A schematic diagram of the process of the automatic urine sample analysis method combined with microscope hyperspectral imaging technology according to an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the structure of a urine sample automatic analysis device combined with microscope hyperspectral imaging technology according to an embodiment of the present invention;

[0047] Figure 3 A microscopic hyperspectral urine sample tangible component target detection model based on target detection transformer DETR according to an embodiment of the present invention;

[0048] Figure 4 illustrative diagram of six types of urine sediment crystallization samples according to an embodiment of the present invention;

[0049] Figure 5 It is a three-dimensional data cube visualization of the hyperspectral image and an average spectral curve graph of six types of urine sediment crystallization samples according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Hyperspectral imaging technology can obtain the spectral dimension information of the imaged object. This technology has been widely used in remote sensing, agriculture, food safety and other fields; the application of hyperspectral imaging technology in biomedicine is an emerging research. Microscopic hyperspectral imaging technology combines microscopic imaging and hyperspectral imaging technology. It can extract the spectral dimension information of biological samples while obtaining the spatial information of biological samples and generate a three-dimensional data cube. It is an emerging biomedical clinical diagnosis aid. Biological tissues with different chemical components have different light absorption rates within a certain band range, which is manifested as differences in the spectral curves of hyperspectral images. Combined with machine learning algorithms, microscopic hyperspectral imaging technology can provide useful information for auxiliary diagnosis.

[0053] The use of microscopic hyperspectral imaging technology in urine sediment detection can realize automatic sampling, collection and analysis of urine samples, and combine machine learning algorithms to analyze spectral and spatial characteristics. This can make up for the shortcomings of traditional urine sediment analysis methods. By introducing information in the spectral dimension, the accuracy and reliability of urine analysis can be improved, providing doctors with more comprehensive and objective patient health information, which is expected to promote the technical upgrading of urine sediment analysis in clinical practice.

[0054] This embodiment provides a method for automatically analyzing urine samples in combination with microscope hyperspectral imaging technology, such as Figure 1 As shown, including:

[0055] S1. Collect urine sample;

[0056] Specifically, collecting urine samples and completing the preliminary urine sample preparation work include:

[0057] S101 obtains a midstream urine sample from a patient, mixes the urine sample, precipitates, centrifuges, pours out the urine supernatant, leaves the urine sediment at the bottom, suspends the urine sediment, and obtains a test tube containing the urine sample;

[0058] S102 places several test tubes containing urine samples on a test tube rack, and places the test tube rack in a sample delivery module to complete the preliminary preparation work.

[0059] S2, start the control software interface and drive the sampling device;

[0060] The operator clicks the start sampling button on the software interface, and the control terminal drives the conveyor belt of the sample delivery module through the communication module. When the barcode label of the urine sample tube passes through the barcode scanning module of the sample delivery module, the barcode scanning module is triggered, and the scanned information is transmitted to the control terminal and the urine sample patient ID information is recorded; the urine sample tube continues to be moved to the sample suction module.

[0061] S3, quantitatively dropping a sample on a urine analysis test strip to obtain a dry chemical analysis result;

[0062] Specifically, the operator selects dry chemical analysis on the software interface, and after the sample suction module detects the test tube, it drives the syringe pump to absorb the urine sample, and the catheter quantitatively drips the urine sample onto the urine analysis test strip used for dry chemical analysis under the action of the syringe pump; the test strip is driven to the dry chemical analyzer for dry chemical analysis; the dry chemical analysis result is transmitted to the control terminal through the communication module, and the dry chemical analysis report is displayed on the user interface and saved.

[0063] S4. Perform microscopic hyperspectral full-film imaging of the urine sample to obtain a microscopic hyperspectral image; construct a urine sample shaped component target detection model, input the microscopic hyperspectral image into the urine sample shaped component target detection model, and obtain the category and coordinate frame of the urine sample shaped component, wherein the urine sample shaped component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images.

[0064] Specifically, S401 is controlled by an imaging control algorithm to perform Z-shaped whole-plate microscopic hyperspectral imaging within the counting pool;

[0065] The operator selects urine formed component microscopic analysis on the software interface, and the catheter drips the urine sample into the counting pool quantitatively under the action of the syringe pump, and moves the counting pool to the preset fixed coordinate position including:

[0066] 1. The user sets the acquisition parameters in the software interface. The acquisition parameters set in this embodiment are as follows: the spectral band range is 400-800nm, the number of bands is 60, the exposure time of the hyperspectral image is set to 200ms, the spatial resolution of the hyperspectral image is, and the image is stored in the order of the bands (Band Sequential, BSQ) as the original format raw file;

[0067] 2. Operation: The user clicks the pre-focus button in the software interface to perform pre-focusing: the imaging range of the counting pool is divided into small blocks, and the program presets equally spaced points as pre-focus points. The stage is moved to the pre-focus points for automatic focusing, and the focusing result is obtained. The focal plane modeling is implemented for the focusing result. The focal plane modeling is based on the three-point method and the pre-focus points, and the focal length z in the imaging area is interpolated;

[0068] The pre-focused points are automatically focused to obtain the focusing results, and focal plane modeling is implemented for the focusing results, including: calculating the four pre-focus points that are closest to the horizontal two-dimensional distances of each acquisition point in the sample area (the small blocks into which the imaging range of the counting pool is divided); among the four pre-focus points, two pre-focus points that are closest are used to fit two focal planes with the three-dimensional coordinates of the third and fourth pre-focus points respectively; interpolation is performed based on the two focal planes to calculate the height of each acquisition point in the two planes; the distance from each acquisition point to the two fitted focal planes is calculated and the distance is used as a weight to weight the Z-axis coordinates in the corresponding two planes to obtain the focusing height, and the focal plane modeling is completed.

[0069] 3. After completing the pre-focus, the user clicks on the software interface to start scanning; the control terminal calculates each coordinate position of the small block in the counting pool imaging area, scans each small block in the area according to the Z-shaped generation acquisition task sequence, drives the stage to the specified coordinate point for imaging and saves it in the storage component of the control terminal in the form of an image block.

[0070] The control terminal calculates each coordinate position of the small block in the imaging area of ​​the counting pool, and generates a collection task sequence in a Z-shaped manner, including: dividing the large imaging area (square) into small blocks of the imaging area of ​​each field of view, obtaining the coordinates (coord_x, coord_y) of the upper left corner of each small block, and the coordinate position (coord_x, coord_y) is divided equally according to the imaging area, and each small block in the imaging area has a corresponding focus distance coord_z in the focal plane modeled in 2; the stage moves to (coord_x, coord_y, coord_z) for imaging. The collection task sequence refers to the coordinates (coord_x, coord_y, coord_z) of the small blocks of the imaging area arranged in a Z-shaped manner, and coord_z is the value at (coord_x, coord_y) calculated by the focal plane modeling.

[0071] S402 performs data preprocessing on the stored microscopic hyperspectral image, including:

[0072] During the imaging process of the microscopic hyperspectral system, there are external conditions such as system equipment noise and light intensity, which will cause the absorption characteristics of the spectrum of the tissue sample to be affected by external conditions, and there will be noise, light spots and bands, which will interfere with subsequent analysis. Therefore, it is necessary to take pre-processing measures for the original hyperspectral image to eliminate the interference caused by the outside world and retain the spectral characteristics of the tissue sample itself. In this embodiment, spectral correction is performed based on the Beer-Lambert law:

[0073]

[0074] Among them, I raw is the original hyperspectral image collected, I blank is a standard blank image, I pre The output preprocessed image is shown in Figure 2. Since the transmittance of a tissue sample is only related to the light intensity passing through itself and the blank glass slide, the spectrum of the tissue sample microscopic hyperspectral image can be corrected by dividing the original image by the blank image.

[0075] After spectral correction, linear function normalization is used to apply linear transformation to the original data and map the result range to [0, 1]. The calculation formula is:

[0076]

[0077] Among them, I min ,I max are the minimum and maximum values ​​of the pixels in the data set, respectively. re-scal is the output hyperspectral data after linear normalization. Figure 5 Shown is a hyperspectral 3D data cube of triple calcium phosphate crystals after preprocessing.

[0078] In this embodiment, the formed elements of the urine sample analyzed include: red blood cells, white blood cells, and urine sediment crystal samples: calcium oxalate crystals (colorless square glittering octahedrons under a microscope), cystine crystals (colorless hexagonal plate-like crystals), phosphate crystals (roof-shaped or prism-shaped, sometimes fern-shaped), uric acid crystals (yellow or dark red rhombus, tri-rhombus, rectangular, rhombus, etc. under a microscope), ammonium urate (yellow-brown opaque, root-shaped, starfish-shaped, echinococcoid-shaped, and dumbbell-shaped), and triple calcium phosphate crystals (mostly spherical or irregular in shape); Figure 4 Shown are pseudo-color image visualizations of six types of urine crystal samples; Figure 5 The following are visualization images and spectral curves of various types of urine sediment crystals. Different crystals show great differences in spectra. The differences in spectral dimensions can improve the ability to distinguish crystal types and improve classification accuracy.

[0079] S403 identifies and classifies the microscopic hyperspectral images in the imaging area of ​​the counting pool based on the urine sample formed component analysis algorithm, including:

[0080] The urine sample formed component analysis algorithm is a microscopic hyperspectral urine sample formed component target detection model based on the target detection transformer DETR. The model outputs the category c of the urine sample formed component and the coordinate frame bounding box (x p ,y p , h, w), where x p ,y p is the coordinate point of the formed component of the urine sample, h, w are the length and width of the coordinate frame;

[0081] like Figure 3 The figure shows the urine sample formed component analysis model. A microscopic hyperspectral urine sample formed component target detection model based on the target detection transformer DETR is constructed, which specifically includes the following steps:

[0082] 1. Collecting urine sample microscopic hyperspectral image data of formed components. In this embodiment, a urine sample microscopic hyperspectral imaging system is used to collect images, and a professional microscopy doctor uses labelme software to annotate the pseudo-color image; the pseudo-color image is synthesized by the RGB three bands of the urine sample microscopic hyperspectral image; in this embodiment, a total of 120 microscopic hyperspectral urine sample images are collected and annotated, and the average number of annotated crystals of each type is 50;

[0083] 2. Construct a microscopic hyperspectral urine sample tangible component target detection model HSI-DETR based on target detection transformer DETR. The HSI-DETR consists of a backbone encoder for extracting joint hyperspectral image space and spectral features, a Transformer encoder for target detection regression prediction, a transformer decoder, an object query, and a forward network FFN.

[0084] In this embodiment, the backbone encoder replaces the first layer conv3-3 of Resnet50 with conv30-3 and adds a batchnorm layer. Except for the first convolutional layer, the parameters of other layers are initialized to resnet50 pre-trained on ImageNet. The input of the backbone encoder is a hyperspectral image, denoted by X∈R h×w×c .

[0085] In this embodiment, the Transformer encoder, transformer decoder, objectquery and fronthaul network FFN are all DETR model settings and are not modified here.

[0086] 3. Training a microscopic hyperspectral urine sample tangible component target detection model based on target detection transformer DETR:

[0087] The following data enhancement methods were used in the training: random inversion and rotation, random cutting of urine sample components from other images according to the size of the bounding box and pasting them in the blank area of ​​the image. The specific embodiment adopted the cut and paste data enhancement method, which would cut the image according to the labeled area (crystallized bounding box and labeled category information) and paste it in the unlabeled area (blank area) of another image, expanding the data volume to 300 microscopic hyperspectral images for model training;

[0088] The loss function used in training is:

[0089] L=L CE +L box

[0090] Among them, L CE To supervise the prediction of the cross entropy loss of the formed classification category, L box Predict the correct loss for supervised regression bounding box;

[0091] The model training hyperparameters are set as follows: learning rate 10 -4 , the weight decreases by 10 -5 , AdamW optimizer was used, the input image size was 384×384×30, batch seize was 4, 200 epochs were trained, and the optimal model saved on the validation set was used as the urine sample formed component recognition model in this embodiment.

[0092] S404 counts the number of various types of urine sample formed components according to the model output results, and generates a microscopic examination report of the urine sample formed components.

[0093] S5. Save the urine sample test results.

[0094] The operating user can browse and print the microscopic examination report in the software interface.

[0095] Save the urine sample test results, the user clicks to complete the analysis, and the control terminal drives the sample suction module and the counting pool to the cleaning tank, and cleans and dries the sampling needle, catheter 1, catheter 2 and the counting pool for the next urine sample analysis; repeat the above steps to perform dry chemical analysis of urine samples and automatic analysis of formed components in batches.

[0096] like Figure 2 As shown, this embodiment also provides a urine sample automatic analysis device combined with microscope hyperspectral imaging technology, including: a urine sample collection device, a microscopic hyperspectral imaging device and a control terminal;

[0097] A urine sample collection device, used for collecting urine samples;

[0098] Specifically, the urine sample collection device includes: a urine sample test tube, a urine sample test tube rack, a sample delivery module, and a sample suction module;

[0099] The function of the urine sample tube is to collect urine samples from patients. The tube should be sealed to ensure the integrity of the urine sample, and a patient information barcode label should be affixed to the urine sample tube to record and match patient information and test results.

[0100] The urine sample test tube rack is used to place a number of urine sample test tubes. In this embodiment, the maximum number of test tubes that can be placed is 5. The size of the urine sample test tube rack should match the sample delivery module.

[0101] The function of the sample delivery module is to deliver the urine in the urine sample tube to the subsequent sample suction module. The device is realized through appropriate mechanical modules such as conveyor belts, drive motors, etc., and can accurately control the position movement; the sample delivery module is equipped with a barcode scanning module. When the barcode on the urine sample tube passes by, the scanner is triggered to start, and the barcode on the tube is scanned. The scanned information is transmitted to the control terminal to record the patient information.

[0102] The function of the sample suction module is to suck the urine sample from the urine sample tube and titrate it into the designated counting pool and test paper. The sample suction module consists of a sample suction needle, a moving mechanism, a syringe pump, a cleaning pool and two catheters. The sample suction needle is used to suck the urine sample from the test tube; one catheter quantitatively drips the urine sample onto the urine analysis test strip for dry chemical analysis under the action of the syringe pump; the other catheter quantitatively drips the urine sample into the counting pool under the action of the syringe pump; after the sample suction is completed, the sample suction needle moves to the cleaning pool under the control of the moving mechanism for cleaning and drying for the next sample.

[0103] The counting pool is used to hold a quantitative urine sediment sample and to serve as a carrier for the urine sample to be sent to the microscopic hyperspectral imaging device for imaging. The counting pool has calibrated scale lines, can hold a quantitative sample, is light-transmitting and has no background interference for the observation of urine components; the counting pool is controlled by a moving mechanism and needs to be moved to a cleaning pool for cleaning and drying after use.

[0104] The function of urine analysis strips is to conduct dry chemical analysis. The urine analysis strips contain various chemical reagents that can react with specific components in urine. The function of dry chemical analyzers is to analyze urine test strips and give parameters of various chemical components in urine such as glucose, protein, ketone bodies, etc. The analyzer can transmit the analysis results to the control terminal.

[0105] A microscopic hyperspectral imaging device, used for performing microscopic hyperspectral full-film imaging of urine samples to obtain microscopic hyperspectral images;

[0106] Specifically, the function of the microscopic hyperspectral imaging device is to image the formed components of urine in the counting pool, and the device is composed of the following modules: a light source, which uses an LED light source in this embodiment; an objective lens, which is used to magnify the formed components of urine in the counting pool, and a 40-fold magnification objective lens is used in this embodiment; a stage, which is used to load the counting pool and move the counting pool between the light source and the objective lens, and the stage can be controlled to move by a stage driver; the stage driver, which communicates with the control terminal, controls the stage to move to the specified coordinates, and transmits the stage coordinates to the control terminal; a spectrometer, which is used to switch the color imaging optical path and the hyperspectral imaging optical path; a color camera, which is used for color image imaging, and a CMOS camera is used in this embodiment; a grayscale camera is used for hyperspectral image imaging, and a grayscale CCD camera is used in this embodiment; the acousto-optic tunable filter AOTF is used to control the wavelength range of the passing light, and communicates with the control terminal to switch the wavelength band of the same light.

[0107] The control terminal is used to construct a urine sample shaped component target detection model, input the microscopic hyperspectral image into the urine sample shaped component target detection model, and obtain the automatic microscopic analysis result, wherein the urine sample shaped component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images, and the automatic microscopic analysis result includes the category and coordinate frame of the urine sample shaped component.

[0108] Specifically, the function of the control terminal is to control the urine formed component analysis device combined with the microscopic hyperspectral imaging system: the software interface provided by the control terminal allows the operator to interact with the hardware in the software (the user interacts with the urine sample collection device, the microscopic hyperspectral imaging device and the dry chemical analysis device), integrates the dry chemical analyzer and automatic microscopic analysis, and prints various indicator reports for doctors' reference; the communication module realizes the communication between the software and the hardware devices, and controls the behavior of each hardware device; the storage device provides storage support for images and outputs; the role of the imaging control module is to automatically collect microscopic hyperspectral images of samples in the counting pool; the role of the urine formed component analysis module is to combine the spectral information and spatial information in the microscopic hyperspectral image, perform component analysis and counting on the collected image, and give statistical results and reports of individual components.

[0109] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for automatically analyzing urine samples in combination with microscope hyperspectral imaging technology, characterized in that: include: Collect urine samples; Performing microscopic hyperspectral whole-piece imaging on the urine sample to obtain a microscopic hyperspectral image; Constructing a urine sample tangible component target detection model, inputting the microscopic hyperspectral image into the urine sample tangible component target detection model, and obtaining the category and coordinate frame of the urine sample tangible component, wherein the urine sample tangible component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images; Performing microscopic hyperspectral full-piece imaging on the urine sample, obtaining a microscopic hyperspectral image includes: Dropping the urine sample into a counting cell; Dividing the imaging range in the counting pool into a plurality of regions of equal size, and presetting equally spaced points as pre-focusing points; Performing automatic focusing according to the pre-focus point, obtaining a focusing result, and performing focal plane modeling based on the focusing result; Obtaining the coordinate positions of a plurality of areas of the same size, and obtaining the focus distance according to the focal plane; Generate a collection task sequence in a zigzag shape according to the coordinate position and the focus distance to obtain the microscopic hyperspectral image; Constructing the urine sample tangible component target detection model includes: constructing the urine sample tangible component target detection model by combining the hyperspectral image space, the spectral feature backbone encoder, the transformer encoder for target detection regression prediction, the transformer decoder, the object query and the forward network FFN, wherein the backbone encoder is constructed by replacing the first layer conv3-3 of Resnet50 with conv30-3 and adding a batchnorm layer.

2. The method for automatic urine sample analysis combined with microscope hyperspectral imaging technology according to claim 1, characterized in that: After collecting the urine sample, it also includes: quantitatively dropping the sample on the urine analysis test strip to obtain the dry chemical analysis results.

3. The method for automatic urine sample analysis combined with microscope hyperspectral imaging technology according to claim 1, characterized in that: Training the urine sample shaped component target detection model by annotated microscopic hyperspectral urine sample images includes: Performing data enhancement based on annotated microscopic hyperspectral urine sample images to obtain a training set, wherein the data enhancement includes random inversion, rotation, and cut and paste data enhancement methods; Training the urine sample tangible component target detection model using the training set; The loss function used in training is: in, To supervise the prediction of the cross entropy loss of the formed classification category, The correct loss for supervised regression bounding box prediction.

4. The automatic urine sample analysis device combined with microscope hyperspectral imaging technology is characterized in that: include: Urine sample collection device, microscopic hyperspectral imaging device and control terminal; The urine sample collection device is used to collect urine samples; The microscopic hyperspectral imaging device is used to perform microscopic hyperspectral full-piece imaging on the urine sample to obtain a microscopic hyperspectral image; Performing microscopic hyperspectral full-piece imaging on the urine sample, obtaining a microscopic hyperspectral image includes: Dropping the urine sample into a counting cell; Dividing the imaging range in the counting pool into a plurality of regions of equal size, and presetting equally spaced points as pre-focusing points; Performing automatic focusing according to the pre-focus point, obtaining a focusing result, and performing focal plane modeling based on the focusing result; Obtaining the coordinate positions of a plurality of areas of the same size, and obtaining the focus distance according to the focal plane; Generate a collection task sequence in a zigzag shape according to the coordinate position and the focus distance to obtain the microscopic hyperspectral image; The control terminal is used to construct a urine sample tangible component target detection model, input the microscopic hyperspectral image into the urine sample tangible component target detection model, and obtain an automatic microscopic analysis result, wherein the urine sample tangible component target detection model is obtained by training with annotated microscopic hyperspectral urine sample images, and the automatic microscopic analysis result includes the category and coordinate frame of the urine sample tangible component; Constructing the urine sample tangible component target detection model includes: constructing the urine sample tangible component target detection model by combining the hyperspectral image space, the spectral feature backbone encoder, the transformer encoder for target detection regression prediction, the transformer decoder, the object query and the forward network FFN, wherein the backbone encoder is constructed by replacing the first layer conv3-3 of Resnet50 with conv30-3 and adding a batchnorm layer.

5. The automatic urine sample analysis device combined with microscope hyperspectral imaging technology according to claim 4 is characterized in that ,The device also includes a dry chemical analysis device, which is used to quantitatively drop samples on the urine analysis test strip to obtain the dry chemical analysis results.

6. The automatic urine sample analysis device combined with microscope hyperspectral imaging technology according to claim 5 is characterized in that: The urine sample collection device comprises: a urine sample test tube, a urine sample test tube rack, a sample delivery module, and a sample suction module; The urine sample tube is used to collect the urine sample and record and match the patient information and test results through the attached information barcode label; The urine sample test tube rack is used to place a plurality of urine sample test tubes; The sample delivery module is used to deliver the urine in the urine sample tube into the sample suction module, scan the information barcode label, and transmit the scanned information to the control terminal; The sample suction module is used to suck the urine sample in the urine sample test tube and titrate the counting cell and the test paper.

7. The automatic urine sample analysis device combined with microscope hyperspectral imaging technology according to claim 6 is characterized in that: The control terminal includes a communication module, an interaction module, a urine formed component analysis module and an imaging control module; The communication module is used to communicate with the microscopic hyperspectral imaging device and the dry chemical analysis device; The interaction module is used for the user to interact with the urine sample collection device, the microscopic hyperspectral imaging device and the dry chemical analysis device, and integrate the dry chemical analysis results and the automatic microscopic analysis results; The imaging control module is used to perform full-film imaging on the sample area in the counting pool to obtain the microscopic hyperspectral image; The urine formed component analysis module is used to identify and classify the microscopic hyperspectral image to obtain the category and coordinate frame of the urine sample formed components.

8. The automatic urine sample analysis device combined with microscope hyperspectral imaging technology according to claim 6 is characterized in that: The sample suction module comprises a sample suction needle, a moving mechanism, a cleaning tank, a first conduit and a second conduit; The sample aspirating needle is used to aspirate the urine sample in the urine sample test tube; The first conduit is used to quantitatively drip samples on the urine analysis test strip under the action of a syringe pump; The second conduit is used to quantitatively drip samples into the counting pool under the action of a syringe pump; The moving mechanism is used to control the sample suction needle or the counting pool to move to the cleaning pool for cleaning and drying; The cleaning pool is used to clean the sample suction needle.

Citation Information

Patent Citations

  • A Deep Learning-Based Urine Sediment Image Recognition System and Method

    CN110473167B

  • Open set identification method for urine red blood cells, white blood cells and calcium oxalate crystals

    CN115471697A

  • Deep learning and context correlation-based urine visible component detection method

    CN113256637A

  • Full-slide data cube acquisition method based on microscopic hyperspectral imaging platform

    CN115060367A

  • Urine sample hyperspectral image-based glomerular disease classification diagnosis model construction method

    CN117036269A