Lateral stream dark field imaging video data processing method and system

Abstract

The invention discloses a sidestream dark field imaging video data processing method and system. The method comprises the following steps: acquiring a sidestream dark field video containing a target capillary; inputting the side stream dark field video into a trained key point tracking model, and marking key points in the side stream dark field video; the key points comprise boundary points of microvessels and starting points and ending points of red blood cell columns; calculating the change value of the diameter of the erythrocyte column in the capillary before and after the leukocyte passes through the capillary according to the key point information so as to obtain the perfusion boundary region value of the erythrocyte column; the glycocalyx injury index is constructed according to the red blood cell column perfusion boundary region value, so that medical workers can further evaluate the glycocalyx injury condition of the target microvessel according to the injury index. According to the method, the capillary video can be processed through the deep neural network model to replace the original whole-course manual observation and analysis mode, so that the workload is reduced, the operation steps are simplified, and the time is saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image analysis, and particularly relates to a method and system for processing side-stream dark-field imaging video data. Background Art

[0002] The microvascular system is an important part of the human circulatory system, and changes in its structure and function are closely related to the development of various diseases. As a polysaccharide structure on the surface of microvascular endothelial cells, the degree of damage to the glycocalyx can be used as a biomarker for disease diagnosis and treatment. Especially in systemic inflammatory response syndrome and sepsis, the deterioration of endothelial glycocalyx plays a key role in vascular barrier dysfunction and ultimate organ failure. Therefore, early detection of glycocalyx damage has become an important goal in intensive care.

[0003] However, traditional methods for evaluating the degree of glycocalyx damage usually require invasive operations. For example, the Chinese invention patent with the application number CN202211169317.9 discloses the application of glycocalyx shedding markers in the diagnosis and prevention of early renal injury in newly diagnosed type 2 diabetic nephropathy, which requires obtaining fasting venous blood from patients and detecting the products of endothelial glycocalyx shedding in the blood, such as syndecan-1 and hyaluronic acid, to evaluate the damage of endothelial glycocalyx and thus judge early renal injury in type 2 diabetic nephropathy.

[0004] In vivo microscopy imaging requires researchers or medical staff to manually screen images before and after the passage of blood vessels and white blood cells for measurement. The degree of automation is low, relying on manual observation and analysis of glycocalyx damage. The efficiency is low and error-prone, and the operation steps are complicated and the analysis time is long. These operations not only cause inconvenience to patients, limit the possibility of continuous monitoring, lack real-time and dynamic measurement means, and cannot capture the immediate changes in microvessels. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for processing side-stream dark-field imaging video data, which can partially solve or alleviate the above deficiencies in the prior art, and can process microvascular videos through a deep neural network model to replace the original method of full-process manual observation and analysis, thereby reducing the workload, simplifying the operation steps and saving time.

[0006] To solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions: In the first aspect of the present invention, there is provided a method for processing side-stream dark-field imaging video data, including: [[ID=2s]] Collecting a side-stream dark-field video containing target microvessels; Inputting the side-stream dark-field video into a trained key-point tracking model to mark the key points in the side-stream dark-field video; the key points include the boundary points of microvessels, the starting point and the ending point of the erythrocyte column; Calculate the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through the capillary according to the key point information, so as to obtain the perfusion boundary zone value of the red blood cell column.

[0007] As an improvement, it further includes the step of constructing a glycocalyx damage index according to the perfusion boundary zone value of the red blood cell column, so that the glycocalyx damage condition of the target microvessel can be evaluated based on this glycocalyx damage index.

[0008] As an improvement, the step of constructing a glycocalyx damage index according to the perfusion boundary zone of the red blood cell column includes: Using the formula: Calculate the glycocalyx damage index; where GI is the glycocalyx damage index, PBR is the perfusion boundary zone value, and GI is positively correlated with PBR; RCWB is the diameter of the red blood cell column before the white blood cell passes through, RCWA is the diameter of the red blood cell column after the white blood cell passes through, RCW is the change value of the red blood cell column; There is a proportional relationship between the glycocalyx damage index and the glycocalyx damage condition of the microvessel.

[0009] As an improvement, the key point tracking model is one of the DeepLabCut, ResNet, EfficientNet, MobileNet deep neural network models.

[0010] As an improvement, the DeepLabCut deep neural network model sequentially includes an input layer, a residual network, and an output layer; the input layer and the residual network layer are connected by a convolutional layer and a max pooling layer; it also includes a deconvolutional layer connected to the output layer, and the deconvolutional layer is used for coordinate regression to track key points and obtain the key point position information of the blood vessel in each frame, so that the key points are displayed in the side-stream dark field video.

[0011] As an improvement, the residual network includes a plurality of residual blocks, and each residual block includes a convolutional calculation unit, a batch normalization unit, a rectified linear unit, and a max pooling unit.

[0012] As an improvement, eliminate the perfusion boundary zone values greater than the boundary zone threshold, or crop the video with abnormal widening of the red blood cell column diameter caused by vasospasm, microthrombus formation or disappearance before inputting the key point tracking model, so as to eliminate the abnormal widening of the red blood cell column diameter caused by vasospasm, microthrombus formation or disappearance.

[0013] As an improvement, import the key point information into MATLAB, and use MATLAB to calculate the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through the capillary.

[0014] As an improvement, frame images when white blood cells pass through are selected from the side - flow dark - field videos, and the frame images before and after the said frame images are imported into ImageJ, and lines are drawn using the Straight tool; the diameter of the red - blood - cell column before the white blood cell passes through and the diameter of the red - blood - cell column after the white blood cell passes through are determined according to the gray - scale changes on the line segment, and the perfusion boundary - zone value of the red - blood - cell column is calculated to verify the accuracy of the key - point tracking model.

[0015] As an improvement, when training the key - point tracking model, several frame images are randomly selected from the collected side - flow dark - field videos containing microvessels for manual key - point marking, thus forming a training set.

[0016] The present invention also provides a side - flow dark - field imaging video data processing system, including: A video acquisition module, used for acquiring side - flow dark - field videos containing target microvessels; A key - point tracking module, used for inputting the side - flow dark - field video into a trained key - point tracking model to mark the key points in the side - flow dark - field video; the key points include the boundary points of the microvessel, the starting point and the ending point of the red - blood - cell column; A perfusion boundary - zone value calculation module, used for calculating the change value of the diameter of the red - blood - cell column in the capillary before and after the white blood cell passes through the capillary according to the key - point information, so as to obtain the perfusion boundary - zone value of the red - blood - cell column; An injury assessment module, used for constructing a glycocalyx injury index according to the perfusion boundary - zone value of the red - blood - cell column.

[0017] Furthermore, the side - flow dark - field imaging video data processing method is for non - diagnostic or non - therapeutic purposes.

[0018] Beneficial effects: The present invention combines deep learning, image - processing technology and medical image analysis, and provides a side - flow dark - field imaging video data processing method and system, which has multiple advantages in the medical field.

[0019] Traditional methods for evaluating the degree of glycocalyx injury, such as collecting fasting venous blood to detect related products, not only require invasive operations but also are difficult to achieve continuous monitoring; intravital microscopy imaging relies on manual screening and analysis of images, with complex operation steps and time - consuming. The present invention uses a deep - neural - network model to process microvessel videos, automatically collects, marks, tracks and analyzes data, reduces manual intervention, simplifies the operation process, improves work efficiency, saves time, and can also perform continuous monitoring.

[0020] The present invention avoids invasive operations of traditional methods, such as collecting venous blood, reduces the risk of patient infection and pain, provides a more convenient and comfortable detection method for patients, and at the same time increases the feasibility of continuous monitoring, and can timely capture the immediate changes of microvessels.

[0021] By training a deep neural network model with key point marking for a large number of images, it can accurately identify and track key points such as microvessels and red blood cell columns. Using MATLAB to calculate the change in the width of the red blood cell column, and estimating the damage of the glycocalyx based on the linear theory of the change in the diameter of the red blood cell column before and after the passage of white blood cells, there is a scientific theoretical basis. It can also be verified by comparing the PBR value obtained manually with the system output value, further ensuring the accuracy of the evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts do not necessarily draw according to the actual scale. Obviously, the following described drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0023] FIG. 1 is a flowchart of the side-stream dark field imaging video data processing method in Embodiment 1.

[0024] FIG. 2 is a schematic structural diagram of a deep learning model suitable for microvascular key point tracking provided in Embodiment 1.

[0025] FIG. 3 is a schematic diagram of the change in the width of the red blood cell column.

[0026] FIG. 4 is a frame image when a white blood cell passes through.

[0027] FIG. 5 is a box plot of the PBR results obtained by the method analysis in Embodiment 1 and the PBR measured manually by ImageJ.

[0028] Figure 6 It is a structural diagram for Embodiment 2; Figure 7 It is a schematic diagram of marking key points of different frames with different colors in the video. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0030] In this article, suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of explaining the present invention, and have no specific meaning in themselves. Therefore, "module", "component", or "unit" can be used interchangeably.

[0031] In this article, the orientation or positional relationship indicated by terms such as "upper", "lower", "inner", "outer", "front", "rear", "one end", "the other end", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation on the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0032] In this article, unless otherwise clearly specified and defined, terms such as "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, and can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] In this article, "and / or" includes any and all combinations of one or more of the listed related items.

[0034] In this article, "a plurality of" means two or more, that is, it includes two, three, four, five, etc.

[0035] Example 1: In systemic inflammatory response syndrome and sepsis, the deterioration of the endothelial glycocalyx plays a key role in vascular barrier dysfunction and ultimate organ failure. Therefore, early detection of glycocalyx damage has become an important goal in intensive care. Researchers often need invasive operations to sample and detect the products of endothelial glycocalyx shedding in plasma or serum to evaluate the damage of the endothelial glycocalyx. For intravital microscopy imaging, researchers or medical staff need to manually screen the images before and after the passage of blood vessels and white blood cells for measurement, so as to analyze the damage of the glycocalyx. The operation steps are cumbersome and the analysis time is long. With the development of imaging technology, it has become possible to non-invasively and real-time evaluate the degree of glycocalyx damage in microvessels through side-stream dark-field imaging (SDF) video analysis, thus providing a reference for medical staff to take corresponding measures for further diagnosis.

[0036] To solve the above problems, as Figure 1 shown, this embodiment provides a method for processing side-stream dark-field imaging video data, including: S1 Collect side-stream dark-field videos containing target microvessels.

[0037] Specifically, in this embodiment, a high-resolution side-stream dark-field imaging system, such as MicroSee V100, is used to collect side-stream dark-field videos containing the target sublingual microvasculature. The resolution and frame rate of this video need to ensure that they meet the requirements of subsequent analysis, providing raw data for subsequent analysis, and its quality directly affects the evaluation accuracy. By collecting the video, dynamic information such as cell flow in the microvasculature and blood vessel morphology can be obtained, and these information are the basis for evaluating the glycocalyx damage situation.

[0038] S2 Input the side-stream dark-field video into the trained key-point tracking model to mark the key points in the side-stream dark-field video; the key points include the boundary points of the microvasculature, the starting point and the ending point of the red blood cell column.

[0039] In order to reduce the error of tracking the target segment of the microvasculature, the selection of key points needs to meet the following conditions at the same time: one is to be the boundary point of the microvasculature, and the other is to be the starting point or the ending point of the red blood cell column. Since the number of these three types of points is relatively small, and the boundary features of the starting point and the ending point are more prominent, which is convenient for marking and the operation is faster. Compared with the middle segment of the microvasculature, the marking errors of these points are also easier to be identified (for example, the mark is located inside the red blood cell column, that is, the black area in the video, or outside the blood vessel, that is, the white area), thereby reducing the difficulty of marking and the probability of marking errors.

[0040] However, when marking through the key-point tracking model, it is inevitable that there will be marking errors. Therefore, the method of this embodiment further includes the following steps to correct the mis-marked key points: First, the average processing method adopted can smooth the influence of individual error points.

[0041] Secondly, by analyzing each marked key point, the marking result can be further optimized: One, export the confidence (likelihood) of the key points marked by the key-point tracking model (this confidence is automatically generated during the marking process by the key-point tracking model), and regard the key points with relatively low confidence, for example, the confidence is lower than the first preset confidence threshold, as suspicious key points, and eliminate the suspicious key points with confidence lower than the second preset confidence threshold; among them, the second preset confidence is less than the first preset confidence threshold; Two, view the key-frame images after marking to judge whether each marked suspicious key point coincides with the actual position. If it coincides, execute step three. If it does not coincide, it is determined that the suspicious key point is mis-marked and re-marked manually; Third, compare the key points marked on adjacent frames (for example, multiple key-frame images adjacent in time, before and after). If any suspicious key point shows a significant jump or deviation in consecutive frames (i.e., the current key-frame image and its adjacent frames), determine that the suspicious key point is mislabeled and re-perform manual labeling. Since the key points will be displayed as points of different colors in the video after key point tracking, that is, the key points will be labeled as points of different colors in different frames, as Figure 7 shown. If the positions of several consecutive frames differ too much, causing any key point in at least one frame to jump or deviate, then any key point in that frame can be determined to be mislabeled. For example, perform data fitting on the position data of the same type of key points in consecutive frames. If the key points of this type on any frame deviate from the fitting curve (or the distance from the fitting curve is greater than the preset threshold), it indicates a jump or deviation.

[0042] If the number of frames with mislabeling is small, the error can be compensated by removing low-confidence data and averaging; if there are many error frames, the mislabeled key points and their key-frame images can be discovered in time through the above-mentioned Step 2 and Step 3, and then the mislabeled key-frame images can be relabeled.

[0043] Of course, if there are large-scale mislabelings, it means that the training set needs to be rebuilt.

[0044] Input the collected video into a pre-trained key point tracking model. This model will automatically identify and label key points such as the boundary points of microvessels, the starting points and ending points of red blood cell columns in the video. By using a deep neural network model to replace traditional manual labeling, the labeling efficiency and accuracy are improved. Key points are the key basis for subsequent calculation of the diameter change of red blood cell columns, and the accuracy of model labeling determines the reliability of subsequent data calculation.

[0045] More specifically, the key point tracking model in this embodiment can be one of CNN models such as DeepLabCut, ResNet, EfficientNet, MobileNet deep neural network models.

[0046] In this embodiment, the key point tracking model is preferably the DeepLabCut deep neural network model because the DeepLabCut deep neural network model performs excellently in key point tracking tasks, especially suitable for biomedical image and video analysis. It can accurately identify and track specific key points and has high robustness in dealing with complex dynamic scenes. Specifically, as Figure 2As shown, the architecture of the DeepLabCut deep neural network model successively includes an input layer, a residual network, and an output layer; the input layer and the residual network layer are connected by a convolutional layer and a max pooling layer; it also includes a transposed convolutional layer connected to the output layer, and the transposed convolutional layer is used for coordinate regression to track key points and obtain the position information of the key points of blood vessels in each frame, so that the key points are displayed in the side-stream dark-field video. The residual network includes multiple residual blocks, and each residual block includes a convolutional calculation unit, a batch normalization unit, a rectified linear unit, and a max pooling unit.

[0047] Among them, the input layer is the entrance for the model to receive data. In this embodiment, the input is the side-stream dark-field video frame image containing the target microvessels. The input layer will perform preliminary processing on the input image data, such as normalization operations, to ensure that the data is within a suitable numerical range for subsequent layer processing.

[0048] The convolutional layer and the max pooling layer play the role of connecting the input layer and the residual network. The convolutional layer slides a convolutional kernel over the input image to extract local features of the image, such as edges and textures. The max pooling layer then downsamples the output of the convolutional layer to reduce the amount of data, lower the computational complexity, and at the same time retain important feature information.

[0049] The residual network consists of multiple residual blocks, and each residual block contains a convolutional calculation unit, a batch normalization unit, a rectified linear unit, and a max pooling unit.

[0050] The convolutional calculation unit is used to further extract the deep features of the image, and by using convolutional kernels of different sizes and numbers, it captures more complex patterns in the image.

[0051] The batch normalization unit is used to normalize the output of the convolutional calculation unit, accelerate the training process of the model, and improve the stability and generalization ability of the model.

[0052] The rectified linear unit is used to introduce non-linear factors and enhance the expressive ability of the model, so that the model can learn more complex function mapping relationships.

[0053] The max pooling unit is used to downsample the feature map, reduce the data dimension, and reduce the computational amount.

[0054] The output layer receives the output of the residual network and passes it to the transposed convolutional layer.

[0055] The transposed convolutional layer, also known as the deconvolution layer, its main function is to perform coordinate regression. It maps the feature map of the output layer back to the size of the original image, thereby determining the specific position of the key points in each frame of the image. Through the transposed convolutional layer, the model can track the movement trajectory of the key points in the side-stream dark-field video and display these key points in the video, providing a basis for subsequent analysis.

[0056] In addition, when training the key-point tracking model in this embodiment, several frames of images are randomly selected from the collected microvascular side-stream dark-field videos for manual key-point marking, thereby forming a training set.

[0057] Specifically, several frames of images are randomly selected from the collected videos. The purpose of random selection is to ensure that the selected images can cover various different situations and features in the video, avoid biases caused by artificial selection, make the training set more representative and random, enable the model to learn a wider range of feature patterns, enhance its generalization ability, and adapt to key-point tracking tasks in different scenarios.

[0058] Use the napari marking tool to mark the key points on the selected frame images. Napari is a tool for image processing and visualization, which can facilitate users to accurately mark the required key points on the images.

[0059] The marked key points cover the boundary points of microvessels, the starting and ending points of red blood cell columns, etc. These key points are the key basis for subsequent calculation of the change in the diameter of red blood cell columns and evaluation of the glycocalyx damage situation. By accurately marking the key points, the model can learn how to identify these important feature positions from the images.

[0060] S3 Calculate the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through according to the key-point information, thereby obtaining the red blood cell column perfusion boundary zone value.

[0061] The principle of evaluating glycocalyx damage in the present invention is as Figure 3 shown. Define the red blood cell column diameter before the white blood cell passes through as RCWB, the red blood cell column diameter after the white blood cell passes through as RCWA, and the glycocalyx damage index as GI. According to the change in the red blood cell column diameter before and after the white blood cell passes through Estimate the GI and display the measurement results. The estimation is based on linear theory. The glycocalyx of endothelial cells restricts the approach of red blood cells to capillary endothelial cells. In contrast, white blood cells are stiffer and less deformable than red blood cells. Therefore, when white blood cells pass through the capillary lumen, they compress the glycocalyx of the capillary endothelium, causing the column of red blood cells immediately following the white blood cell to briefly widen after the white blood cell has passed. When the human glycocalyx is healthy, the glycocalyx is arranged densely and is not easily compressed, and the widening of the red blood cell column after the passage of white blood cells is not obvious; when the human glycocalyx is damaged, the glycocalyx is easily compressed, allowing more red blood cells to penetrate deeper into the endothelial surface, and the widening phenomenon is obvious. Therefore, dividing the change in the diameter width of the red blood cell column before and after the passage of white blood cells by 2 is defined as the perfused boundary region (PBR) value, which is related to the degree of damage to the microvascular glycocalyx transiently compressed by the passing white blood cells. The more severe the glycocalyx damage, the larger the GI value.

[0062] The purpose of this step is to calculate the PBR value from the key point data.

[0063] Specifically, the collected side-stream dark-field video containing the target microvessels is loaded into a previously trained key point tracking model (such as the DeepLabCut model) for analysis. Check the "Save result(s) as csv" option, and the model will output the information on the change in the positions of the required key points (such as microvascular boundary points, starting and ending points of the red blood cell column, etc.) in the video over time and save it as a data file.

[0064] Use MATLAB to read the data file for further processing and analysis of the data later. MATLAB has powerful data processing and computing capabilities and is suitable for processing complex medical image data. More specifically, dividing the change in the diameter of the red blood cell column before and after the passage of white blood cells by 2 gives the perfusion boundary region PBR, that is 。

[0065] The negative sign is to ensure that the value of PBR is positively correlated with the degree of glycocalyx damage.

[0066] During the process of evaluating the damage of the human microvascular glycocalyx, in order to ensure the accuracy and reliability of the evaluation results, factors that may interfere with the measurement need to be processed.

[0067] Vasospasm, microthrombus formation or disappearance can all lead to abnormal widening of the diameter of the red blood cell column. In the actually collected side-stream dark-field videos, such abnormalities can be identified by observing the morphology, motion state of the red blood cell column and the overall condition of the blood vessel. For example, during vasospasm, the blood vessel will suddenly contract and narrow, and then the diameter of the red blood cell column may change abnormally; microthrombus formation will cause blood flow obstruction, and the flow and morphology of the red blood cell column will also show abnormalities; when the microthrombus disappears, the sudden change in the blood flow state will also be reflected in the diameter of the red blood cell column.

[0068] Cropping these abnormal videos before inputting the videos into the key-point tracking model can avoid abnormal data from entering the model training and analysis process. After cropping out the abnormal video segments, the data processed by the model can better represent the normal physiological state or the real situation related to glycocalyx damage, thus ensuring the reliability of subsequent calculations of the change in the width of the red blood cell column, PBR value, and assessment of glycocalyx damage degree.

[0069] When cropping the video, the user should at least select one blood vessel with obvious contrast as the target area (i.e., the region of interest), ensure that the cropping range covers this target area, and avoid cropping out the key blood vessels; preferably select video segments with stable shooting to ensure that the blood vessel is always within the field of view and reduce the difficulty of marking caused by the loss of the field of view.

[0070] Although a large amount of cropping can increase the number of training samples, it may lead to overfitting of the model and reduce the generalization ability; while insufficient cropping results in a limited number of training samples, which may lead to relatively large analysis errors. Therefore, this problem can be partially alleviated by adjusting training parameters (such as regularization or data augmentation).

[0071] In addition, the amount of cropping has a very large impact on computing resources. When the amount of cropping is small, the video memory required for the training and analysis process is very large, which may pose higher requirements on the hardware performance. Therefore, it is also recommended to crop moderately on the premise of ensuring sufficient training samples.

[0072] In addition, the problem of abnormal disturbance can also be solved by the method of data screening, that is, setting a boundary zone threshold, which is a reference standard set based on a large amount of perfusion boundary zone (PBR) data under normal physiological conditions. Under normal circumstances, the PBR value is within a certain reasonable range, which reflects the normal change degree of the microvascular glycocalyx before and after the passage of white blood cells. This threshold plays a role in screening "normal" and "abnormal" data.

[0073] When the PBR value is greater than the boundary region threshold, it means that the measured perfusion boundary region value exceeds the normal range. This may be due to abnormal conditions such as vasospasm, microthrombus formation or disappearance, which cause the abnormal widening of the red blood cell column diameter, and then lead to deviation of the PBR value. Therefore, by excluding the PBR values greater than the boundary region threshold, the interference of these abnormal factors can be excluded, and the accuracy of the evaluation can be improved.

[0074] S4 constructs a glycocalyx damage index based on the perfusion boundary region value of the red blood cell column to evaluate the glycocalyx damage of the target microvessels.

[0075] Based on the PBR value, a glycocalyx damage index (GI) is constructed. Since GI is proportional to PBR (GI ∝ PBR), the larger the PBR value, the higher the glycocalyx damage index, indicating more severe glycocalyx damage. Through this glycocalyx damage index, doctors or researchers can intuitively understand the glycocalyx damage of the target microvessels, providing a strong basis for disease diagnosis and treatment.

[0076] In some embodiments, to verify the accuracy of the above method, by comparing the PBR values obtained manually and the PBR values output by the system (keypoint tracking model combined with data analysis process), it is judged whether the evaluation method and the keypoint tracking model used are accurate when calculating the perfusion boundary region value of the red blood cell column.

[0077] Specifically, select the frame image when white blood cells pass through from the side flow dark field video, as Figure 4 shown. The arrow in the figure indicates the flow direction, and the "*" mark indicates white blood cells. Import the frame images before and after the frame image into ImageJ and use the Straight tool to draw a line; determine the red blood cell column diameter RCWB before the white blood cell passes through and the red blood cell column diameter RCWA after the white blood cell passes through according to the gray scale change on the line segment, and use the formula: Calculate the perfusion boundary region value of the red blood cell column.

[0078] Enter the PBR obtained by system analysis and the PBR results measured manually by ImageJ into Table 1, draw a box plot, and evaluate the consistency of the two methods, as Figure 5 shown.

[0079] Table 1 PBR results and errors obtained by DeepLabCut and ImageJ It can be seen that the PBR acquisition method provided by the present invention has high reliability.

[0080] Example two: As Figure 6As shown, the present invention also provides a side-flow dark-field imaging video data processing system for evaluating the damage of the glycocalyx in human microvessels. Specifically, it includes: A video acquisition module for acquiring a side-flow dark-field video containing the target microvessel; A key point tracking module for inputting the side-flow dark-field video into a trained key point tracking model to mark the key points in the side-flow dark-field video; the key points include the boundary points of the microvessel, the starting point and the ending point of the red blood cell column; A perfusion boundary zone value calculation module for calculating the change value of the red blood cell column diameter in the capillary before and after the white blood cell passes through the capillary according to the key point information, so as to obtain the red blood cell column perfusion boundary zone value; A damage evaluation module for constructing a glycocalyx damage index according to the red blood cell column perfusion boundary zone value to evaluate the damage of the glycocalyx in the target microvessel. Specifically, the damage evaluation module uses the formula: To calculate the glycocalyx damage index; where GI is the glycocalyx damage index, PBR is the perfusion boundary zone value, and GI is positively correlated with PBR; RCWB is the red blood cell column diameter before the white blood cell passes through, RCWA is the red blood cell column diameter after the white blood cell passes through, RCW is the change value of the red blood cell column.

[0081] In some embodiments, the system further includes a cropping module for eliminating the perfusion boundary zone value greater than the boundary zone threshold, or cropping the video with an abnormally widened red blood cell column diameter caused by vasospasm, microthrombus formation or disappearance before inputting it into the key point tracking model, so as to eliminate the abnormally widened red blood cell column diameter caused by vasospasm, microthrombus formation or disappearance.

[0082] In some embodiments, the perfusion boundary zone value calculation module is specifically used to call MATLAB and import the key point information into MATLAB to calculate the change value of the red blood cell column diameter in the capillary before and after the white blood cell passes through the capillary.

[0083] In some embodiments, the key point tracking module is specifically used to select the frame image when the white blood cell passes through from the side-flow dark-field video, import the frame images before and after the frame image into ImageJ, and call the Straight tool to draw a line; then determine the red blood cell column diameter before the white blood cell passes through and the red blood cell column diameter after the white blood cell passes through according to the gray-scale change on the line segment, and calculate the red blood cell column perfusion boundary zone value to verify the accuracy of the key point tracking model. [[ID=2,3]]

[0084] It should be noted that in this text, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes such element.

[0085] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described method of the embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a computer terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0086] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the claims of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A method for processing side-stream dark-field imaging video data, characterized in that Including: Collecting a side - stream dark - field video containing the target microvessels; Inputting the side - stream dark - field video into a trained key - point tracking model to mark the key points in the side - stream dark - field video; the key points include the boundary points of the microvessels, the starting point and the ending point of the red blood cell column; Calculating the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through the capillary according to the key - point information, so as to obtain the red blood cell column perfusion boundary zone value; Constructing a glycocalyx damage index according to the red blood cell column perfusion boundary zone value.

2. The method for processing side - stream dark - field imaging video data according to claim 1, the step of constructing a glycocalyx damage index according to the red blood cell column perfusion boundary zone includes: Using the formula: Calculate the glycocalyx injury index; among them, GI is the glycocalyx injury index, PBR is the perfusion boundary region value, and GI is positively correlated with PBR; RCWB is the diameter of the red blood cell column before the passage of white blood cells, RCWA is the diameter of the red blood cell column after the passage of white blood cells, RCW is the change value of the red blood cell column; There is a positive proportional relationship between the glycocalyx damage index and the glycocalyx damage condition of the microvessels.

3. A method for processing side-stream dark-field imaging video data according to claim 1, characterized in that: The key - point tracking model is one of the DeepLabCut, ResNet, EfficientNet, MobileNet deep neural network models.

4. A method for processing side-stream dark-field imaging video data according to claim 3, characterized in that: The DeepLabCut deep neural network model successively includes an input layer, a residual network and an output layer; the input layer and the residual network layer are connected by a convolutional layer and a max - pooling layer; it also includes a de - convolutional layer connected to the output layer, and the de - convolutional layer is used for coordinate regression to track the key points and obtain the position information of the key points of the blood vessels in each frame, so that the key points are displayed in the side - stream dark - field video.

5. A method for processing side-stream dark-field imaging video data according to claim 4, characterized in that: The residual network includes a plurality of residual blocks, and each residual block includes a convolutional calculation unit, a batch normalization unit, a rectified linear unit, and a max - pooling unit.

6. A method for processing side-stream dark-field imaging video data according to claim 1, characterized in that: Eliminating the perfusion boundary zone values greater than the boundary zone threshold, or cropping the video with abnormal widening of the red blood cell column diameter caused by vasospasm, micro - thrombus formation or disappearance before inputting it into the key - point tracking model, so as to remove abnormal data.

7. A method for processing side-stream dark-field imaging video data according to claim 1, characterized in that: Importing the key - point information into MATLAB, and using MATLAB to calculate the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through the capillary.

8. A method for processing side-stream dark-field imaging video data according to claim 1, characterized in that: Selecting the frame images when the white blood cell passes through from the side - stream dark - field video, importing the frame images before and after the frame images into ImageJ, and using the Straight tool to draw a line; determining the diameter of the red blood cell column before the white blood cell passes through and the diameter of the red blood cell column after the white blood cell passes through according to the gray - scale change on the line segment, and calculating the red blood cell column perfusion boundary zone value to verify the accuracy of the key - point tracking model.

9. A method for processing side-stream dark-field imaging video data according to claim 1, characterized in that: When training the key - point tracking model, randomly selecting several frame images from the collected side - stream dark - field video containing microvessels for manual key - point marking to form a training set.

10. A lateral flow dark field imaging video data processing system, characterized in that Including: A video acquisition module for collecting a side - stream dark - field video containing the target microvessels; A key - point tracking module for inputting the side - stream dark - field video into a trained key - point tracking model to mark the key points in the side - stream dark - field video; the key points include the boundary points of the microvessels, the starting point and the ending point of the red blood cell column; A perfusion boundary zone value calculation module for calculating the change value of the diameter of the red blood cell column in the capillary before and after the white blood cell passes through the capillary according to the key - point information, so as to obtain the red blood cell column perfusion boundary zone value; An injury assessment module for constructing a glycocalyx injury index based on the erythrocyte column perfusion boundary region value.

Images