A Microvascular Segmentation Method Based on Video Shake Reduction

Abstract

The present invention discloses a microvascular segmentation method based on video de - shaking, which relates to the technical field of image processing and includes: collecting a microvascular video and dividing it into n frame images, pre - processing each frame image to generate processed image data; selecting an image of the microvascular structure according to the division sequence of the images, and marking the region of interest (ROI) containing the microvascular structure as a template image; scaling the template image and the current frame image to low resolution and outputting a low - resolution matching result; scaling the template image and the current frame image back to high resolution, and obtaining a high - resolution matching result and performing de - shaking processing according to the matching logic of the low - resolution matching result to obtain a stable image; using the stable image as input data and outputting the region of interest containing the microvessels as the segmentation region through a deep learning model. The present invention performs automatic image segmentation through a deep learning model and efficiently segments the region of interest containing the microvessels.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a microvascular segmentation method and system based on video anti-shake. Background Art

[0002] In medical diagnosis and research, the observation and analysis of microvessels are crucial for the diagnosis and treatment of various diseases. With the development of optical imaging technology, a needle-type microscope-based in-vivo cell optical scanner enables doctors to observe the microvascular blood flow phenomenon under in-vivo conditions. However, due to factors such as improper operation, environmental interference, and physical limitations of imaging devices, the microvascular videos collected often have jitter problems. These jitters not only affect the stability of the images but also interfere with subsequent blood vessel segmentation and blood flow analysis, thereby affecting the accuracy of diagnosis.

[0003] Currently, due to factors such as the use of handheld devices and patient body movement, video images are prone to jitter, which affects the stability of the images and reduces the reliability of subsequent analysis results. Traditional anti-shake methods often rely on simple filtering techniques and are difficult to effectively eliminate complex jitter problems in videos. In addition, the microvascular structure is often fine and easily masked by background noise, and existing image processing methods have limitations in enhancing contrast and highlighting microvascular details. Usually, contrast enhancement techniques cannot balance the global contrast and local details of the image, and annotation-based image segmentation methods are time-consuming and laborious and are easily affected by human factors, making it difficult to achieve high-efficiency automation in large-scale data processing.

[0004] At the same time, there is a problem that the segmentation algorithm has low detection accuracy for microvascular regions under complex backgrounds and different lighting conditions. Due to the interference of lighting changes and background noise, the automatic segmentation accuracy of microvessels is often not ideal, which affects the accuracy of microvascular lesion detection. These problems directly limit the application of microvascular imaging technology in medical diagnosis and research. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a microvascular segmentation method based on video anti-shake.

[0006] A microvascular segmentation method based on video anti-shake, the method comprising:

[0007] Collect a microvascular video and divide it into n frame images according to the time sequence, and preprocess each frame image to generate processed image data;

[0008] Select an image of the microvascular structure from the processed image data according to the image division sequence, and mark the region of interest (ROI) containing the microvascular structure, and use it as a template image;

[0009] Scale the template image and the current frame image to a low resolution, and perform preliminary matching using the Normalized Squared Difference Matching (NSSD) algorithm to output a low-resolution matching result;

[0010] Regress the template image and the current frame image to a high resolution, and obtain a high-resolution matching result and perform anti-shake processing according to the matching logic of the low-resolution matching result to obtain a stable image;

[0011] Use the anti-shaken stable image as input data, and output the region of interest containing microvessels through a preset deep learning model as the segmentation region.

[0012] Further, the logic for preprocessing each frame of image is as follows:

[0013] S111: Collect a microvascular fluorescence video through a microscope device. Assume the video duration is a seconds, there are c frames per second, and a total of n frames of images are obtained, n = a × c, where a and c are greater than 0;

[0014] S112: Perform histogram equalization processing on each frame of image and repeat it to obtain processed image data.

[0015] Further, the logic for performing histogram equalization processing on each frame of image and repeating it is as follows:

[0016] Q1: Convert each frame of image into a grayscale image. The logical formula for the converted grayscale value is: Y = k1 × R + k2 × G + k3 × B, where R, G, and B respectively represent the pixel values of the red channel, green channel, and blue channel in the image, Y is the converted grayscale value, k1, k2, and k3 are respectively the weight factors of the pixel values of the red channel, green channel, and blue channel, k2 > k1 > k3, 0 < Y < 255, and the grayscale value Y corresponds to the gray level (0 to 255);

[0017] Q2: Perform statistics on the grayscale image, calculate the histogram of the gray level (0 to 255), and obtain the number distribution of pixel points for each gray level. The gray histogram is a 256-dimensional vector H, where H[i] represents the number of pixels with a gray value of i;

[0018] Q3: Calculate the cumulative distribution function CDF according to the gray histogram and reassign the pixel values. The calculation logic of the cumulative distribution function CDF is: . Among them, represents the cumulative number of pixels from gray level 0 to i. Normalize the cumulative distribution function CDF to obtain the proportional value CFN of the cumulative distribution. The logical formula for the proportional value CFN of the cumulative distribution is: , among them, is the smallest non-zero pixel cumulative value, and u and m are respectively the number of rows and columns of the grayscale image;

[0019] Q4: Update each pixel value in the image using CFN to obtain the new gray value of the pixel. The calculation formula for the new gray value Y'(x, v) of the pixel point (x, v) is: Y'(x, v) = CFN Y' × Y(x, v), where Y(x, v) is the gray value of the original image pixel point (x, v), and Y'(x, v) is the new gray value of the pixel point (x, v) after histogram equalization processing;

[0020] Q5: Use the updated gray image as the processed image data;

[0021] Q6: Repeat the above steps Q1 - Q5 for each frame image in the video sequence to obtain all the processed image data in the microvascular video.

[0022] Further, the logic for selecting an image of the microvascular structure from the processed image data according to the image division sequence is:

[0023] Define a representative image, and the representative image is an image of the area with the target microvascular structure in the divided images.

[0024] Further, the logic for marking the region of interest (ROI) containing the microvascular structure and using it as the template image is:

[0025] Define the region of interest (ROI) as the area in the image containing the target microvascular structure. Open the representative image frame, start the annotation tool, and according to the contour of the target microvascular structure, mark several consecutive marker points on its contour path, and connect the adjacent marker points to form a closed marker area to form the region of interest.

[0026] Further, the logic for using the normalized sum of squared differences (NSSD) algorithm for preliminary matching to determine the jitter region is:

[0027] Calculate the NSSD value between the template image and each possible matching position in the current frame image. The calculation formula for the NSSD value is: NSSD(d, f) = , where NSSD(d, f) is the normalized sum of squared differences matching value of the template image and the current frame image at the offset (d, f), is the sum over all template image pixel positions (d, f), is the gray value at the pixel position in the template image, is the gray value at the pixel position in the current frame image, that is, translated (d, f) units relative to the pixel position of the template image, is the standard deviation of the grayscale values of the template image, and the logical formula is , where n is the number of divided images, is the average grayscale value of the template image;

[0028] Select the position with the smallest NSSD value as the best initial matching position, and output this position as the low-resolution matching result of the template image in the current frame.

[0029] Furthermore, the logic for upsampling the template image and the current frame image to high resolution and obtaining the high-resolution matching result according to the matching logic of the low-resolution matching result is as follows:

[0030] Similar to the matching logic of the low-resolution matching result, return to the original high-resolution image, define the region of interest (ROI), which surrounds the center point of the low-resolution matching result, and the region size is larger than the low-resolution template;

[0031] Find the position with the smallest NSSD value in high resolution, and output this position as the high-resolution matching result of the template image in the current frame.

[0032] Furthermore, the logic for anti-shake processing is as follows:

[0033] Record the displacement of the best matching position found in the high-resolution matching relative to the original position to obtain the displacement vector (βx, βy);

[0034] Use the calculated displacement vector (βx, βy) to adjust the position of the current frame through image translation operations to align it with the template image, that is, translate all pixel points in the current frame in the horizontal direction of βx and the vertical direction of βy, and save the corrected image frame as the stable image after anti-shake processing.

[0035] Furthermore, the construction logic of the deep learning model is as follows: The

[0036] Obtain historical image segmentation data, and divide the historical image segmentation data into a historical image segmentation training set and a historical image segmentation test set; the historical image segmentation training set contains stable images and the regions of interest of their corresponding microvessels;

[0037] Construct a regression network, use the de-shaken stable images in the historical image segmentation training set as the input data of the regression network, and use the regions of interest of the microvessels in the historical image segmentation training set as the output data of the regression network, and train the regression network to obtain an initial deep learning network;

[0038] Use the historical image segmentation test set to perform model verification on the initial deep learning network, and output the initial deep learning network with a test error threshold less than or equal to the preset value as the pre-constructed deep learning model.

[0039] A microvascular segmentation system based on video anti-shake, implemented based on the method of any one of the above-mentioned microvascular segmentation systems based on video anti-shake, includes:

[0040] Image preprocessing module: used to collect microvascular videos and divide them into n frames of images according to the time sequence, preprocess each frame of image, and generate processed image data;

[0041] Image annotation module: used to select an image of the microvascular structure from the processed image data according to the division sequence of the images, and label the region of interest (ROI) containing the microvascular structure, and use it as a template image;

[0042] First analysis module: used to scale the template image and the current frame image to low resolution, and perform preliminary matching using the normalized sum of squared differences (NSSD) algorithm to output a low-resolution matching result;

[0043] Second analysis module: used to scale the template image and the current frame image back to high resolution, and obtain a high-resolution matching result and perform anti-shake processing according to the matching logic of the low-resolution matching result to obtain a stable image;

[0044] Image segmentation module: used to use the de-shaken stable image as input data, and output the region of interest containing microvessels as the segmentation region through a preset deep learning model.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] The present invention collects microvascular fluorescence videos through a microscope device, and combines low-resolution preliminary matching and high-resolution precise matching, which can effectively eliminate the shaking phenomenon in the video, thereby significantly improving the stability of the image. Histogram equalization processing further enhances the contrast of the image, making the microvascular structure clearer and more obvious in the image. Through these anti-shake and contrast enhancement technologies, the present invention provides a stable and clear image data basis, providing reliable support for subsequent microvascular analysis and template matching. These improvements ensure that the details of microvessels can be effectively captured, improving the visibility and resolution of the image, and significantly enhancing the observation effect of the microvascular region. In addition, the present invention performs automated image segmentation through a deep learning model, efficiently and accurately identifying and outputting the region of interest (ROI) containing microvessels. This method can adapt to complex backgrounds and different lighting conditions, avoiding the subjectivity and inconsistency brought by annotation, improving the segmentation accuracy and robustness, and reducing the need for manual intervention through an automated processing process, improving the processing efficiency. Description of the Drawings

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other accompanying drawings can also be obtained based on these drawings.

[0048] Figure 1 Flowchart of a microvascular segmentation method based on video de - shaking provided in Embodiment 1 of the present invention;

[0049] Figure 2 De - shaking comparison diagram of a microvascular segmentation method based on video de - shaking provided in Embodiment 1 of the present invention;

[0050] Figure 3 Module diagram of a microvascular segmentation system based on video de - shaking provided in Embodiment 2 of the present invention. Detailed implementation manners

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0052] Embodiment 1

[0053] Please refer to Figure 1 As shown, this embodiment discloses a microvascular segmentation method based on video de - shaking. The method includes:

[0054] S110: Collect microvascular videos and divide them into n frames of images according to the time sequence, and pre - process each frame of image to generate processed image data;

[0055] Specifically, the logic for pre - processing each frame of image is:

[0056] S111: Collect microvascular fluorescence videos through a microscope device. Assume the video duration is a seconds, there are c frames per second, and a total of n frames of images are obtained, where n = a × c, and a and c are greater than 0;

[0057] It should be noted that: the microscope device includes but is not limited to a confocal microscope, a multi - photon microscope, a super - resolution microscope, and a light - sheet microscope;

[0058] S112: Perform histogram equalization processing on each frame of image and repeat it to obtain processed image data;

[0059] Specifically, the logic of performing histogram equalization processing on each frame of image and repeating the process is as follows:

[0060] Q1: Convert each frame of image into a grayscale image. The logical formula for the converted grayscale value is: Y = k1×R + k2×G + k3×B, where R, G, and B represent the pixel values of the red channel, green channel, and blue channel in the image respectively, Y is the converted grayscale value, k1, k2, and k3 are the weight factors of the pixel values of the red channel, green channel, and blue channel respectively, k2 > k1 > k3, 0 < Y < 255, and the grayscale value Y corresponds to the gray level (0 to 255);

[0061] It should be noted that: The human eye is less sensitive to red, so the weight factor of the red channel is relatively small; The human eye is most sensitive to green, so the weight coefficient of the green channel is the largest; The human eye is least sensitive to blue, so the weight factor of the blue channel is the lowest;

[0062] Q2: Statistically analyze the grayscale image, calculate the histogram of the gray level (0 to 255), and obtain the quantity distribution of pixel points at each gray level. The gray histogram is a 256-dimensional vector H, where H[i] represents the number of pixels with a gray value of i;

[0063] Q3: Calculate the cumulative distribution function CDF based on the gray histogram and reallocate the pixel values. The calculation logic of the cumulative distribution function CDF is: . Among them, represents the cumulative number of pixels from gray level 0 to i. Normalize the cumulative distribution function CDF to obtain the proportional value CFN of the cumulative distribution. The logical formula for the proportional value CFN of the cumulative distribution is: , where, is the minimum non-zero pixel cumulative value, and u and m are the number of rows and columns of the grayscale image respectively;

[0064] Q4: Update each pixel value in the image using CFN to obtain the new grayscale value of the pixel point. The calculation formula for the new grayscale value Y′(x, v) of the pixel point (x, v) is: Y′(x, v) = CFNY′×Y(x, v), where Y(x, v) is the grayscale value of the original image pixel point (x, v), and Y′(x, v) is the new grayscale value of the pixel point (x, v) after histogram equalization processing;

[0065] Q5: Use the updated grayscale image as the processed image data;

[0066] It should be noted that: This process enhances the image contrast, making the microvascular structure more obvious in the image, facilitating subsequent template matching and de-jitter processing;

[0067] Q6: Repeat the above steps Q1 - Q5 for each frame image in the video sequence to obtain all processed image data in the microvascular video.

[0068] S120: Select an image of the microvascular structure from the processed image data according to the image division sequence, and mark the region of interest (ROI) containing the microvascular structure, which is used as the template image.

[0069] Specifically, the logic for selecting an image of the microvascular structure from the processed image data according to the image division sequence is as follows:

[0070] Define a representative image, which is an image of the region with the target microvascular structure in the divided images.

[0071] It should be noted that: Video playback tools or image processing software include but are not limited to ImageJ, MATLAB, and OpenCV. This frame of image is located at the center of the video, but other positions can be selected according to needs.

[0072] Specifically, the logic for marking the region of interest (ROI) containing the microvascular structure and using it as the template image is as follows:

[0073] Define the region of interest (ROI) as the region in the image containing the target microvascular structure. Open the representative image frame, start the marking tool, and according to the contour of the target microvascular structure, mark several consecutive marker points on its contour path, and connect adjacent marker points to form a closed marker region to form the region of interest.

[0074] It should be noted that: Target microvascular detail images include but are not limited to vascular trauma information, vascular branches, and vascular diameters. Marking tools include but are not limited to MATLAB, Adobe Photoshop, and Fiji. The selection of the template image through manual marking ensures the accuracy of the template. Especially in the case of complex image structures or cluttered backgrounds, manually selecting the template can improve the reliability of the overall de - jitter effect.

[0075] S130: Scale the template image and the current frame image to low resolution, and use the Normalized Squared - Difference Matching (NSSD) algorithm for preliminary matching to output the low - resolution matching result.

[0076] Specifically, the logic for using the Normalized Squared - Difference Matching (NSSD) algorithm for preliminary matching to determine the jitter region is as follows:

[0077] Calculate the NSSD value at each possible matching position between the template image and the current frame image. The calculation formula for the NSSD value is: NSSD(d,f)= , where NSSD(d,f) is the template image and the current frame image The normalized sum of squared differences matching value at the offset (d,f), is the sum over all template image pixel positions (d,f), is the pixel position in the template image at which the gray value is located, is the pixel position in the current frame image at which the gray value is located, that is, translated (d,f) units relative to the pixel position of the template image, is the standard deviation of the gray values of the template image, and the logical formula is , where n is the number of divided images, is the average gray value of the template image;

[0078] It should be noted that: NSSD(d,f) is the normalized sum of squared differences matching value between the template image and the current frame image at the offset (d,f) is used to measure the similarity between the two images,

[0079] Select the position with the smallest NSSD value as the best preliminary matching position, and output this position as the low-resolution matching result of the template image in the current frame.

[0080] S140: Resize the template image and the current frame image to high resolution. According to the matching logic of the low-resolution matching result, obtain the high-resolution matching result and perform anti-shake processing to obtain a stable image;

[0081] Specifically, the logic of resizing the template image and the current frame image to high resolution and obtaining the high-resolution matching result according to the matching logic of the low-resolution matching result is as follows:

[0082] Similar to the matching logic of the low-resolution matching result, return to the original high-resolution image, define the region of interest (ROI), which surrounds the center point of the low-resolution matching result, and the region size is larger than the low-resolution template;

[0083] It should be noted that: The defined region size of the region of interest (ROI) of the high-resolution original image is larger than the low-resolution template, which can be used to contain possible small displacement errors;

[0084] Find the position with the smallest NSSD value in high resolution, and output this position as the high-resolution matching result of the template image in the current frame;

[0085] Specifically, the logic of anti-shake processing is as follows:

[0086] Record the displacement of the best matching position found in the high-resolution matching relative to the original position to obtain the displacement vector (βx, βy);

[0087] It should be noted that: the displacement vector represents the offset of the template image in the current frame;

[0088] Use the calculated displacement vector (βx, βy) to adjust the position of the current frame through image translation operations to align it with the template image, that is, translate all pixel points in the current frame in the horizontal direction of βx and the vertical direction of βy, and save the corrected image frame as the stable image after de-jitter processing;

[0089] It should be noted that: through the above high-resolution precise matching steps, more accurate position information can be obtained from the results of the low-resolution preliminary matching, further eliminating the jitter phenomenon in the video. This process combines the details of the high-resolution image and the speed of the low-resolution matching, providing an efficient and accurate image stabilization method, laying a foundation for subsequent microvascular segmentation and analysis;

[0090] S150: Use the de-jittered stable image as input data, and output the region of interest containing microvessels as the segmentation region through a preset deep learning model;

[0091] Specifically, the construction logic of the deep learning model is as follows:

[0092] Obtain historical image segmentation data, and divide the historical image segmentation data into a historical image segmentation training set and a historical image segmentation test set; the historical image segmentation training set contains stable images and their corresponding regions of interest of microvessels;

[0093] Construct a regression network, use the de-jittered stable images in the historical image segmentation training set as the input data of the regression network, and use the regions of interest of microvessels in the historical image segmentation training set as the output data of the regression network, and train the regression network to obtain an initial deep learning network;

[0094] Use the historical image segmentation test set to verify the model of the initial deep learning network, and output the initial deep learning network with a test error less than or equal to the preset test error threshold as the pre-constructed deep learning model;

[0095] It should be noted that: the regression network is one of algorithm models such as support vector machine, regression decision tree regression, linear regression, or neural network;

[0096] An example of this embodiment is given:

[0097] A microvascular video was collected using a high-resolution pinhole microscope. The video was 10 seconds long, with 30 frames per second, for a total of 300 frames. The resolution of the images was 640×480 pixels. When processing these images according to the steps described in the embodiment, each frame of the image was preferably converted into a grayscale image. Assuming that the RGB values of a certain frame of the collected image were R = 120, G = 150, B = 100, the grayscale value obtained after grayscale conversion was 135;

[0098] Furthermore, histogram equalization processing was performed on each frame of the image to calculate the distribution of each grayscale value. Assuming that the total number of pixels in the image was 307,200, it was obtained that the number of pixels with a grayscale value of 0 in the grayscale histogram was 500, the number of pixels with a grayscale value of 1 was 300, and so on. Through the calculation of the cumulative distribution function, the cumulative distribution ratio value of the grayscale value of 135 was 0.9, and the updated grayscale value was 122;

[0099] Furthermore, the processed 300 frames of images were browsed through the MATLAB tool, and the 150th frame was selected as the representative image. This frame of the image had a clear microvascular structure, and the region of interest (ROI) was marked, with a size of 50×50 pixels;

[0100] Furthermore, the representative template image and the current frame image were scaled to a resolution of 128×96, and the normalized sum of squared differences (NSSD) algorithm was used for preliminary matching. According to the description in the embodiment, the best preliminary matching position obtained was the offset (d = 2, f = 3); in the high-resolution matching stage, the resolution was reverted to 640×480, the high-resolution ROI was set to 80×80 pixels, and finally the high-resolution matching position was obtained. The displacement vector was (βx = 3, βy = 4), and the current frame image was translated using this displacement vector, and the de-jittered and stabilized image obtained after correction was used as the output;

[0101] Finally, the de-jittered stabilized image was input into a pre-constructed deep learning model. This model was trained on historical image segmentation data (the training set contained 4,000 images, and the test set contained 1,000 images), and finally the region of interest containing the microvascular structure was segmented;

[0102] As Figure 2 shown, multiple groups of region-of-interest images containing the microvascular structure were extracted and then output as the final stabilized video, Figure 2 The five images on the left are the images without de-jitter processing, Figure 2 The five images on the right are the images with de-jitter processing of the microvessels.

[0103] Example 2

[0104] Refer to Figure 3It can be seen that based on a unified inventive concept, this embodiment discloses a microvascular segmentation system based on video anti-shake, including:

[0105] An image preprocessing module S210: used to collect microvascular videos and divide them into n frames of images according to the time sequence, and preprocess each frame of image to generate processed image data;

[0106] Specifically, the logic for preprocessing each frame of image is as follows:

[0107] S111: Collect a microvascular fluorescence video through a microscope device. Assume the video duration is a seconds, there are c frames per second, and a total of n frames of images are obtained, where n = a × c, and a and c are greater than 0;

[0108] S112: Perform histogram equalization processing on each frame of image and repeat it to obtain processed image data;

[0109] Specifically, the logic for performing histogram equalization processing on each frame of image and repeating it is as follows:

[0110] Q1: Convert each frame of image into a grayscale image. The logical formula for the converted grayscale value is: Y = k1 × R + k2 × G + k3 × B, where R, G, and B respectively represent the pixel values of the red channel, green channel, and blue channel in the image, Y is the converted grayscale value, k1, k2, and k3 are respectively the weighting factors of the pixel values of the red channel, green channel, and blue channel, k2 > k1 > k3, 0 < Y < 255, and the grayscale value Y corresponds to the gray level (0 to 255);

[0111] Q2: Perform statistics on the grayscale image, calculate the histogram of the gray level (0 to 255), and obtain the number distribution of pixel points at each gray level. The gray histogram is a 256-dimensional vector H, where H[i] represents the number of pixels with a gray value of i;

[0112] Q3: Calculate the cumulative distribution function CDF according to the gray histogram and reallocate the pixel values. The calculation logic of the cumulative distribution function CDF is as follows: . Among them, represents the cumulative number of pixels from gray level 0 to i. Normalize the cumulative distribution function CDF to obtain the proportional value CFN of the cumulative distribution. The logical formula for the proportional value CFN of the cumulative distribution is: , where, is the smallest non-zero pixel cumulative value, and u and m are respectively the number of rows and columns of the grayscale image;

[0113] Q4: Use CFN to update each pixel value in the image to obtain the new grayscale value of the pixel. The calculation formula for the new grayscale value Y′(x, v) of the pixel point (x, v) is: Y′(x, v) = CFNY′ × Y(x, v), where Y(x, v) is the grayscale value of the original image pixel point (x, v), and Y′(x, v) is the new grayscale value of the pixel point (x, v) after histogram equalization processing;

[0114] Q5: Use the updated grayscale image as the processed image data;

[0115] Q6: Repeat the above steps Q1 - Q5 for each frame image in the video sequence to obtain all the processed image data in the microvascular video.

[0116] Image annotation module S220: It is used to select an image of the microvascular structure from the processed image data according to the image division sequence, and annotate the region of interest (ROI) containing the microvascular structure, and use it as the template image;

[0117] Specifically, the logic for selecting an image of the microvascular structure from the processed image data according to the image division sequence is:

[0118] Define a representative image, and the representative image is an image of the region with the target microvascular structure in the divided images;

[0119] Specifically, the logic for annotating the region of interest (ROI) containing the microvascular structure and using it as the template image is:

[0120] Define the region of interest (ROI) as the region in the image containing the target microvascular structure. Open the representative image frame, start the annotation tool, and according to the contour of the target microvascular structure, mark several consecutive marker points on its contour path, and connect the adjacent marker points to form a closed marker region to form the region of interest;

[0121] First analysis module S230: It is used to scale the template image and the current frame image to low resolution, and perform preliminary matching using the normalized sum of squared differences (NSSD) algorithm to output the low-resolution matching result;

[0122] Specifically, the logic for using the normalized sum of squared differences (NSSD) algorithm to perform preliminary matching and determine the jitter region is:

[0123] Calculate the NSSD value of each possible matching position between the template image and the current frame image. The calculation formula for the NSSD value is: NSSD(d, f) = , where NSSD(d, f) is the template image and the current frame image The normalized squared difference matching value at the offset (d,f), is the sum over all template image pixel positions (d,f), where is the gray value at the pixel position in the template image, and is the gray value at the pixel position in the current frame image, that is, translated by (d,f) units relative to the pixel position of the template image. is the standard deviation of the gray values of the template image, and the logical formula is where n is the number of divided images,

[0124] Select the position with the smallest NSSD value as the best preliminary matching position, and output this position as the low-resolution matching result of the template image in the current frame.

[0125] The second analysis module S240: used to upscale the template image and the current frame image, and according to the matching logic of the low-resolution matching result, obtain the high-resolution matching result and perform anti-shake processing to obtain a stable image;

[0126] Specifically, the logic of upscaling the template image and the current frame image and obtaining the high-resolution matching result according to the matching logic of the low-resolution matching result is as follows:

[0127] Similar to the matching logic of the low-resolution matching result, return to the original high-resolution image, define the region of interest (ROI), which surrounds the center point of the low-resolution matching result, and the region size is larger than the low-resolution template;

[0128] Find the position with the smallest NSSD value in high resolution, and output this position as the high-resolution matching result of the template image in the current frame.

[0129] Specifically, the logic of anti-shake processing is as follows:

[0130] Record the displacement of the best matching position found in the high-resolution matching relative to the original position to obtain the displacement vector (βx,βy);

[0131] Use the calculated displacement vector (βx,βy) to adjust the position of the current frame through image translation operations to align it with the template image, that is, translate all pixel points in the current frame in the horizontal direction of βx and the vertical direction of βy, and save the corrected image frame as the stable image after anti-shake processing.

[0132] The image segmentation module S250: used to take the anti-shaken stable image as input data and output the region of interest containing microvessels as the segmentation region through a preset deep learning model;

[0133] Specifically, the construction logic of the deep learning model is as follows:

[0134] Obtain historical image segmentation data, and divide the historical image segmentation data into a historical image segmentation training set and a historical image segmentation test set; the historical image segmentation training set includes stable images and the corresponding regions of interest of microvessels;

[0135] Construct a regression network, use the de-shaken stable images in the historical image segmentation training set as the input data of the regression network, and use the regions of interest of microvessels in the historical image segmentation training set as the output data of the regression network, and train the regression network to obtain an initial deep learning network;

[0136] Use the historical image segmentation test set to verify the model of the initial deep learning network, and output the initial deep learning network with a test error threshold less than or equal to the preset value as the pre-constructed deep learning model.

[0137] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more collections of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0138] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present invention can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0139] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0140] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a method for microvascular segmentation based on video anti-shake. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, in each embodiment of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0143] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention and should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0144] Finally: The above is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A microvascular segmentation method based on video anti-shake, characterized in that, The method includes: Collecting a microvascular video and dividing it into n frame images according to a time series, and preprocessing each frame image to generate processed image data; The logic for preprocessing each frame image is: Collecting a microvascular fluorescence video through a microscope device. Suppose the video duration is a seconds, with c frames per second, and a total of n frames are obtained, where n = a × c, and a and c are greater than 0; performing histogram equalization processing on each frame image and repeating it to obtain processed image data; Selecting an image of the microvascular structure from the processed image data according to the image division sequence, and marking the region of interest ROI containing the microvascular structure as a template image; Scaling the template image and the current frame image to a low resolution, and using the normalized sum of squared differences matching NSSD algorithm for preliminary matching to output a low resolution matching result; Restoring the template image and the current frame image to a high resolution, and obtaining a high resolution matching result and performing anti-shake processing according to the matching logic of the low resolution matching result to obtain a stable image; The logic for obtaining the high resolution matching result is: Similar to the matching logic of the low resolution matching result, restoring to the original high resolution image, defining a region of interest ROI, which surrounds the center point of the low resolution matching result, and the region size is larger than the low resolution template; Finding the position with the minimum NSSD value in the high resolution, and outputting this position as the high resolution matching result of the template image in the current frame; Taking the anti-shaken stable image as input data, and outputting the region of interest containing microvessels through a preset deep learning model as the segmentation region.

2. The method for microvascular segmentation based on video anti-shake according to claim 1, wherein The logic for performing histogram equalization processing on each frame image and repeating it is: Q1: Converting each frame image into a grayscale image, and the logical formula for the converted grayscale value is: Y = k1×R + k2×G + k3×B, where R, G, and B respectively represent the pixel values of the red channel, green channel, and blue channel in the image, Y is the converted grayscale value, k1, k2, and k3 are respectively the weight factors of the pixel values of the red channel, green channel, and blue channel, k2 > k1 > k3, 0 < Y < 255, and the grayscale value Y corresponds to gray levels 0 to 255; Q2: Performing statistics on the grayscale image, calculating the histogram of gray levels 0 to 255, and obtaining the quantity distribution of pixel points for each gray level. The gray histogram is a 256-dimensional vector H, where H[i] represents the number of pixels with a gray value of i; Q3: Calculate the cumulative distribution function CDF according to the grayscale histogram, and reassign the pixel values. The calculation logic of the cumulative distribution function CDF is as follows: ; where represents the cumulative number of pixels from grayscale level 0 to i. Normalize the cumulative distribution function CDF to obtain the proportional value CFN of the cumulative distribution. The logical formula for the proportional value CFN of the cumulative distribution is: , where is the smallest non-zero cumulative pixel value, and u and m are the number of rows and columns of the grayscale image respectively; Q4: Using CFN to update each pixel value in the image to obtain the new grayscale value of the pixel point. The calculation formula for the new grayscale value Y′(x,v) of the pixel point (x,v) is: Y′(x,v) = CFNY′×Y(x,v), where Y(x,v) is the grayscale value of the original image pixel point (x,v), and Y′(x,v) is the new grayscale value of the pixel point (x,v) after histogram equalization processing; Q5: Taking the updated grayscale image as the processed image data; Q6: Repeating the above steps Q1-Q5 for each frame image in the video sequence to obtain all the processed image data in the microvascular video.

3. The method for microvascular segmentation based on video anti-shake according to claim 2, characterized in that, The logic for selecting an image of the microvascular structure from the processed image data according to the image division sequence is as follows: Define a representative image, which is an image of the region with the target microvascular structure in the divided images.

4. The method for microvascular segmentation based on video anti-shake according to claim 3, wherein The logic for labeling the region of interest ROI containing the microvascular structure and using it as the template image is as follows: Define the region of interest ROI as the region in the image that contains the target microvascular structure. Open the representative image frame, start the labeling tool, and according to the contour of the target microvascular structure, mark several consecutive marker points on its contour path. Connect the adjacent marker points to form a closed marker region to form the region of interest.

5. The method for microvascular segmentation based on video anti-shake according to claim 4, wherein The logic for using the normalized sum of squared differences NSSD algorithm for preliminary matching to determine the jitter region is as follows: Calculate the NSSD value between the template image and each possible matching position in the current frame image. The calculation formula for the NSSD value is: NSSD(d,f)= , where NSSD(d,f) is the normalized squared difference matching value between the template image and the current frame image at the offset (d,f), is the sum over all template image pixel positions (d,f), is the gray value at the pixel position in the template image, is the gray value at the pixel position in the current frame image, that is, translated by (d,f) units relative to the pixel position of the template image, is the standard deviation of the gray values of the template image, and the logical formula is , where n is the number of divided images, is the average gray value of the template image; Select the position with the minimum NSSD value as the best preliminary matching position, and output this position as the low-resolution matching result of the template image in the current frame.

6. The method for microvascular segmentation based on video anti-shake according to claim 5, characterized in that, The logic for de-jitter processing is as follows: Record the displacement of the best matching position found in the high-resolution matching relative to the original position to obtain the displacement vector (βx, βy); Use the calculated displacement vector (βx, βy) to adjust the position of the current frame through image translation operations to align it with the template image, that is, translate all pixel points in the current frame in the βx horizontal direction and the βy vertical direction, and save the corrected image frame as the stable image after de-jitter processing.

7. The method for microvascular segmentation based on video anti-shake according to claim 6, wherein The construction logic of the deep learning model is: Obtain historical image segmentation data, and divide the historical image segmentation data into a historical image segmentation training set and a historical image segmentation test set; the historical image segmentation training set contains stable images and their corresponding regions of interest of the microvasculature; Construct a regression network. Use the de-jittered stable images in the historical image segmentation training set as the input data of the regression network, and use the regions of interest of the microvasculature in the historical image segmentation training set as the output data of the regression network. Train the regression network to obtain an initial deep learning network; Use the historical image segmentation test set to verify the model of the initial deep learning network, and output the initial deep learning network with an error less than or equal to the preset test error threshold as the pre-constructed deep learning model.

8. A microvascular segmentation system based on video anti-shake, which is used to implement the microvascular segmentation method based on video anti-shake described in any one of claims 1-7, and is characterized in that, It includes: Image preprocessing module: used to collect the microvascular video and divide it into n frames of images according to the time sequence, and preprocess each frame of image to generate processed image data; Image annotation module: used to select an image of the microvascular structure from the processed image data according to the image division sequence, and label the region of interest ROI containing the microvascular structure and use it as the template image; First analysis module: used to scale the template image and the current frame image to low resolution, and use the normalized sum of squared differences NSSD algorithm for preliminary matching to output the low-resolution matching result; Second analysis module: used to scale the template image and the current frame image back to high resolution, obtain the high-resolution matching result according to the matching logic of the low-resolution matching result, and perform de-jitter processing to obtain the stable image; Image segmentation module: It is used to take the de-shaken stable image as input data and output the region of interest containing microvessels as the segmentation region through a preset deep learning model.

Images