Medical image mirror reflection repair method and system

Abstract

The application discloses a medical image mirror reflection repairing method, comprising the following steps: S1, decomposing a collected medical video sequence into single-frame images; S2, screening a mirror reflection area in the single-frame images according to brightness and saturation double thresholds to generate a mask matrix; S3, repairing pixels in the mirror reflection area by using a frequency domain selection reconstruction algorithm according to the mask matrix; and S4, merging the repaired images according to processing orders to obtain a video sequence after mirror reflection repairing. The application can effectively repair the pixels in the mirror reflection area in the medical images, so that the images are more natural and have better visual effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a medical image mirror reflection repair method and system. BACKGROUND

[0002] With the rapid development of medical imaging technology and computer technology, high-quality medical imaging technology plays an increasingly key role in assisting clinical disease diagnosis and treatment. The emergence of these medical imaging technologies provides favorable conditions for doctors to improve diagnosis efficiency and reduce surgical risk. However, due to the influence of image acquisition equipment and acquisition environment, various degradation and degradation problems inevitably occur in medical images, such as smooth surface and high reflectivity of human organ tissues, metal surgical instruments, etc. After imaging, mirror reflection regions are prone to occur in these areas. A large range of mirror reflection regions will cover the image features and textures in the original region, and at the same time, due to the saturation of the brightness in the region, it will distract the attention of the doctor and affect the doctor's observation and diagnosis. Therefore, repairing the mirror reflection region is helpful to improve the visual level and optimize the visual experience of the doctor in the diagnosis process, which has practical significance and application value.

[0003] At present, the mirror reflection repair method generally includes two stages of detection and repair: first, the mask matrix of the mirror reflection region is obtained through detection, and then the reflection region is repaired by using a repair algorithm. The current main idea is mainly divided into two categories: (1) the algorithm of mirror reflection separation based on two-color reflection model; (2) the algorithm based on image repair. The former idea is to separate the mirror reflection component and extract the diffuse reflection component, which is suitable for repairing the reflection region with non-saturated and existing image information. The latter regards the mirror reflection region as a missing region, and the reflection repair is equivalent to the reconstruction of the missing pixels, that is, the missing pixels are reconstructed by using the known pixels outside the target region. Considering that the reflection coefficient in the actual medical image is usually high, and the pixels are generally saturated or close to saturation. Therefore, it is urgent to provide a robust image repair algorithm that can effectively remove the mirror reflection region in the medical image. SUMMARY

[0004] The main purpose of the present application is to provide a medical image mirror reflection repair method which can effectively eliminate the mirror reflection region in the medical image.

[0005] The present application provides a medical image mirror reflection repair method, which comprises the following steps:

[0006] S1: decomposing the collected medical video sequence into single-frame images;

[0007] S2: screening out the mirror reflection region in the single-frame image according to the brightness and saturation double threshold, and generating a mask matrix;

[0008] S3: repairing the specular reflection region pixel according to the mask matrix and using the frequency domain selection reconstruction algorithm;

[0009] S4: merging the repaired image according to the processing order to obtain a specular reflection repaired video sequence.

[0010] The above technical solution, wherein step S2 specifically comprises the following steps:

[0011] S21: obtaining a luminance component L and a saturation component S of a single frame image;

[0012] S22: traversing each pixel, and detecting a specular reflection pixel according to a luminance and saturation double threshold value;

[0013] S23: generating a mask matrix of the specular reflection region according to a detection result;

[0014] S24: processing the mask matrix to generate a final mask.

[0015] The above technical solution, wherein the luminance component L is a V component in HSV space, an I component in HSI space, a Y component in YUV space, or an L component in CIELAB space; and the saturation component S is an S component in HSV or HSI space or a C component in CIELCH space.

[0016] The above technical solution, wherein in step S22, the luminance threshold value comprises a first luminance threshold value L1, a second luminance threshold value L2, and a third luminance threshold value L3, and the saturation threshold value is S1; let Ω be a set of screened specular reflection pixels, then for any pixel p i ∈Ω, one of the following criteria needs to be met:

[0017] ①

[0018] ②

[0019] Wherein, N(p i ) r represents a set of all pixel values of a neighborhood with the pixel p i as the center, and r is the radius.

[0020] The above technical solution, wherein the mask matrix is an element 0-1 matrix, 1 corresponds to a normal pixel, and 0 corresponds to a specular reflection pixel.

[0021] The above technical solution, wherein step S3 specifically comprises the following steps:

[0022] S31: uniformly dividing the target image and the mask matrix into fixed-size subblocks, and marking as to-be-processed image blocks;

[0023] S32: constructing a reconstruction region for each image block to be processed, and classifying pixels in the reconstruction region;

[0024] S33: calculating the priority of all image blocks to be processed, determining the repair order and the number of iterations;

[0025] S34: calculating the projection coefficients of the estimated residual error of the reconstruction region in the frequency domain;

[0026] S35: selecting a Fourier basis function and an estimated expansion coefficient, and calculating the projection coefficients;

[0027] S36: updating the estimate and the reconstruction residual of the reconstruction region according to the projection coefficients;

[0028] S37: stopping at the number of iterations of steps S34-S36, and outputting the repair result of the image block to be processed;

[0029] S38: all image blocks to be processed are repaired, and the complete repaired image is output.

[0030] According to the above technical solution, in step S31, the size of the sub-block is MxN, and all sub-blocks in the mask matrix whose elements are not all 1 are marked as image blocks to be repaired.

[0031] According to the above technical solution, in step S32, the size of the reconstruction region of the image block to be processed b is 2Mx2N, and the pixels in the reconstruction region are classified into three categories: available, repaired, and to be repaired, which are stored in different sets A, R, and B, respectively.

[0032] According to the above technical solution, in step S33, the method for determining the repair order of the image block to be processed is as follows:

[0033]

[0034]

[0035] wherein, N p is the number of pixels in the image block to be processed b that belong to set B, -1≤a2

[0036] According to the above technical solution, step S2 is specifically: dividing the reflection types of pixels in a single frame image according to the brightness and saturation threshold values, including saturated specular reflection, non-saturated specular reflection, and diffuse reflection, and then extracting the target reflection region according to the reflection types to obtain the saturated specular reflection region mask matrix and the non-saturated specular reflection region mask matrix;

[0037] The step S3 is specifically: repairing the pixels in the saturated specular reflection region according to the saturated region mask matrix; and repairing the pixels in the non-saturated specular reflection region according to the non-saturated specular reflection region mask matrix.

[0038] The present application has the advantages that: the present application screens the specular reflection region in a single frame image through the brightness and saturation double thresholds, can effectively remove the specular reflection region often appearing in a medical image, eliminates the image abruptness caused by the reflected pixels, optimizes the visual experience of doctors, and the technical scheme of the present application can reconstruct the specular reflection region by using a frequency domain image repair algorithm, optimizes and improves the repair quality and efficiency on the basis of the original algorithm, and improves the robustness and practicality.

[0039] In addition, the present application also provides the specular reflection pixels (saturated, non-saturated) that can be comprehensively covered, different repair methods are used for different types of reflected pixels, therefore, the robustness is stronger, and better repair effect can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0040] The present application will be further described below in combination with the drawings and embodiments, and the drawings are as follows:

[0041] Figure 1 is a flow chart of the endoscope image specular reflection repair method of the embodiment of the present application;

[0042] Figure 2 is Figure 1 a specific flow chart of the embodiment;

[0043] Figure 3 is a flow chart of the endoscope image specular reflection repair method of the embodiment of the present application;

[0044] Figure 4 is Figure 3 a specific flow chart of the embodiment;

[0045] Figure 5 is a comparison chart of the medical image before and after repair of the embodiment of the present application;

[0046] Figure 6 is a structural schematic diagram of the medical image specular reflection removal system of the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0048] The endoscope image specular reflection repair method of the embodiment of the present application has the specific flow as shown in Figure 1 The present application includes the following steps:

[0049] S1: Acquire the endoscopic video sequence through the image acquisition system and decompose it into single-frame images;

[0050] S2: Use a detection algorithm to obtain the mask matrix of the mirror reflection area of ​​the endoscope image;

[0051] S3: Use frequency domain selection reconstruction algorithm to repair pixels in the specular reflection area;

[0052] S4: After sequentially merging the processed images, the endoscope video sequence after specular reflection restoration is obtained;

[0053] Furthermore, the step S2 of detecting specular reflection pixels to obtain the mask matrix includes the following steps:

[0054] S21: Endoscopic images are converted to the HSV color space, and the luminance component V and saturation component S are extracted.

[0055] V = max(R, G, B)

[0056]

[0057] S22: Set the first brightness threshold L1 = 0.73, the second brightness threshold L2 = 0.85, the third brightness threshold L2 = 0.34, and the first saturation threshold S1 = 0.24. Iterate through components V and S, detect specular reflection pixels according to the following criteria, and store them in set Ω:

[0058] ①

[0059] ②

[0060] in, Represents pixel p i brightness value, Represents pixel p i Saturation value; N(p i )5 represents pixel p i is the set of neighboring pixel values ​​with a center radius of 5; Mean is the function for calculating the mean.

[0061] S23: Based on the detection results, the position of the specular reflection pixel is 1 and the other positions are 0. Obtain the specular reflection area mask matrix M of the endoscope image.

[0062] S24: Morphological dilation of the mask matrix M is performed using a circular structure with a 5×5 structuring element as follows:

[0063]

[0064] Further, the step S3 frequency selection reconstruction algorithm repairs the mirror reflection area of the endoscope image as follows:

[0065] S31: divide the endoscope image and the mask matrix M into image blocks of 16x16, and mark the sub-blocks in M that are not all 1 as to-be-repaired image blocks;

[0066] S32: traverse the image blocks b, construct a reconstruction area of 32x32, divide the L pixels into three categories of available, repaired and to-be-repaired, and store the pixel coordinates in sets A, R and B in sequence; the reconstruction area includes the image blocks and the available pixel area surrounding the image blocks to provide information required for the repair process. In the embodiment of the present application, the reconstruction area is twice the image blocks, and the size of the reconstruction area can be determined according to specific conditions.

[0067] S33: set α1=-0.2 and α2=-0.5 (to adjust the contribution of the pixels in sets A and R to the calculation of the processing priority of the image blocks, respectively), calculate the center coordinates (X, Y) of the pixels in set B (i.e. to-be-repaired pixels) in the b area and the priority level T of b, and the formulas are as follows:

[0068]

[0069]

[0070] wherein f[m, n] represents the reconstruction area, and m and n are integers. Sort the T values of the blocks to generate the processing order of the image blocks.

[0071] Set the maximum number of iterations I max = 100, the minimum number of iterations I max = 10, and η = 40, calculate the gradient value Grad of the image blocks, and determine the number of iterations of the sub-blocks:

[0072] I = (I max -I min ) x Grad

[0073]

[0074] Gradient template

[0075] S34: calculate the estimated residual error Set σ1=10 and σ2=20 to generate the spatial domain weight for each pixel in the image blocks:

[0076]

[0077] Calculate the projection coefficient of the weighted residual error of the reconstruction area estimate in the frequency domain:

[0078]

[0079] S35: Set the orthogonality defect compensation factor g = 0.5, and generate the frequency domain weight with a second-order Butterworth low-pass filter:

[0080]

[0081] d(k, l) = [(k - M / 2) 2 +(l - N / 2) 2 ] 1 / 2

[0082] Optimize the Fourier basis function selection target model as follows, and estimate the corresponding expansion coefficient:

[0083]

[0084] where k, l are frequency coordinates, w f [k, l] represents the frequency weight under the k, l coordinates; the basis function is The estimation of the expansion coefficient is

[0085] S36: Update the estimate of the reconstruction region and the estimated residual:

[0086]

[0087] S37: Stop when the iteration steps S34-S36 reach the iteration number, and output the repair result of the image block;

[0088] S38: All image blocks are repaired, and the repaired complete endoscope image is output.

[0089] The effectiveness of the technical solution proposed in the present application is verified by several groups of endoscope pictures. The pictures in the left column Figure 5 The left column of pictures simultaneously exist obvious specular reflection regions, especially, there are continuous large-area specular reflection regions in left 2 and left 3. The reflection regions significantly reduce the overall visual experience of the endoscope image, which may affect the doctor's diagnosis in actual endoscopic examination. Figure 3 The right column is the endoscope picture repaired by applying the technical solution. From the result, the specular reflection region is basically removed, and the region after pixel replacement is integrated with the adjacent region, and the boundary transition is relatively natural. Although the repair is a process of generating pixels that do not exist in the original image, there is a difference between the replaced pixels and the real pixels, but the image visual effect has been significantly improved, so the technical solution has practical significance and application value.

[0090] In the above embodiment, the existing frequency domain selection reconstruction algorithm (Frequency Selective Reconstruction) (see another embodiment of the present application below for details) is mainly improved in three aspects:

[0091] 1. The processing priority of each to-be-repaired sub-image block is determined.

[0092] In the prior art, the priority is calculated by summing up the Euclidean distances from the coordinates of all pixels in sets A and R to the coordinate of the geometric center of the image block, and the center coordinate of the to-be-repaired pixel group is the center coordinate of the image block by default.

[0093] The improved algorithm of the above embodiment is as follows: (1) the center coordinate of the to-be-repaired pixel group is the average of the coordinates of all to-be-repaired pixels, which is more reasonable; (2) parameters a1 and a2 are introduced to balance the contribution (importance) of the pixels in sets A and R to the calculation of the T value, which is not considered in the prior art, but a simple Euclidean distance accumulation sum is directly performed.

[0094] 2. Determination of the number of iterations of the algorithm

[0095] The running efficiency of the frequency domain selection reconstruction algorithm is very critical. This algorithm is an iterative algorithm, and a certain number of iterations is required to achieve good repair effect. Because the algorithm repairs in units of image blocks, the number of iterations required for an observable and weak-texture image block (image is flat, and the pixel edge changes little) is small, and more iterations will not bring improvement in repair effect but will increase time consumption; and the number of iterations required for a strong-texture image block (image surface is undulating, and the pixel value changes sharply) is large, and fewer iterations will affect the repair effect.

[0096] The prior art: the same number of iterations is set for all image blocks. In order to take into account the strong / weak-texture image blocks, the number of iterations for the weak-texture image block will inevitably overflow, increasing the time cost.

[0097] The improved algorithm of the above embodiment: the gradient value is used as a standard for measuring the strength of the texture of the image block, thereby establishing a one-to-one correspondence between the gradient value and the number of iterations. By setting the minimum and maximum number of iterations, the number of iterations is set according to the calculated gradient value of each image block, which can ensure the repair effect while reducing the time consumption.

[0098] 3. Spatial domain and frequency domain weighting functions

[0099] First, the spatial domain weighting function: the spatial domain weighting function takes the Euclidean distance from the pixels in the reconstruction region to the center of the to-be-repaired pixel as the function variable, and the spatial domain weight value reflects the importance of different pixels to the repair.

[0100] The prior art: an exponential function is used as the weighting function. Because the pixels in R are less important to the repair than the pixels in A, the original algorithm simply adds a decay factor to the weight calculation function of the pixels in A.

[0101] The above embodiment algorithm: two different scale Gaussian functions are used to assign spatial domain weights to pixels in A and R, and the pixel assignment is more flexible than the original algorithm, and the weight assignment in A and R is more targeted.

[0102] Secondly, the frequency domain weighting function: the role of the frequency domain weighting function is to repair more low frequency information, because there is a high probability of high frequency information loss or pseudo high frequency in the image block to be repaired, and giving more weight to high frequency information will lead to repair error.

[0103] The existing algorithm: the otf is used to calculate the frequency domain weighting function.

[0104] The improved algorithm of the above embodiment: the Butterworth low-pass filter is used as the frequency domain weighting function. The overall shape of the two functions is similar, but the function curve of the improved algorithm is smoother, and the weight changes more gently and naturally with the frequency.

[0105] Another embodiment of the present application, as shown in the figure, the embodiment of the medical image mirror reflection repair method, comprising the following steps: Figure 3

[0106] S301: acquiring a video sequence through an image acquisition system, and decomposing it into single frame images;

[0107] S302: dividing the reflection type of pixels in the single frame image according to the brightness and saturation threshold, including saturated mirror reflection, non-saturated mirror reflection and diffuse reflection, then extracting the target reflection area according to the reflection type, and obtaining the saturated mirror reflection area mask and the non-saturated mirror reflection area mask;

[0108] S303: repairing the pixel value of the target reflection area by using the algorithm combining image repair and mirror separation, repairing the pixels in the saturated mirror reflection area according to the saturated area mask, and repairing the pixels in the non-saturated mirror reflection area according to the non-saturated mirror reflection area mask;

[0109] S304: merging the repaired images in sequence to obtain the video sequence after mirror reflection repair;

[0110] Further, the step S302 of dividing the reflection type of pixels by using the detection algorithm comprises the following steps:

[0111] S3021: obtaining the brightness and saturation components L and S of the image;

[0112] S3022: traversing the image pixels, and dividing the reflection type of pixels according to the brightness and saturation threshold;

[0113] S3023: obtaining the mask of the mirror reflection area to be repaired;

[0114] ​S3024: Perform morphological processing on the mask image to obtain the final mask image of the specular reflection region;

[0115] In step S3021, the luminance component L can be the V component of HSV space, the I component of HSI space, the Y component of YUV space, or the L channel of CIELAB space; the saturation component can come from the S channel of HSV space and the C channel of CIELCH space.

[0116] In step S3022, the thresholds include luminance thresholds: Lthr1, Lthr2∈(0,1] and saturation thresholds: Sthr1, Sthr2∈(0,1]; the reflection types include saturated specular reflection Ω1, unsaturated specular reflection Ω2 and diffuse reflection Ω3.

[0117] like Figure 4 As shown, in a specific embodiment of the present invention, Lthr1 = 0.73, Lthr2 = 0.85, Sthr1 = 0.24, and Sthr2 = 0.35. The specific classification process is as follows:

[0118] Let sets Ω1, Ω2, Ω3 store pixels of three types of reflection. For any pixel p in the L component... i N(p) i ) is its neighborhood, which is classified as follows:

[0119] ① Then p i ∈Ω1;

[0120] ② And satisfy Then p i ∈Ω1; otherwise, p i ∈Ω2;

[0121] ③ satisfy Then p i ∈Ω2; otherwise, then p i ∈Ω3;

[0122] ④ Then p i ∈Ω3.

[0123] In step S3023, the mask is a 0-1 matrix, where the 0 element corresponds to a specular reflection pixel. M1 is the saturated specular reflection region mask; M2 is the unsaturated specular reflection region mask.

[0124] In step S3024, M1 and M2 undergo morphological expansion to reduce unnatural boundaries in the repair area. The specific structural element is a 5×5 circular structure.

[0125]

[0126] Further, the repairing of the Ω1, Ω2 pixels in step S303 specifically includes the following steps.

[0127] S3031: Repairing the Ω1 pixels using a frequency selection reconstruction algorithm;

[0128] S3032: Repairing the Ω2 pixels using a specular reflection separation algorithm combined with pixel clustering;

[0129] S3033: Replacing the original image pixels at the corresponding positions with the repaired pixels based on the masks M1 and M2 after the inflation processing, and keeping the pixels in other non-reflection areas unchanged.

[0130] The frequency selection reconstruction algorithm in step S3031 can use the frequency selection reconstruction algorithms in the above embodiments S31-S37, or use the following frequency selection reconstruction algorithm, which specifically includes the following steps.

[0131] S30311: Let I c , c ∈ {r, g, b} represent a channel of the to-be-repaired image in the RGB space, and divide I c into subblocks (such as 4 × 4) of the size W × H for a completely consistent subblock division processing on M1;

[0132] S30312: Determining the processing order of the reflection pixels in each subblock according to the local density characteristics of the reflection pixels. Performing a Gaussian filtering on M1, and determining the processing order of the subblocks according to the accumulated values of the pixel values of the subblocks after the filtering. The accumulated values of the pixel values of the subblocks after the filtering are calculated, and the accumulated value is greater, the priority is higher. The filtering radius is selected as max (W, H), such as a Gaussian filtering with a radius of 8;

[0133] S30313: For a subblock b i , the extrapolation region L i of b i includes the pixels of b i and the neighboring pixels, and has a size of X × Y (X > W, Y > H), such as 8 × 8. The L i region includes three types of pixels, i.e., available, repaired, and to-be-repaired, which are sequentially stored in sets A, R, and B. The pixels in A and R dominate the repair of the pixels in B;

[0134] S30314: Calculating the residual projection coefficient. Let f [x, y] ∈ L i , and consider the following sparse model:

[0135]

[0136] wherein, is an estimation result of L i , is a Fourier basis function, These are the expansion coefficients, and the set K stores a tuple of indices representing all basis functions in the sparse model described above. Let... Let f[x,y] represent the residual between f[x,y] and the i-th estimate. During the iteration process, the frequency domain reconstruction algorithm estimates the projection coefficients of the residual onto the Fourier basis by minimizing the weighted residual energy function:

[0137]

[0138]

[0139] in, The weighted residuals are projected onto the Fourier basis, where w[x,y] is the weighting function in the spatial domain.

[0140]

[0141] in, This is the weight decay factor, which can be selected. The value of w[x,y] reflects L i The contribution and importance of each pixel in the repair process are considered. Elements in set B are pixels to be repaired and do not participate in the repair process, with a weight of 0. The closer a pixel is to the center of the sub-block, the higher its correlation with the sub-block to be repaired, and the greater its weight. Pixels in set A are more reliable than pixels in set R, and therefore have a greater weight.

[0142] S30315: Maximize the basis functions and expansion coefficients of the following model:

[0143]

[0144] It is the estimated basis function index The basis functions are The estimation of the expansion coefficients is g is the orthogonality defect compensation factor, w f [k,l] is the frequency domain weighting function:

[0145]

[0146] in,

[0147] S30316: Update L i Estimate and residuals:

[0148]

[0149] S30317: In the iterative steps S314-S317, if the number of iterations reaches a preset value (e.g., 100 times), the iteration process stops, and the repaired sub-block b is obtained. i ;

[0150] S30318: Repair I c All sub-blocks, output the repaired image;

[0151] Further, the step S3032 specular reflection separation algorithm includes the following steps:

[0152] S30321: Let I be the RGB image to be repaired, according to the two-color reflection model:

[0153] I(x) = w d Λ(x) + w s Γ

[0154] Wherein, Λ(x) and Γ are diffuse reflection and specular reflection chrominance values respectively, w d and w s are the weights of the corresponding components, w d corresponding to the diffuse reflection component Λ(x), w s corresponding to the specular reflection component Γ; Calculate I max (x), I min (x), I range (x):

[0155] I min (x) = min(I r (x), I g (x), I b (x)) = w d Λ min (x) + w s (x) Γ

[0156] I max (x) = max(I r (x), I g (x), I b (x)) = w d Λ max (x) + w s (x) Γ

[0157] I range (x) = I max (x) - I min (x) = w d (Λ max (x) - Λ min (x))

[0158] Wherein, Λ(x) = [Λ r (x), Λ g (x), Λ b (x)], Λ min (x) = min(Λ r (x), Λg (x),Λ b (x)),Λ max (x) = max(Λ r (x),Λ g (x),Λ b (x))。

[0159] S30322: Calculate pseudo-specular-free image where denotes the mean of I min , calculate the chrominance value of I psf (x):

[0160]

[0161] S30323: Construct maximum-minimum chrominance vector for all pixels p Based on the chrominance space distance metric, the K-means algorithm is used for pixel clustering.

[0162] S30324: For each cluster of pixels, sort the intensity ratios I ratio (x) = I max (x) / I range (x) in a non-strictly increasing order, and select the intensity ratio at the quantile position as the estimate of I ratio (x), and the separated diffuse reflection component is calculated as follows:

[0163]

[0164] The medical image mirror reflection repair system of the embodiment of the application is mainly used for implementing the repair method of the above-mentioned embodiment, as shown in Figure 6 , comprising:

[0165] A single-frame decomposition module is configured to decompose the collected medical video sequence into single-frame images.

[0166] A mask acquisition module is configured to filter out the mirror reflection area in the single-frame image according to the brightness and saturation double thresholds, and generate a mask matrix.

[0167] A repair module is configured to repair the pixels in the mirror reflection area according to the mask matrix and using a frequency domain selection reconstruction algorithm.

[0168] A video merging module is configured to merge the repaired images according to the processing order to obtain a video sequence after mirror reflection repair.

[0169] Each module is mainly used for implementing the above-mentioned method embodiment, and will not be described here.

[0170] In another embodiment of the application, the medical image mirror reflection repair system comprises:

[0171] a single-frame decomposition module configured to decompose the acquired medical video sequence into single-frame images;

[0172] a mask acquisition module configured to divide the reflection types of pixels in the single-frame images according to luminance and saturation threshold values, including saturated specular reflection, non-saturated specular reflection and diffuse reflection, and to extract target reflection regions according to the reflection types to acquire a saturated specular reflection region mask and a non-saturated specular reflection region mask;

[0173] a repair module configured to repair the pixels in the saturated specular reflection region according to the saturated region mask and to repair the pixels in the non-saturated specular reflection region according to the non-saturated specular reflection region mask;

[0174] a video merging module configured to merge the repaired images according to the processing order to obtain a video sequence after specular reflection repair.

[0175] Each module is mainly used to implement the above method embodiments, and will not be described here.

[0176] The application also provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., which stores a computer program, and the program is executed by a processor to realize corresponding functions. The computer readable storage medium of the embodiment is used to realize the method embodiment medical image specular reflection repair method when executed by the processor.

[0177] The above embodiments are taken as endoscope images, but are not limited to endoscope images, and medical images acquired by other devices can also be processed by the method of the application as long as the images meet the image characteristics processed by the technical solution. The specific embodiments of the application repair the specular reflection in the endoscope images, and the image in this embodiment is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solution and the inventive concept of the application within the technical range disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A method for restoring specular reflections in medical images, characterized in that, Includes the following steps: S1: Decompose the acquired medical video sequence into single-frame images; S2: Filter out the specular reflection areas in a single frame image based on the dual thresholds of brightness and saturation, and generate a mask matrix; S3: Repair pixels in the specular reflection region based on the mask matrix and using a frequency domain selection reconstruction algorithm; S4: Merge the repaired images according to the processing order to obtain the video sequence after specular reflection repair; Step S3 specifically includes the following steps: S31: Divide the target image and mask matrix into fixed-size sub-blocks and mark them as image blocks to be processed; S32: Construct a reconstructed region for each image block to be processed, and classify the pixels within the reconstructed region; S33: Calculate the priority of all image patches to be processed, and determine the repair order and number of iterations; S34: Calculate the projection coefficients of the estimated residuals of the reconstructed region in the frequency domain; S35: Select Fourier basis functions and estimate expansion coefficients, then calculate projection coefficients; S36: Update the estimated reconstruction region and reconstruction residuals based on the projection coefficients; S37: Stop when the iteration steps S34-S36 reach the specified number of iterations, and output the repair result of the image block to be processed; S38: All image blocks to be processed have been repaired, and the repaired complete image is output; In step S33, the method for determining the repair order of the image blocks to be processed is as follows: within the region of the image block to be processed... Center coordinates of pixels in the set and sub-block priority level The calculation formula is as follows: in, It belongs to the image block b to be processed The number of pixels in the set. , The smaller the value, the higher the priority. Indicates the area to be rebuilt. m , n It is an integer; Sort the T values ​​of each block to generate the repair order of the image blocks; Set the maximum number of iterations I max Minimum number of iterations I min Calculate the gradient value Grad of the image patch, and determine the iteration number I = (I... max -I min )×Grad.

2. The method for restoring specular reflections in medical images according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21: Obtain the luminance component L and saturation component S of a single frame image; S22: Traverse each pixel and detect specular reflection pixels based on dual thresholds of brightness and saturation; S23: Generate a mask matrix for the specular reflection region based on the detection results; S24: Process the mask matrix to generate the final mask matrix.

3. The method for restoring specular reflections in medical images according to claim 1, characterized in that, The luminance component L is the V component of HSV space, the I component of HSI space, the Y component of YUV space, or the L component of CIELAB space; the saturation component S is the S component of HSV or HSI space or the C component of CIELCH space.

4. The method for restoring specular reflections in medical images according to claim 2, characterized in that, In step S22, the brightness threshold includes a first brightness threshold. Second brightness threshold and the third brightness threshold The saturation threshold is ;set up For the selected set of specular reflection pixels, then for any pixel It must meet one of the following criteria: ① ； ② ； in, Represents any pixel brightness value, Represents any pixel The saturation value, Represented in pixels Centered on, The set of all pixel values ​​in the neighborhood of the radius.

5. The method for restoring specular reflections in medical images according to claim 2, characterized in that, The mask matrix is ​​a 0-1 type matrix, where 1 corresponds to a normal pixel and 0 corresponds to a specular reflection pixel.

6. The method for restoring specular reflections in medical images according to claim 1, characterized in that, In step S31, the size of the sub-block is M×N All sub-blocks in the mask matrix whose elements are not all 1 are marked as image blocks to be repaired.

7. The method for restoring specular reflections in medical images according to claim 1, characterized in that, In step S32, the size of the reconstructed region of the image block b to be processed is 2. M×2N The pixels within the reconstructed area are divided into three categories: usable, repaired, and to be repaired, and are stored in different sets accordingly. , and middle.

8. The method for restoring specular reflections in medical images according to claim 1, characterized in that, Step S2 specifically involves classifying the reflection type of pixels in a single frame image based on brightness and saturation thresholds, including saturated specular reflection, unsaturated specular reflection, and diffuse reflection. Then, the target reflection region is extracted based on the reflection type, and the mask matrix of the saturated specular reflection region and the mask matrix of the unsaturated specular reflection region are obtained. Step S3 specifically involves: repairing the pixels in the saturated specular reflection region based on the saturated region mask matrix; and repairing the pixels in the unsaturated specular reflection region based on the unsaturated specular reflection region mask matrix.

Images