Endoscope video image real-time enhancement method

Abstract

The invention relates to the technical field of image enhancement, in particular to an endoscope video image real-time enhancement method, which comprises the following steps of: based on a single-frame endoscope video image, converting the image from an RGB space to a YCbCr space, and separating chrominance components Cb and Cr to obtain an original chrominance component graph; according to the method, the image is converted from the RGB space to the YCbCr space, chrominance components Cb and Cr are finely separated, more sensitive recognition of focus features can be achieved, focus tissue texture details can be highlighted and background interference can be inhibited by further calculating pixel point gradient energy and fusing a vascular filter response value, and a focus target is clearly highlighted; and linear combination is carried out on the original chrominance components by using the fusion feature weight value, and a dynamic color subspace is constructed, so that the lesion region and the background region are distinguished more obviously.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a real-time enhancement method for endoscopic video images. Background Art

[0002] The field of image enhancement technology mainly focuses on optimizing the original image through computer image processing methods to improve the visual quality of the image or facilitate subsequent analysis.

[0003] Existing image enhancement techniques typically focus solely on improving the overall visual quality of the image, ignoring the color space differences between the lesion and background areas. This approach makes it difficult to highlight local details and color variations in the lesion tissue and tends to amplify noise and interference from non-target areas during the enhancement process. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a real-time enhancement method for endoscopic video images.

[0005] In order to achieve the above object, the present invention adopts the following technical solution, a real-time enhancement method for endoscopic video images, comprising the following steps:

[0006] Based on a single-frame endoscopic video image, the image is converted from RGB space to YCbCr space, and the chrominance components Cb and Cr are separated to obtain the original chrominance component map;

[0007] Based on the original chromaticity component image, the gradient energy of each pixel in the image is calculated to obtain a gradient energy feature, and based on the gradient energy feature, the gradient energy feature is fused with a preset blood vessel morphology filter response value to establish a fusion feature weight value;

[0008] Based on the fusion feature weight value, the fusion feature weight value is used as an adjustment coefficient to linearly combine the two channels of the original chromaticity component image to establish a dynamic color subspace, and based on the dynamic color subspace, the pixel value distribution is statistically analyzed and the numerical interval corresponding to the lesion tissue is determined to generate a target enhancement mapping function;

[0009] Based on the target enhancement mapping function, the pixel values ​​of the original chromaticity component image are applied to perform nonlinear chromaticity stretching and saturation adjustment to obtain an enhanced chromaticity component image. Based on the enhanced chromaticity component image, the enhanced chromaticity component image is merged and recombined with the original chromaticity component image to generate a lesion-specific enhanced pseudo-color image.

[0010] Preferably, the step of obtaining the original chrominance component image is:

[0011] Based on the single-frame endoscopic video image, pixel value matrices of the red channel, green channel, and blue channel of the image are sequentially extracted, and the pixel value matrices of the red channel, green channel, and blue channel are combined according to the conversion coefficients of the YCbCr color space to generate value matrices corresponding to the luminance component Y and the chrominance components Cb and Cr, thereby generating a YCbCr color space image;

[0012] Based on the YCbCr color space image, extract the pixel value matrix corresponding to the chrominance component Cb and the pixel value matrix corresponding to the chrominance component Cr according to the channel position index, store the two sets of pixel value matrices in a two-dimensional array in synchronization with the original image resolution, and generate a chrominance channel matrix set;

[0013] Based on the chromaticity channel matrix set, a set of two-dimensional image frames with the same size as the original image is constructed, and the chromaticity component Cb pixel values ​​and the chromaticity component Cr pixel values ​​are mapped to two grayscale image channels respectively. The two grayscale channel images are combined into a pseudo-color image structure to generate the original chromaticity component image.

[0014] Preferably, the step of acquiring the gradient energy feature is:

[0015] Based on the original chromaticity component image, the grayscale intensity values ​​of the chromaticity component Cb and the chromaticity component Cr are extracted for each pixel point in turn, and the grayscale intensity matrices of the chromaticity component Cb and the chromaticity component Cr are established respectively to obtain the chromaticity component grayscale intensity matrix;

[0016] Based on the chrominance component grayscale intensity matrix, for each pixel position in the matrix, respectively, calculating the intensity change difference in the horizontal direction and the vertical direction, taking the absolute value of the horizontal and vertical differences and summing them, to obtain a gradient difference value matrix for each pixel position;

[0017] Based on the gradient difference value matrix, each gradient difference value in the matrix is ​​normalized, and the processed gradient difference values ​​are averaged in a local window to form a gradient energy feature.

[0018] Preferably, the step of obtaining the fusion feature weight value is:

[0019] Traverse each pixel in the gradient energy feature map, extract the gradient energy value of the pixel point according to the pixel index position, and extract the gradient energy values ​​of all adjacent pixels in its 3×3 neighborhood range, calculate the average value of the gradient energy values ​​of the adjacent pixels, and construct the neighborhood gradient energy mean corresponding to the pixel position to form a gradient and neighborhood energy parameter pair;

[0020] According to the gradient and neighborhood energy parameter pair of each pixel, the pixel value of the Cb channel and the pixel value of the Cr channel of the pixel in the original chrominance component image are simultaneously extracted, the absolute value of the pixel difference between the two channels is calculated, and the response intensity value of the pixel in the vascular morphology filter response image is extracted, and the gradient energy value, the neighborhood energy mean, the Cb channel value, the Cr channel value and the filter response value are used to form a pixel-level fusion calculation input set;

[0021] Based on the pixel-level fusion calculation input set, a fusion feature weight value is calculated.

[0022] Preferably, the step of acquiring the dynamic color subspace is:

[0023] Traversing all pixel positions in the original chrominance component image, extracting the pixel value of each pixel in the Cb channel and Cr channel in turn, and constructing the original chrominance channel value matrix;

[0024] Calculating fused color channel values ​​based on the original chromaticity channel value matrix;

[0025] Based on the fused color channel values, the fused color channel values ​​of all pixel positions are assembled into a two-dimensional pixel array image consistent with the original image structure, and after the combination, the minimum-maximum value normalization is uniformly performed and stretched to the display range, and the normalization result is used as the dynamic color subspace.

[0026] Preferably, the step of obtaining the target enhancement mapping function is:

[0027] Traversing all pixel values ​​in the dynamic color subspace, counting the frequency of occurrence of all pixel grayscale values ​​in the range of 0 to 255, constructing a pixel grayscale histogram and calculating the cumulative distribution function, dividing the cumulative distribution function into 20 segments at equal percentile intervals, determining whether the maximum frequency in each segment is higher than 1.5 times the global average frequency, screening continuous segments that meet the conditions and extracting the starting pixel value and the ending pixel value, and generating a candidate lesion response interval boundary group;

[0028] According to the lesion response candidate interval boundary group, a continuous boundary pair with the largest pixel frequency area is selected as a target response interval, and the lower boundary value of the target response interval is marked as a pixel value response lower limit, and the upper boundary value of the interval is marked as a pixel value response upper limit, to obtain a pixel value response boundary pair;

[0029] Based on the pixel value response boundary pairs, a target enhancement mapping function expression is constructed.

[0030] Preferably, the step of obtaining the enhanced chrominance component image is:

[0031] Based on the target enhancement mapping function, the pixel values ​​of each pixel in the Cb channel and the Cr channel of the original chrominance component image are extracted one by one, and the pixel values ​​are respectively substituted into the target enhancement mapping function for mapping, so as to obtain the mapped chrominance enhancement value of each pixel and generate a chrominance enhancement value matrix;

[0032] Based on the chroma enhancement value matrix, nonlinear stretching is performed on the enhanced chroma values ​​of the Cb channel and the Cr channel on a pixel-by-pixel basis, and the processed Cb channel and Cr channel enhancement values ​​are stored in corresponding new channel positions, thereby generating a nonlinear chroma enhancement channel matrix;

[0033] Based on the nonlinear chroma enhancement channel matrix, all enhanced Cb channel and Cr channel pixel values ​​are traversed, and pixel values ​​exceeding the display range are truncated to a valid range to generate an enhanced chroma component map.

[0034] Preferably, the steps of acquiring the lesion-specific enhanced pseudo-color image are:

[0035] Based on the enhanced chrominance component image, the enhanced pixel values ​​of the Cb channel and the Cr channel in the enhanced chrominance component image are extracted pixel by pixel, and a one-to-one correspondence is established with the pixel values ​​of the Cb channel and the Cr channel at corresponding positions in the original chrominance component image to form enhanced and original chrominance value pixel pairs;

[0036] Based on the enhanced and original chromaticity value pixel pairs, a weighted combination of pixel values ​​is performed on each pixel position, and the enhanced chromaticity value is used as the main component to fuse the original chromaticity value of the corresponding position, highlight the characteristics of the lesion tissue, and form a fused chromaticity channel pixel value;

[0037] Based on the fused chromaticity channel pixel values, the fused Cb channel and Cr channel pixel values ​​are respectively mapped back to the corresponding channel positions in the YCbCr space, and combined with the Y channel value of the luminance component of the original chromaticity component image, a complete YCbCr pseudo-color space image is reconstructed to obtain a lesion-specific enhanced pseudo-color image.

[0038] Compared with the prior art, the advantages and positive effects of the present invention are:

[0039] The present invention converts the image from RGB space to YCbCr space and finely separates the chromaticity components Cb and Cr, which can achieve more sensitive recognition of lesion features. By further calculating the pixel gradient energy and fusing the vascular filter response value, the texture details of the lesion tissue can be highlighted and background interference can be suppressed, thereby achieving clear highlighting of the lesion target; then, the original chromaticity components are linearly combined using the fusion feature weight values ​​and a dynamic color subspace is constructed, making the distinction between the lesion area and the background area more obvious; finally, the target enhancement mapping function is used to perform nonlinear chromaticity stretching and saturation adjustment, thereby enhancing the color contrast of the lesion and improving the visual clarity of the image details, thereby improving the accuracy and effectiveness of real-time enhancement of endoscopic video images and achieving effective mining and highlighting of lesion detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0042] See also Figure 1 The present invention provides a technical solution, a real-time enhancement method for endoscopic video images, comprising the following steps:

[0043] Based on a single-frame endoscopic video image, the image is converted from RGB space to YCbCr space, and the chrominance components Cb and Cr are separated to obtain the original chrominance component map;

[0044] Based on the original chromaticity component image, the gradient energy of each pixel in the image is calculated to obtain the gradient energy feature. Based on the gradient energy feature, it is fused with the preset blood vessel morphology filter response value to establish the fusion feature weight value;

[0045] Based on the fusion feature weight value, the fusion feature weight value is used as an adjustment coefficient to linearly combine the two channels of the original chromaticity component image to establish a dynamic color subspace. Based on the dynamic color subspace, the pixel value distribution is statistically analyzed and the value interval corresponding to the lesion tissue is determined to generate the target enhancement mapping function;

[0046] Based on the target enhancement mapping function, the pixel values ​​of the original chromaticity component image are applied to perform nonlinear chromaticity stretching and saturation adjustment to obtain an enhanced chromaticity component image. Based on the enhanced chromaticity component image, it is merged and recombined with the original chromaticity component image to generate a lesion-specific enhanced pseudo-color image.

[0047] The steps to obtain the original chrominance component image are:

[0048] Based on a single-frame endoscopic video image, the pixel value matrices of the red channel, green channel, and blue channel of the image are sequentially extracted. The pixel value matrices of the red channel, green channel, and blue channel are combined according to the conversion coefficients of the YCbCr color space to generate the corresponding value matrices of the luminance component Y and the chrominance components Cb and Cr, thereby generating a YCbCr color space image;

[0049] Based on the YCbCr color space image, the pixel value matrix corresponding to the chrominance component Cb and the pixel value matrix corresponding to the chrominance component Cr are extracted according to the channel position index respectively, and the two sets of pixel value matrices are stored in a two-dimensional array in synchronization with the original image resolution to generate a chrominance channel matrix set;

[0050] Based on the chromaticity channel matrix set, a set of two-dimensional image frames with the same size as the original image is constructed. The chromaticity component Cb pixel values ​​and the chromaticity component Cr pixel values ​​are mapped to the two grayscale image channels respectively. The two grayscale channel images are combined into a pseudo-color image structure to generate the original chromaticity component image.

[0051] Specifically, based on a single-frame endoscopic video image, the 8-bit (0-255 range) pixel values ​​of the three color channels of red (R), green (G), and blue (B) of the image are first read pixel by pixel to form three two-dimensional numerical matrices with the same size as the original image (for example, width W pixels, height H pixels), namely, the red channel pixel value matrix, the green channel pixel value matrix, and the blue channel pixel value matrix. Then, in order to convert the image from the RGB color space to the YCbCr color space, the calculation coefficient of the brightness component Y is the red channel coefficient K r =0.299, green channel coefficient K g =0.587, blue channel coefficient K b=0.114. For example, for a pixel in the image, its RGB value is (R=210, G=150, B=90), then its corresponding Y component is calculated as Y=0.299×210+0.587×150+0.114×90, that is, Y=62.79+88.05+10.26=161.1, rounded to 161, and its Cb component is calculated as Cb=-0.168736×R-0.331264×G+0.5×B+128, that is, Cb=-0.168736×210-0.331264×150+0.5×90+128=-35.43-49.6 9+45+128=87.88, rounded to 88, and its Cr component is calculated as Cr=0.5×R-0.418688×G-0.081312×B+128, that is, Cr=0.5×210-0.418688×150-0.081312×90+128=105-62.80-7.32+128=162.88, rounded to 163. This conversion operation is performed on each pixel in the image, and finally three independent numerical matrices are obtained, representing the brightness component Y, the chrominance component Cb, and the chrominance component Cr, respectively. These three matrices together constitute the YCbCr color space image.

[0052] Based on the YCbCr color space image, the image contains the numerical matrix of three channels: brightness component Y, blue chroma component Cb and red chroma component Cr. Next, the pixel numerical matrix corresponding to the chroma component Cb and the pixel numerical matrix corresponding to the chroma component Cr are extracted from the YCbCr color space image according to the predefined channel position index. For example, if the YCbCr image data is stored in channel order, the Cb channel may correspond to the second index position and the Cr channel corresponds to the third index position. The extraction operation will traverse each pixel position of the original image (from (0, 0) to (W -1, H-1)), accurately copy the pixel values ​​of the corresponding channels to the new matrix, ensure that the dimensions (width W, height H) of the extracted Cb pixel value matrix and Cr pixel value matrix are exactly the same as the dimensions of the original endoscopic image, and then, in order to ensure the correspondence between the spatial positions of the two chromaticity components in subsequent processing, the two sets of extracted pixel value matrices (i.e., Cb matrix and Cr matrix) are synchronously stored in two independent two-dimensional array structures in the memory in a way that maintains the original image resolution. For example, you can declare Cb_matrix[H][W] and Cr_mat rix[H][W] are two integer or floating-point arrays, where H is the image height and W is the image width. The element Cb_matrix[i][j] of the Cb matrix stores the Cb value of the position (i, j), and the element Cr_matrix[i][j] of the Cr matrix stores the Cr value of the position (i, j). This synchronous storage method ensures that the Cb value and Cr value of any pixel position (i, j) can be accessed simultaneously and accurately, laying the data foundation for subsequent gradient energy calculation and feature fusion, performing validity verification on the extracted chrominance data, and setting a data integrity check. Threshold, for example, counting the ratio of pixels whose median values ​​of the Cb and Cr matrices exceed the standard range of digital video signals (for example, the valid range of Cb / Cr in an 8-bit system is usually 16 to 240). If the ratio exceeds a preset inspection threshold, such as 0.05%, a potential data quality problem is recorded. This 0.05% threshold is based on an analysis of historical endoscopic image data, and it is found that under typical acquisition conditions, exceeding this ratio usually indicates an abnormality in the acquisition device or a transmission error. Through such steps, a chrominance channel matrix set containing two independent but spatially synchronized numerical matrices of Cb and Cr is finally generated.

[0053] Based on the chroma channel matrix set, which contains the chroma component Cb pixel value matrix and the chroma component Cr pixel value matrix consistent with the original image resolution, first construct a set of two-dimensional image frames with the same size (width W, height H) as the original image. The specific operation is to create a grayscale image representation for each Cb component and Cr component, and directly (or after normalization) map each pixel value in the Cb pixel value matrix to the brightness value of a grayscale image channel. Similarly, map each pixel value in the Cr pixel value matrix to the brightness value of another independent grayscale image channel. If the original Cb and Cr values ​​are not in the standard display grayscale range (such as 0-255), normalization stretching is required. For example, if it is known that the valid range of Cb / Cr values ​​in digital video is 16 to 240, and the display range of the target grayscale image channel is 0 to 255, then a linear stretching transformation is applied to each Cb pixel value: Among them, Min in Set to 16, which is the lower limit reference value specified for 8-bit digital video chrominance components. in The span of the input range is 240-16=224, where 240 is the upper limit reference value of the chroma component. out Set to 0, which is the starting value of the target grayscale range, Range out is the span of the target grayscale range, i.e. 255-0=255, so the stretching factor is Taking a pixel with a Cb value of 100 as an example, its mapped grayscale value is round((100-16)×1.13839+0)=round(84×1.13839)=round(95.625)=96. The same normalization stretching process is performed on the Cr pixel value matrix to obtain the Cb grayscale image and the Cr grayscale image. These two grayscale channel images are then combined into a pseudo-color image structure. One implementation method is to assign the pixel value of the Cb grayscale image to the red (R) channel of the pseudo-color image, and the pixel value of the Cr grayscale image to the green (G) channel of the pseudo-color image, while the blue (B) channel is uniformly set to a fixed value, such as 0 or 128. In this way, the color of each pixel is determined by its Cb and Cr values. In this way, the original chrominance component image is generated.

[0054] The steps for obtaining gradient energy features are:

[0055] Based on the original chromaticity component image, the grayscale intensity values ​​of the chromaticity component Cb and the chromaticity component Cr are extracted for each pixel point in turn, and the grayscale intensity matrices of the chromaticity component Cb and the chromaticity component Cr are established respectively to obtain the chromaticity component grayscale intensity matrix;

[0056] Based on the chrominance component grayscale intensity matrix, for each pixel position in the matrix, the intensity change difference in the horizontal direction and the vertical direction are calculated respectively. The absolute value of the horizontal and vertical differences are taken and summed to obtain the gradient difference value matrix of each pixel position;

[0057] Based on the gradient difference value matrix, each gradient difference value in the matrix is ​​normalized, and the processed gradient difference values ​​are locally window averaged to form a gradient energy feature.

[0058] Specifically, based on the original chromaticity component image, the image has been constructed as a pseudo-color image structure in the previous stage, in which the pixel values ​​of the chromaticity component Cb and the pixel values ​​of the chromaticity component Cr are respectively mapped to two independent grayscale image channels, or exist in the form of two independent Cb and Cr grayscale images with the same size as the original image. For each pixel point in the original chromaticity component image, such as a pixel with coordinates (x, y), first extract its grayscale intensity value from its corresponding Cb grayscale channel and Cr grayscale channel respectively. These values ​​are usually in the range of 0 to 255, representing the intensity of the pixel point in the Cb and Cr chromaticity dimensions. For example, if the original chromaticity component The grayscale value of the pixel (x, y) in the Cb channel of the image is 120, and the grayscale value of the pixel (x, y) in the Cr channel is 150. These two values ​​are recorded separately. By traversing all pixels of the original chrominance component image (from (0, 0) to (W-1, H-1), where W is the image width and H is the image height), the Cb grayscale intensity values ​​of all pixels are organized into a two-dimensional matrix with the same dimension as the original image, recorded as the Cb grayscale intensity matrix. Similarly, the Cr grayscale intensity values ​​of all pixels are organized into another two-dimensional matrix of the same dimension, recorded as the Cr grayscale intensity matrix. These two matrices together constitute the chrominance component grayscale intensity matrix.

[0059] Based on the chrominance component grayscale intensity matrix, that is, the Cb grayscale intensity matrix and the Cr grayscale intensity matrix obtained in the previous stage, for each pixel position in the two matrices (for example, the pixel with coordinates (i, j), where i represents the row index and j represents the column index), the intensity change difference in the horizontal direction and the vertical direction is calculated respectively. Specifically, for the pixel point Cb(i, j) in the Cb grayscale intensity matrix, the intensity change difference dCb in the horizontal direction is h (i, j) can be obtained by calculating the difference between its right adjacent pixel and its left adjacent pixel, such as dCb h (i, j) = Cb(i, j+1) - Cb(i, j-1), the vertical intensity change difference dCb v (i, j) can be obtained by calculating the difference between the adjacent pixel below it and the adjacent pixel above it, such as dCb v(i, j) = Cb(i+1, j) - Cb(i-1, j). For pixels at the edge of the image, for example, when j = 0, the j-1 position cannot be accessed. Then, the boundary pixel replication strategy is adopted, that is, Cb(i, -1) = Cb(i, 0), or a single-side difference is adopted, such as dCb h (i,0)=

[0060] Cb(i,1)-Cb(i,0), similarly, the same horizontal intensity change difference dCr is also performed on each pixel point Cr(i,j) in the Cr grayscale intensity matrix h (i, j) and the vertical intensity change difference dCr v (i, j) is calculated, and then, for each pixel point, the absolute value of the difference between the horizontal and vertical intensity changes of its Cb channel is added to obtain the gradient component G of the Cb channel Cb (i,j)=|dCb h (i,j)|+

[0061] |dCb v (i, j)|, similarly calculate the gradient component G of the Cr channel Cr (i,j)=|dCr h (i,j)|+|dCr v (i, j)|, finally, the gradient components of the two channels are summed at each pixel position to form the final gradient difference value GD(i, j)=G at the pixel position Cb (i,j)+G Cr (i, j), organize the gradient difference values ​​GD(i, j) calculated for all pixel points into a new two-dimensional matrix with the same size as the original image, so as to obtain the gradient difference value matrix of each pixel position.

[0062] Based on the gradient difference value matrix, which is calculated by the previous stage and contains the gradient difference value of each pixel in the image combined with the Cb and Cr channel information, each gradient difference value in the gradient difference value matrix is ​​first normalized. This normalization step uses the minimum-maximum normalization method to find the minimum value GD of all gradient difference values ​​in the matrix. min and the maximum value GD max , then for any gradient difference value GD(i,j) in the matrix, its normalized value GD norm (i,j) is calculated as follows: Among them, GD(i,j) is the original gradient difference value, GD min is the minimum gradient value in the entire gradient difference matrix, GD maxis the maximum gradient value. By this formula, all gradient difference values ​​are linearly mapped to the range of 0 to 1. For example, if the GD(i, j) of a pixel is 150, the GD of the current image is min =50 and GD max =250, then its normalized value is (150-50) / (250-50)=100 / 200=0.5. Next, the local window averaging operation is performed on the normalized gradient difference matrix. The specific method is to calculate the local window averaging operation for each pixel GD norm (i, j) defines a fixed-size neighborhood window, such as a 3×3 window. This window size (3×3) is preset based on experience to balance the needs of noise smoothing and detail preservation. For the scale of common vascular structures in endoscopic images, a 3×3 window is a common choice. The average value of all normalized gradient difference values ​​in the window is calculated, and this average value is used as the new value after processing the central pixel point (i, j), that is, the gradient energy value. This local window averaging process is repeated for all pixels in the normalized gradient difference value matrix to form the final gradient energy feature.

[0063] The steps to obtain the fusion feature weight value are:

[0064] Traverse each pixel in the gradient energy feature map, extract the gradient energy value of the pixel point according to the pixel index position, and extract the gradient energy values ​​of all adjacent pixels in its 3×3 neighborhood range, calculate the average value of the gradient energy values ​​of the adjacent pixels, and construct the neighborhood gradient energy mean corresponding to the pixel position to form a gradient and neighborhood energy parameter pair;

[0065] Based on the gradient and neighborhood energy parameter pair of each pixel, the pixel value of the Cb channel and the pixel value of the Cr channel in the original chrominance component image are extracted simultaneously, the absolute value of the pixel difference between the two channels is calculated, and the response intensity value of the pixel in the vascular morphology filter response image is extracted. The gradient energy value, neighborhood energy mean, Cb channel value, Cr channel value and filter response value constitute the pixel-level fusion calculation input set;

[0066] Based on the pixel-level fusion calculation input set, the fusion feature weight value is calculated. The calculation formula is:

[0067]

[0068] Among them, F k is the fusion feature weight value of the k-th pixel position, E k is the gradient energy value of the kth pixel, B k is the mean gradient energy of the k-th pixel in the 3×3 neighborhood, C kis the pixel value of the kth pixel in the Cb channel of the original chrominance component image, D k is the pixel value of the kth pixel in the Cr channel of the original chrominance component image, H k is the response intensity value of the kth pixel in the vascular morphology filter response map, w1 and w2 represent the weight coefficients of the two fusion items in the combination.

[0069] Specifically, based on the gradient energy feature map, which is a two-dimensional data of the same size as the original endoscopic image generated in the previous step, each pixel position stores the normalized gradient energy value. Each pixel in the gradient energy feature map (identified by pixel index k or coordinates (i, j)) is processed. First, the gradient energy value of the pixel itself is directly read, which is recorded as E k For example, if the stored value of pixel (i, j) in the gradient energy feature map is 0.65, then E k =0.65, then a 3×3 neighborhood window is defined around the pixel point (i, j). The size of this 3×3 window is pre-set based on the statistical analysis of typical lesion areas and vascular texture features in endoscopic images, aiming to effectively capture local context information. The window contains the 8 directly adjacent pixels around the pixel point, namely, the pixels at (i-1, j-1), (i-1, j), (i-1, j+1), (i, j-1), (i, j+1), (i+1, j-1), (i+1, j), (i+1, j+1) For pixels at the image boundary, when their neighboring pixels are less than 8, the boundary pixel value duplication method is used to fill them. For example, for the upper left corner pixel (0, 0), the neighboring pixel values ​​to the left and above will be filled with the gradient energy values ​​of themselves or their valid neighbors. The gradient energy values ​​of these 8 adjacent pixels are extracted from the gradient energy feature map, and the 8 gradient energy values ​​are added and divided by 8 to calculate the average value of the gradient energy values ​​of these adjacent pixels. This average value is the neighborhood gradient energy mean corresponding to the current pixel position (i, j), which is recorded as B k For example, if the gradient energy values ​​of 8 neighboring pixels are 0.5, 0.6, 0.55, 0.62, 0.68, 0.53, 0.61, and 0.59 respectively, then their average value B k =(0.5+0.6+0.55+0.62+0.68+0.53+0.61+0.59) / 8=4.68 / 8=0.585. By performing the above operation on all pixels in the gradient energy feature map, the gradient energy value E of each pixel is constructed. k and its neighborhood gradient energy mean B k , which together constitute the gradient and neighborhood energy parameter pair of the pixel position.

[0070] According to the gradient and neighborhood energy parameter pair of each pixel, the pixel value of the Cb channel is extracted for the corresponding position of the pixel in the original chrominance component image (the image is obtained by converting and processing RGB in the early steps, and contains two chrominance channels Cb and Cr, whose pixel values ​​are usually in the range of 0-255), and is recorded as C k , and extract the pixel value of its Cr channel, recorded as D k For example, for pixel k, if the value of the Cb channel in the original chrominance component image is 120 and the value of the Cr channel is 160, then C k =120, D k =160, then calculate the absolute value of the difference between the two chrominance channel pixel values, i.e. |C k -D k |, in this case |120-160|=|-40|=40. In addition, it is necessary to extract the corresponding response intensity value of the pixel in the vascular morphology filter response map, which is recorded as H k The vascular morphology filter response map is pre-generated by applying one or a group of vascular enhancement filters (such as Frangi filters) to the original endoscopic image. Its parameters (for example, for the Frangi filter, the scale parameter σ is usually set to a range of 1 to 5 pixels with a step size of 0.5 pixels to detect blood vessels of different thicknesses, the grayscale compensation factor β is set to 0.5, and the objectivity measurement factor c is set to half of the maximum grayscale level of the current image. These parameters are optimized based on a large number of endoscopic image databases in order to maximize the vascular response intensity) are preset based on clinical experience and analysis of the target vascular structure characteristics. The intensity value of each pixel on the response map (usually normalized to the range of 0 to 1, such as 0.9 for strong vascular response and 0.1 for weak response) reflects the possibility that the pixel belongs to the vascular structure or the significance of the vascular feature. For example, if the value of pixel k in the vascular morphology filter response map is 0.85, then H k = 0.85, and the gradient energy value E of the pixel thus obtained k , neighborhood gradient energy mean B k , original chrominance component image Cb channel value C k , original chrominance component image Cr channel value D k And the blood vessel morphology filter response strength value H k These five values ​​together constitute the pixel-level fusion calculation input set for subsequent calculations.

[0071] formula: The benefit of the formula is that it calculates the fusion feature weight value of each pixel by weighted fusion of two different types of image features. The first term Combined with the pixel's own gradient energy E kand the neighborhood gradient energy mean B k , which can effectively characterize the local texture complexity and edge strength of the image and is sensitive to the structural abnormalities that may be presented by the lesion tissue. The second term |C k -D k |·H k The chromaticity information (the absolute value of the difference between the Cb and Cr channels) and the vascular morphology characteristics (the vascular morphology filter response value H k ), which aims to highlight the vascular proliferation or color abnormality areas related to the lesions. By adjusting the weight coefficients w1 and w2, we can flexibly focus on the contribution of different features to the final weight value, thereby specifically enhancing the characteristics of specific types of lesions.

[0072] Parameter E k The steps to obtain E are: k Represents the gradient energy value of the k-th pixel. This value is directly read from the corresponding pixel position in the "gradient energy feature map" generated in the previous step. The gradient energy feature map is obtained by calculating the gradient difference of the original chrominance component map, normalizing it, and averaging it in the local window. It reflects the edge or texture intensity of the local area of ​​the pixel point. For example, in the gradient energy feature map, the value of the k-th pixel position is 0.72, then E k =0.72.

[0073] Parameter B k The steps to obtain B are: k represents the average gradient energy of the k-th pixel in its 3×3 neighborhood. This value is calculated by the step of “traversing each pixel in the gradient energy feature map…” in this application. Specifically, the gradient energy values ​​of the 8 neighboring pixels around the k-th pixel in the “gradient energy feature map” are extracted, and then the average of these 8 values ​​is calculated, which reflects the overall gradient activity of the area around the pixel. For example, if the average gradient energy value of the 8 neighboring pixels of the k-th pixel is 0.65, then B k =0.65.

[0074] Parameter C k The steps to obtain C are: k Represents the pixel value of the kth pixel in the Cb (blue chrominance) channel of the original chrominance component image. This value is directly extracted from the Cb channel of the previously converted and stored "original chrominance component image" according to the pixel index k (or its corresponding two-dimensional coordinate). Its value is usually an 8-bit unsigned integer ranging from 0 to 255, representing the blue chrominance component intensity of the pixel. For example, in the Cb channel of the original chrominance component image, the pixel value of the kth pixel is 115, then C k =115.

[0075] Parameter D k The steps to obtain are: Dk Represents the pixel value of the kth pixel in the Cr (red chrominance) channel of the original chrominance component image, and C k The acquisition method is similar. This value is directly extracted from the Cr channel of the "original chromaticity component image" according to the pixel index k. It ranges from 0 to 255 and represents the red chromaticity component intensity of the pixel. For example, in the Cr channel of the original chromaticity component image, the pixel value of the kth pixel is 155, then D k =155.

[0076] Parameter H k The steps to obtain H are: k represents the response intensity value of the kth pixel in the “vascular morphology filter response map”. This response map is obtained in advance by applying a specific vascular enhancement filter (such as the Frangi filter) to the image. The filter parameters are preset according to the typical morphological characteristics of the target blood vessels and the characteristics of the imaging device. For example, the scale parameter σ of the Frangi filter is set to range from 1.0 to 4.0 pixels with a step size of 0.5 to detect blood vessels of different diameters. The sensitivity parameters α, β, and c are set to 0.5, 0.5, and half of the maximum response value, respectively (empirical values, based on test optimization of 100 images containing blood vessels). The response value H k It is usually normalized to a range of 0 to 1. The larger the value, the higher the possibility that the pixel belongs to the vascular structure or the more significant the vascular feature. For example, for the k-th pixel, its value in the vascular morphology filter response map is 0.88, then H k =0.88.

[0077] The steps for obtaining parameters w1 and w2 are as follows: w1 and w2 represent the weight coefficients of the two fusion items in the combination, respectively. Their setting basis is the consideration of the emphasis on different features and the range of each feature value. Usually, the sum of these two weight coefficients can be set to 1 (i.e., w1+w2=1). The specific value can be determined by performing optimization experiments on a set of training endoscopic images containing typical lesions. For example, different w1 values ​​are set (such as from 0.1 to 0.9, with a step size of 0.1, corresponding to w2=1-w1), and F is calculated. k It is applied to the subsequent enhancement process, and then medical experts score the enhancement effect, or use objective evaluation indicators (such as the contrast between the lesion area and the background area, the signal-to-noise ratio, etc.) to evaluate it, and select a set of w1 and w2 values ​​that make the enhancement effect optimal (such as the highest expert score, or the objective indicator reaches a peak). For example, by testing 50 endoscopic images containing early gastric cancer lesions, it was found that when w1=0.3 and w2=0.7, the color contrast and texture clarity of the lesion area had the best overall performance, so w1=0.3 and w2=0.7 were set.

[0078] Calculation process:

[0079] Taking a specific pixel k as an example, substitute the sample parameter values ​​obtained previously for calculation:

[0080] Known E k =0.72, B k =0.65, C k =115, D k =155,H k =0.88, w1=0.3, w2=0.7.

[0081] First calculate the square root of the first term:

[0082]

[0083] Then calculate the absolute value and product part of the second term:

[0084] |C k -D k |·H k =|115-155|·0.88;

[0085] =|-40|·0.88;

[0086] =40·0.88;

[0087] =35.2;

[0088] Then multiply the two calculated items by the corresponding weight coefficients and add them together:

[0089] F k = w1·(first result)+w2·(second result);

[0090] F k =0.3 0.9700 + 0.7 35.2;

[0091] F k =0.291+24.64;

[0092] F k =24.931;

[0093] The results show that for the selected pixel k, its fusion feature weight value F k The value is calculated as 24.931, which combines the gradient energy information, chromaticity difference information and vascular morphology response information of the pixel, and the fusion feature weight value F k It will be used as the adjustment coefficient for adjusting the original chrominance component image in the subsequent steps.

[0094] The steps to obtain the dynamic color subspace are:

[0095] Traverse all pixel positions in the original chrominance component image, extract the pixel value of each pixel in the Cb channel and Cr channel in turn, and construct the original chrominance channel value matrix;

[0096] Based on the original chromaticity channel value matrix, the fused color channel value is calculated using the following formula:

[0097]

[0098] Among them, Q k is the color channel value after fusion at the k-th pixel position, M k is the pixel value of the Cb channel of the kth pixel in the original chrominance component image, N k is the pixel value of the kth pixel in the Cr channel of the original chrominance component image, F k is the fusion feature weight value corresponding to the k-th pixel position;

[0099] Based on the fused color channel values, the fused color channel values ​​of all pixel positions are assembled into a two-dimensional pixel array image with the same structure as the original image. After the combination, the minimum-maximum normalization is uniformly performed and stretched to the display range, and the normalized result is used as the dynamic color subspace.

[0100] Specifically, all pixel positions in the original chromaticity component image are traversed. The original chromaticity component image is obtained after color space conversion and channel extraction in the aforementioned steps, and contains Cb and Cr chromaticity information of each pixel. For each pixel in the image, for example, the pixel index is k (or its two-dimensional coordinates are (i, j), where i represents the row, j represents the column, from (0, 0) to (H-1, W-1), H is the image height, and W is the image width), the pixel value of the pixel is extracted from the Cb channel of the original chromaticity component image in sequence, and the pixel value of the pixel is extracted from the Cr channel at the same time. These two values ​​are usually integers in the range of 0 to 255. The Cb channel pixel values ​​extracted from all pixels are organized into a two-dimensional matrix with the same size as the original image (H×W), which is recorded as the Cb original channel value matrix. Similarly, the Cr channel pixel values ​​extracted from all pixels are organized into another two-dimensional matrix of the same size, which is recorded as the Cr original channel value matrix. These two matrices (Cb original channel value matrix and Cr original channel value matrix) together constitute the original chromaticity channel value matrix.

[0101] formula: The benefit of the formula is that it combines the original chrominance information (the average of Cb and Cr) with the previously calculated fusion feature weight value F k A nonlinear combination is performed by taking the average value of Cb and Cr The basic chromaticity information of the pixel is retained, and the product term (1+ln(1+F k)) utilizes the fusion feature weight value F k The basic chromaticity is modulated, and the logarithmic function ln(1+F k ) is introduced, so that when F k When F is large (i.e., when the correlation between the pixel and the lesion feature is high), the adjustment amplitude will also increase accordingly, but the growth rate will increase with the F k This nonlinear adjustment method helps to highlight the color of the high feature response area while avoiding oversaturation or distortion in the area with extremely high feature response. k Smaller areas require smaller adjustments, preserving the natural look and feel of the original image. This design enables the fused color channel values ​​to more effectively reflect the specificity of the lesion and enhances the distinguishability between the lesion area and the surrounding normal tissue.

[0102] Parameter M k The steps to obtain M are: k Represents the pixel value of the Cb (blue chrominance) channel of the k-th pixel in the original chrominance component image. This value is directly extracted from the Cb original channel value matrix in the constructed "original chrominance channel value matrix" according to the pixel index k (or its corresponding two-dimensional coordinate). For example, for the k-th pixel, its value in the Cb original channel value matrix is ​​115, then M k =115.

[0103] Parameter N k The steps to obtain N are: k Represents the pixel value of the Cr (red chrominance) channel of the kth pixel in the original chrominance component image, and M k The acquisition method is similar. This value is directly extracted from the Cr original channel value matrix in the constructed "original chrominance channel value matrix" according to the pixel index k. For example, for the kth pixel, its value in the Cr original channel value matrix is ​​155, then N k =155.

[0104] Parameter F k The steps to obtain F are: k is the fusion feature weight value corresponding to the k-th pixel position, which is calculated according to the previous main step "calculating the fusion feature weight value based on the pixel-level fusion calculation input set". For example, in the example of the previous step, the calculated F k =24.931.

[0105] Calculation process:

[0106] Taking a specific pixel k as an example, substitute the sample parameter values ​​obtained previously for calculation:

[0107] Known M k =115, Nk =155, F k =24.931.

[0108] First calculate the average of the Cb and Cr channel pixel values:

[0109]

[0110] Then calculate the logarithmic adjustment factor part:

[0111] 1+ln(1+F k )=1+ln(1+24.931);

[0112] =1+ln(25.931);

[0113] ln(25.931)≈3.2554;

[0114] Therefore, 1+3.2554=4.2554;

[0115] Finally, multiply the two parts to get the fused color channel value Q k :

[0116] Q k =(average chromaticity value)·(logarithmic adjustment factor);

[0117] Q k =135·4.2554;

[0118] Q k =574.479;

[0119] The results show that for the selected pixel k, its fused color channel value Q k The calculation is 574.479, which is based on the original Cb and Cr chromaticity information of the pixel and its fusion feature weight value F k After nonlinear enhancement adjustment, higher Q k A value of usually means that the pixel is significant in both raw chromaticity and feature response, or that the feature response is extremely high.

[0120] Based on the fused color channel value, that is, the Q calculated for each pixel in the image k The fused color channel values ​​of all pixel positions (from (0, 0) to (H-1, W-1)) are assembled into a two-dimensional pixel array image that is completely consistent with the original endoscopic video image structure (i.e., height H, width W) according to their spatial position relationship in the original image. The value of each pixel point of this newly generated two-dimensional image is the corresponding Q k value, due to the calculated Q kThe value may exceed the standard image display range (such as 0-255), and its dynamic range may be very large. Therefore, after the combination is completed, it is necessary to perform a minimum-maximum normalization stretching process on the entire two-dimensional pixel array image. The specific operation is to first traverse the entire two-dimensional pixel array image and find all Q k The minimum value Q min and maximum Q max , then for each Q in the image k The (i,j) value applies the linear stretch transformation formula: Among them, DisplayMin and DisplayMax are the minimum and maximum values ​​of the target display range. Usually for 8-bit grayscale images, they are set to DisplayMin = 0 and DisplayMax = 255. For example, if all Q k Value, Q min =50.0 and Q max =800.0, for a Q k The pixel with (i, j) = 574.479 has a normalized value of After this normalized stretching process, all pixel values ​​of the obtained new two-dimensional pixel array image will fall within a preset display range (eg, 0-255). This normalized two-dimensional pixel array image is used as a dynamic color subspace.

[0121] The steps to obtain the target enhancement mapping function are:

[0122] All pixel values ​​in the dynamic color subspace are traversed, and the frequency of occurrence of all pixel grayscale values ​​in the range of 0 to 255 is counted. A pixel grayscale histogram is constructed and the cumulative distribution function is calculated. The cumulative distribution function is divided into 20 segments at equal percentile intervals. Whether the maximum frequency in each segment is higher than 1.5 times the global average frequency is determined. Continuous segments that meet the conditions are screened and the starting and ending pixel values ​​are extracted to generate a group of candidate lesion response interval boundaries.

[0123] According to the lesion response candidate interval boundary group, the continuous boundary pair with the largest pixel frequency area is selected as the target response interval, and the lower boundary value of the target response interval is marked as the pixel value response lower limit, and the upper boundary value of the interval is marked as the pixel value response upper limit, to obtain the pixel value response boundary pair;

[0124] Based on the pixel value response boundary pair, the target enhancement mapping function expression is constructed, which is:

[0125]

[0126] Wherein, T(x) represents the output value of the input pixel value x under the target enhancement mapping, A represents the lower limit of the pixel value response in the pixel value response boundary pair, representing the starting point of the response of the lesion area, ZB represents the upper limit of the pixel value response in the pixel value response boundary pair, representing the end point of the response of the lesion area, and x is the original pixel value of any pixel position in the dynamic color subspace.

[0127] Specifically, all pixel values ​​in the dynamic color subspace are traversed. The dynamic color subspace is a two-dimensional pixel array image obtained by calculating the fused color channel values ​​and performing normalized stretching in the previous stage. Its pixel value range is 0 to 255. First, the number of times each grayscale value (from 0 to 255) appears in the dynamic color subspace is counted, that is, the frequency, and a pixel grayscale histogram containing 256 entries is constructed, where each entry corresponds to a grayscale level and the number of pixels in the image. For example, the grayscale value 100 appears 580 times, the grayscale value 150 appears 1200 times, and so on. Based on this histogram, its cumulative Distribution function (CDF), the value of CDF at any gray level g represents the proportion or total number of pixels in the image whose gray value is less than or equal to g. Next, the entire cumulative distribution function (whose value range is from 0 to 1, representing 0% to 100% of pixels) is divided into 20 consecutive segments at equal intervals according to percentiles. Each segment represents 5% of the total number of pixels (because 100% / 20 segments = 5% per segment). For example, the first segment corresponds to the pixel value range of CDF from 0 to 0.05, the second segment corresponds to the pixel value range of CDF from 0.05 to 0.10, and so on. The 20th segment corresponds to the pixel value range of CDF from 0.95 to 1.0 For the 20 segments divided, find the gray value with the highest frequency in the original pixel grayscale histogram in each segment and its frequency (i.e., the local peak frequency in the segment). At the same time, calculate the global average frequency of all grayscale levels (0-255) in the entire dynamic color subspace, that is, the total number of pixels divided by 256. Then, determine whether the maximum frequency in each segment is higher than 1.5 times the global average frequency. This 1.5-fold threshold coefficient is set based on statistical analysis of the pixel distribution characteristics of the lesion area in a large number of endoscopic images and expert experience. It aims to screen those pixels with relatively concentrated distribution that may correspond to specific For the grayscale interval of tissue (such as lesions), all continuous segments that meet the condition of "maximum frequency within a segment>1.5×global average frequency" are merged, and the starting pixel values ​​and ending pixel values ​​of the original grayscale values ​​covered by these continuous merged segments are extracted to form a set of lesion response candidate interval boundary pairs. For example, if segments 5, 6, and 7 all meet the conditions, and the grayscale range corresponding to segment 5 is [80, 95], segment 6 is [96, 110], and segment 7 is [111, 125], then a candidate interval boundary pair [80, 125] is formed, and finally a lesion response candidate interval boundary group containing one or more such boundary pairs is generated.

[0128] According to the lesion response candidate interval boundary group, the group is obtained by screening in the previous step by analyzing the histogram and cumulative distribution function of the dynamic color subspace, and contains one or more grayscale value intervals that may correspond to the lesion tissue. For example, the candidate interval boundary group contains {[80, 125], [160, 190]}. Next, it is necessary to select an optimal target response interval from these candidate intervals. The selection criterion is the continuous boundary pair with the largest pixel frequency area. The specific calculation method is that for each candidate interval (for example, [A, ZB]), return to the original pixel grayscale histogram, accumulate the frequency of occurrence of pixels of all grayscale levels from grayscale value A to grayscale value ZB, and obtain the total number of pixels covered by the candidate interval. This total number of pixels represents the "pixel frequency area" of the interval. For example, for the candidate interval [80, 125], the pixel frequency area it covers is the frequency of grayscale value 80 + the frequency of grayscale value 81 +... + the frequency of grayscale value 125 in the histogram. If the sum is 15,000 pixels, and for the candidate interval [160, 190], the pixel frequency area it covers is 10,000 pixels, then compare these two areas and select the candidate interval with the largest area as the final target response interval. In this example, [80, 125] is selected as the target response interval, and then the lower boundary value of the selected target response interval (80 in this example) is marked as the pixel value response lower limit, recorded as A, and the upper boundary value of the interval (125 in this example) is marked as the pixel value response upper limit, recorded as ZB. Through this process, a set of determined pixel value response boundary pairs (A, ZB) is obtained.

[0129] formula: The benefit of the formula is that it defines a Sigmoid-type nonlinear mapping function, which is often used to smoothly map input values ​​to an output range of 0 to 1 and has good controllability. By setting the parameters A (lower limit of pixel value response) and ZB (upper limit of pixel value response), the response range of the mapping function can be controlled so that the pixel values ​​within the selected target response range (defined by A and ZB, representing the characteristic pixel value range of the lesion tissue in the dynamic color subspace) are significantly stretched and enhanced, while the pixel values ​​outside the range are suppressed or remain unchanged. Specifically, when the input pixel value x is close to A, T(x) is close to 0; when x is close to ZB, T(x) is close to 1; when x is at the center of A and ZB (i.e., x = (A + ZB) / 2), the power of the exponential term is 0, and at this time T(x) = 1 / (1 + e 0)=1 / 2, which indicates that the mapping function has the steepest rising slope near the center point of the target interval and can distinguish the subtle differences in the target interval to the greatest extent. The constants 4 and -2 are empirical parameters used to adjust the shape and center position of the Sigmoid function, so that the mapping curve has appropriate transition characteristics near points A and ZB.

[0130] The steps for obtaining parameter A are as follows: A represents the lower limit of the pixel value response in the pixel value response boundary pair, which represents the starting point of the pixel value interval identified as being related to the lesion area response in the dynamic color subspace. The value is determined by the method described in detail in "Based on the lesion response candidate interval boundary group..." of this application, that is, the lower limit boundary value of the interval with the largest pixel frequency area covered is selected from multiple candidate intervals, and its numerical value range is between 0 and 255. For example, in the calculation example of the previous step, the lower limit of the selected target response interval is 80, then A=80.

[0131] The steps for obtaining the parameter ZB are as follows: ZB represents the upper limit of the pixel value response in the pixel value response boundary pair. It represents the end point of the pixel value interval identified as related to the response of the lesion area in the dynamic color subspace. It is similar to the method of obtaining the parameter A, that is, the upper limit boundary value of the interval with the largest pixel frequency area is selected from multiple candidate intervals. Its numerical range is also between 0 and 255, and ZB>A. The specific value also depends on the characteristics of the current image. For example, in the example of the previous step, the upper limit of the selected target response interval is 125, then ZB=125.

[0132] The steps for obtaining the parameter x are as follows: x is the original pixel value of any pixel position in the dynamic color subspace. When applying this target enhancement mapping function, it is necessary to traverse each pixel in the dynamic color subspace, read its grayscale value in the subspace (range 0-255), and substitute the grayscale value as the input x into the formula to calculate its mapped output value. For example, if a pixel in the dynamic color subspace currently being processed has a grayscale value of 100, then x = 100.

[0133] The steps for obtaining the enhanced chrominance component image are:

[0134] Based on the target enhancement mapping function, the pixel values ​​of each pixel in the Cb channel and Cr channel of the original chrominance component image are extracted one by one, and respectively substituted into the target enhancement mapping function for mapping, and the mapped chrominance enhancement value of each pixel is obtained to generate a chrominance enhancement value matrix;

[0135] Based on the chroma enhancement value matrix, the enhanced chroma values ​​of the Cb channel and the Cr channel are subjected to nonlinear stretching pixel by pixel, and the processed Cb channel and Cr channel enhancement values ​​are stored in the corresponding new channel positions to generate a nonlinear chroma enhancement channel matrix;

[0136] Based on the nonlinear chroma enhancement channel matrix, all enhanced Cb channel and Cr channel pixel values ​​are traversed, and the pixel values ​​that exceed the display range are truncated to the valid range to generate an enhanced chroma component map.

[0137] Specifically, based on the target enhancement mapping function, that is, the one constructed in the previous step Wherein the parameters A and ZB are the lower and upper limits of the pixel value response determined according to the dynamic color subspace, for example, 80 and 125 respectively. For each pixel point (for example, pixel index k, corresponding to coordinates (i, j)) in the original chrominance component image (which contains the unmodified Cb and Cr channel pixel values, usually in the range of 0-255), the original Cb value of the k-th pixel is first extracted from the Cb channel of the original chrominance component image, denoted as Cb orig,k , and extract its original Cr value from the Cr channel, recorded as Cr orig,k , for example, if the Cb of pixel k orig,k =95 and Cr orig,k =140, then, the extracted Cb orig,k The value is substituted into the target enhancement mapping function T(x) as input, and the mapped chromaticity enhancement value T of the pixel Cb channel is calculated. Cb,k , the specific calculation is: first calculate the intermediate variable but Similarly, Cr orig,k The value is substituted into the target enhancement mapping function T(x) as input, and the mapped chromaticity enhancement value T of the pixel Cr channel is calculated. Cr,k , calculated as: intermediate variable but 0.9655, this operation is performed on all pixels in the original chrominance component image, and the T obtained for all pixels is Cb,k The values ​​are organized into a Cb channel mapped chroma enhancement value matrix of the same size as the original image, and all T Cr,k The values ​​are organized into the chroma enhancement value matrix after Cr channel mapping, and these two matrices together constitute the chroma enhancement value matrix.

[0138] Based on the chroma enhancement value matrix, the matrix contains the chroma enhancement value T after Cb channel mapping calculated for each pixel in the previous stage Cb,k and chroma enhancement value T after Cr channel mapping Cr,k (its value range is approximately between 0 and 1), and the enhanced chromaticity values ​​of these two channels are subjected to nonlinear stretching pixel by pixel. This nonlinear stretching adopts the gamma correction method. The specific formula is: Where V mappedis the input chroma enhancement value after mapping (ie T Cb,k or T Cr,k ), γ s is the gamma correction coefficient, S max is the maximum value of the target output range, set to 255, the gamma correction coefficient γ s The setting of γ was determined by experiments on a set of 100 endoscopic images with different lesion types and imaging conditions. s The value range is set to 0.5 to 1.5, with a step size of 0.1. For each γ s The complete nonlinear stretching process was performed on the test image set. After that, three endoscopists with more than five years of clinical experience independently blindly evaluated the color contrast and texture detail clarity of the lesion area in the enhanced image. The scoring standard was 1 to 5, with 5 being the best and 1 being the worst. The statistical results of all test images at different γ s The average physician score under the value of γ is selected to obtain the highest average physician score. s The value is used as the final gamma correction coefficient. For example, after the above experimental evaluation, when γ s = 0.8, the average physician score was 4.2 points, which was the highest among all the tested γ s The value is the highest, so set γ s =0.8, taking pixel k as an example, its T Cb,k ≈0.3392, then the value of the Cb channel after stretching is Cb stretched,k =round((0.3392) 0.8 ·255)=round(0.4184·255)=round(106.692)=107, its T Cr,k ≈0.9655, then the value of the Cr channel after stretching is Cr stretched,k =round((0.9655) 0.8 ·255)=round(0.9722·255)=round(247.911)=248. The Cb channel enhancement values ​​obtained after this nonlinear stretching process for all pixels are stored in a new Cb channel matrix, and the Cr channel enhancement values ​​are stored in a new Cr channel matrix. These two new matrices together constitute the nonlinear chroma enhancement channel matrix.

[0139] Based on the nonlinear chroma enhancement channel matrix, this matrix contains the result of nonlinear stretching of the chroma enhancement value after mapping the Cb and Cr channels of each pixel in the previous stage, such as Cb stretched,k =107 and Cr stretched,k=248. In theory, these values ​​have been scaled to approach the range of 0-255. However, due to the calculation accuracy or the characteristics of the stretching function, there may still be individual pixel values ​​that slightly exceed the standard 8-bit display range. Therefore, it is necessary to traverse the enhanced Cb channel pixel values ​​and Cr channel pixel values ​​of all pixels in the nonlinear chroma enhancement channel matrix, check each value, and truncate it to a predefined valid display range. The valid display range is set according to the standard 8-bit image channel, that is, the minimum value is 0 and the maximum value is 255. The specific operation is: for any enhanced Cb channel pixel value Cb val , if Cb val <0, then correct its value to 0; if Cb val >255, then correct its value to 255; if 0≤Cb val ≤255, then its value remains unchanged, and the enhanced Cr channel pixel value Cr val The same truncation operation is also performed. For example, if the Cb of a pixel stretched If the calculated result is -5, it will be truncated to 0. If the calculated result is 260, it will be truncated to 255. For Cb in the example stretched,k =107 and Cr stretched,k =248. Since they are all in the range of [0, 255], the truncated values ​​remain unchanged, still 107 and 248. The Cb channel values ​​and Cr channel values ​​of all pixels after this truncation are stored separately to form two final chrominance channels that ensure that all values ​​are within the valid display range. These two channels together constitute the enhanced chrominance component map.

[0140] The steps for acquiring lesion-specific enhanced pseudo-color images are as follows:

[0141] Based on the enhanced chroma component image, the enhanced pixel values ​​of the Cb channel and the Cr channel in the enhanced chroma component image are extracted pixel by pixel, and a one-to-one correspondence is established with the pixel values ​​of the Cb channel and the Cr channel at the corresponding position of the original chroma component image to form enhanced and original chroma value pixel pairs;

[0142] Based on the enhanced and original chromaticity value pixel pairs, a weighted combination of pixel values ​​is performed at each pixel position. The enhanced chromaticity value is used as the main component, and the original chromaticity value of the corresponding position is fused to highlight the characteristics of the lesion tissue to form the fused chromaticity channel pixel value;

[0143] Based on the fused chromaticity channel pixel values, the fused Cb channel and Cr channel pixel values ​​are mapped back to the corresponding channel positions in the YCbCr space respectively, and combined with the Y channel value of the luminance component of the original chromaticity component image, the complete YCbCr pseudo-color space image is reconstructed to obtain a lesion-specific enhanced pseudo-color image.

[0144] Specifically, based on the enhanced chrominance component map, which is obtained through a series of mapping, stretching and truncation processes in the previous steps, and contains the Cb and Cr channel pixel values ​​(range 0-255) of each pixel after enhancement, an operation is performed on each pixel point in the enhanced chrominance component map (for example, pixel index k, corresponding coordinates (i, j)). First, the enhanced Cb pixel value of the kth pixel is extracted from the Cb channel of the enhanced chrominance component map, which is recorded as Cb enh,k And extract the enhanced Cr pixel value from its Cr channel, recorded as Cr enh,k , for example, if the Cb of pixel k enh,k =107 and Cr enh,k =248. At the same time, it is also necessary to access the original chrominance component image obtained at an earlier stage (that is, without any enhancement processing, only the Cb and Cr channel images obtained after RGB to YCbCr conversion), and extract the original Cb pixel value at the same position as the current pixel k from the Cb channel of the original chrominance component image, which is recorded as Cb orig,k , and extract the original Cr pixel value from its Cr channel, recorded as Cr orig,k For example, if the value of pixel k in the original chrominance component image is Cb orig,k =95 and Cr orig,k =140, in this way, for each pixel k in the image, a set of corresponding relationships consisting of four values ​​is established: (Cb enh,k , Cr enh,k , Cb orig,k , Cr orig,k ), this set of values ​​constitutes the enhanced and original chrominance value pixel pair.

[0145] Based on the enhanced and original chrominance value pixel pairs, the pixel pair provides the enhanced Cb and Cr values ​​(Cb enh,k , Cr enh,k ) and the original Cb and Cr values ​​(Cb orig,k , Cr orig,k ), then perform a weighted combination of pixel values ​​at each pixel position to enhance the chromaticity value as the main component and fuse the original chromaticity value of the corresponding position. The specific calculation formula of the weighted combination is: For the Cb channel, the fused Cb pixel value Cb fused,k =α·Cb enh,k +(1-α)·Cb orig,k , for the Cr channel, the fused Cr pixel value Cr fused,k =α·Cr enh,k +(1-α)·Cr orig,k, where α is a preset weight coefficient with a value range of 0 to 1, which is used to control the proportion of enhanced chromaticity values ​​in the final fusion result. The value of α is set based on the visual evaluation of the enhancement effect and clinical needs. By conducting expert evaluation on endoscopic images (including at least 200 images of different lesions and normal mucosa) processed with a series of different α values ​​(for example, from 0.5 to 0.9, with a step size of 0.05) (three senior endoscopists independently evaluated the prominence of the lesion features and the naturalness of the overall image, using a 5-point scale, with 5 being the best), the α value that makes the lesion features most clearly distinguishable while maintaining the overall acceptability of the image is selected. For example, if the experiment shows that when α = 0.8, the outline and color details of the lesion are best highlighted, and the image does not have artifacts or unnaturalness caused by over-enhancement, then α = 0.8 is set. Taking this as an example, substituting the example pixel value: Cb fused,k =0.8·107+(1-0.8)·95=0.8·107+0.2·95=85.6+19=104.6, rounded to 105, Cr fused,k =0.8·248+(1-0.8)·140=0.8·248+0.2·140=198.4+28=226.4, rounded to 226. This weighted combination operation is performed on all pixels in the image, and the calculated Cb fused,k and Cr fused,k They are organized separately to form the fused chroma channel pixel values.

[0146] Based on the fused chroma channel pixel value, that is, the fused Cb value Cb calculated for each pixel in the previous stage fused,k and Cr value Cr fused,k (For example, Cb fused,k =105 and Cr fused,k = 226, these values ​​are still in the range of 0-255), and these fused Cb channel pixel values ​​and Cr channel pixel values ​​are accurately mapped back to the corresponding channel positions in the standard YCbCr color space. Specifically, a new YCbCr image structure is created, which has the same dimensions (height H, width W) as the original endoscopic single-frame video image. For each pixel position k in the image, the calculated Cb fused,k Assign the value to the Cb channel of the position in the new YCbCr image, and change its Cr fused,k The value is assigned to the Cr channel of the position in the new YCbCr image. At the same time, it is also necessary to combine the numerical matrix of the original brightness component Y channel separated and saved in the initial step "Based on a single frame of endoscopic video image, convert the image from RGB space to YCbCr space..." (This matrix contains the original brightness information of each pixel. For example, for pixel k, its original brightness value is Y orig,k, for example, Y orig,k =161), the original brightness component Y channel value matrix is ​​directly used for the Y channel of the new YCbCr image, that is, the Y channel value of pixel k in the new YCbCr image is equal to Y orig,k , by filling the newly filled Y channel (containing Y orig,k value), Cb channel (including Cb fused,k Value) and Cr channel (including Cr fused,k The three components (value) are recombined at each pixel position to reconstruct a complete YCbCr color space image. The final YCbCr image is the lesion-specific enhanced pseudo-color image.

[0147] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A real-time enhancement method for endoscopic video images, characterized in that: The following steps are involved: Based on a single-frame endoscopic video image, the image is converted from RGB space to YCbCr space, and the chrominance components Cb and Cr are separated to obtain the original chrominance component map; Based on the original chromaticity component image, the gradient energy of each pixel in the image is calculated to obtain a gradient energy feature, and based on the gradient energy feature, the gradient energy feature is fused with a preset blood vessel morphology filter response value to establish a fusion feature weight value; Based on the fusion feature weight value, the fusion feature weight value is used as an adjustment coefficient to linearly combine the two channels of the original chromaticity component image to establish a dynamic color subspace, and based on the dynamic color subspace, the pixel value distribution is statistically analyzed and the numerical interval corresponding to the lesion tissue is determined to generate a target enhancement mapping function; Based on the target enhancement mapping function, the pixel values ​​of the original chromaticity component image are applied to perform nonlinear chromaticity stretching and saturation adjustment to obtain an enhanced chromaticity component image. Based on the enhanced chromaticity component image, the enhanced chromaticity component image is merged and recombined with the original chromaticity component image to generate a lesion-specific enhanced pseudo-color image.

2. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for obtaining the original chrominance component image are: Based on the single-frame endoscopic video image, pixel value matrices of the red channel, green channel, and blue channel of the image are sequentially extracted, and the pixel value matrices of the red channel, green channel, and blue channel are combined according to the conversion coefficients of the YCbCr color space to generate value matrices corresponding to the luminance component Y and the chrominance components Cb and Cr, thereby generating a YCbCr color space image; Based on the YCbCr color space image, extract the pixel value matrix corresponding to the chrominance component Cb and the pixel value matrix corresponding to the chrominance component Cr according to the channel position index, store the two sets of pixel value matrices in a two-dimensional array in synchronization with the original image resolution, and generate a chrominance channel matrix set; Based on the chromaticity channel matrix set, a set of two-dimensional image frames with the same size as the original image is constructed, and the chromaticity component Cb pixel values ​​and the chromaticity component Cr pixel values ​​are mapped to two grayscale image channels respectively. The two grayscale channel images are combined into a pseudo-color image structure to generate the original chromaticity component image.

3. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for obtaining the gradient energy feature are: Based on the original chromaticity component image, the grayscale intensity values ​​of the chromaticity component Cb and the chromaticity component Cr are extracted for each pixel point in turn, and the grayscale intensity matrices of the chromaticity component Cb and the chromaticity component Cr are established respectively to obtain the chromaticity component grayscale intensity matrix; Based on the chrominance component grayscale intensity matrix, for each pixel position in the matrix, respectively, calculating the intensity change difference in the horizontal direction and the vertical direction, taking the absolute value of the horizontal and vertical differences and summing them, to obtain a gradient difference value matrix for each pixel position; Based on the gradient difference value matrix, each gradient difference value in the matrix is ​​normalized, and the processed gradient difference values ​​are averaged in a local window to form a gradient energy feature.

4. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for obtaining the fusion feature weight value are: Traverse each pixel in the gradient energy feature map, extract the gradient energy value of the pixel point according to the pixel index position, and extract the gradient energy values ​​of all adjacent pixels in its 3×3 neighborhood range, calculate the average value of the gradient energy values ​​of the adjacent pixels, and construct the neighborhood gradient energy mean corresponding to the pixel position to form a gradient and neighborhood energy parameter pair; According to the gradient and neighborhood energy parameter pair of each pixel, the pixel value of the Cb channel and the pixel value of the Cr channel of the pixel in the original chrominance component image are simultaneously extracted, the absolute value of the pixel difference between the two channels is calculated, and the response intensity value of the pixel in the vascular morphology filter response image is extracted, and the gradient energy value, the neighborhood energy mean, the Cb channel value, the Cr channel value and the filter response value are used to form a pixel-level fusion calculation input set; Based on the pixel-level fusion calculation input set, a fusion feature weight value is calculated.

5. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for obtaining the dynamic color subspace are: Traversing all pixel positions in the original chrominance component image, extracting the pixel value of each pixel in the Cb channel and Cr channel in turn, and constructing the original chrominance channel value matrix; Calculating fused color channel values ​​based on the original chromaticity channel value matrix; Based on the fused color channel values, the fused color channel values ​​of all pixel positions are assembled into a two-dimensional pixel array image consistent with the original image structure, and after the combination, the minimum-maximum value normalization is uniformly performed and stretched to the display range, and the normalization result is used as the dynamic color subspace.

6. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for obtaining the target enhancement mapping function are: Traversing all pixel values ​​in the dynamic color subspace, counting the frequency of occurrence of all pixel grayscale values ​​in the range of 0 to 255, constructing a pixel grayscale histogram and calculating the cumulative distribution function, dividing the cumulative distribution function into 20 segments at equal percentile intervals, determining whether the maximum frequency in each segment is higher than 1.5 times the global average frequency, screening continuous segments that meet the conditions and extracting the starting pixel value and the ending pixel value, and generating a candidate lesion response interval boundary group; According to the lesion response candidate interval boundary group, a continuous boundary pair with the largest pixel frequency area is selected as a target response interval, and the lower boundary value of the target response interval is marked as a pixel value response lower limit, and the upper boundary value of the interval is marked as a pixel value response upper limit, to obtain a pixel value response boundary pair; Based on the pixel value response boundary pairs, a target enhancement mapping function expression is constructed.

7. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps of obtaining the enhanced chrominance component image are: Based on the target enhancement mapping function, the pixel values ​​of each pixel in the Cb channel and the Cr channel of the original chrominance component image are extracted one by one, and the pixel values ​​are respectively substituted into the target enhancement mapping function for mapping, so as to obtain the mapped chrominance enhancement value of each pixel and generate a chrominance enhancement value matrix; Based on the chroma enhancement value matrix, nonlinear stretching is performed on the enhanced chroma values ​​of the Cb channel and the Cr channel on a pixel-by-pixel basis, and the processed Cb channel and Cr channel enhancement values ​​are stored in corresponding new channel positions, thereby generating a nonlinear chroma enhancement channel matrix; Based on the nonlinear chroma enhancement channel matrix, all enhanced Cb channel and Cr channel pixel values ​​are traversed, and pixel values ​​exceeding the display range are truncated to a valid range to generate an enhanced chroma component map.

8. The real-time enhancement method for endoscopic video images according to claim 1, characterized in that: The steps for acquiring the lesion-specific enhanced pseudo-color image are as follows: Based on the enhanced chrominance component image, the enhanced pixel values ​​of the Cb channel and the Cr channel in the enhanced chrominance component image are extracted pixel by pixel, and a one-to-one correspondence is established with the pixel values ​​of the Cb channel and the Cr channel at corresponding positions in the original chrominance component image to form enhanced and original chrominance value pixel pairs; Based on the enhanced and original chromaticity value pixel pairs, a weighted combination of pixel values ​​is performed on each pixel position, and the enhanced chromaticity value is used as the main component to fuse the original chromaticity value of the corresponding position, highlight the characteristics of the lesion tissue, and form a fused chromaticity channel pixel value; Based on the fused chromaticity channel pixel values, the fused Cb channel and Cr channel pixel values ​​are respectively mapped back to the corresponding channel positions in the YCbCr space, and combined with the Y channel value of the luminance component of the original chromaticity component image, a complete YCbCr pseudo-color space image is reconstructed to obtain a lesion-specific enhanced pseudo-color image.

Images