A fluorescence coloring processing method for a single-chip endoscope camera system

Abstract

The present invention relates to the technical field of endoscope image processing, and in particular to a fluorescence coloring processing method of a single-chip endoscope camera system. The method comprises the following steps: performing image processing on an original fluorescence image to obtain an overlap difference value, comparing the overlap difference value with a threshold coefficient, and obtaining a signal image according to the comparison result. The signal image and a reference image are subjected to full-screen pixel comparison to obtain color difference points, extracting the color difference points to form a color difference image, extracting color difference values ​​and brightness curves in the color difference image, calculating RGB values ​​of pixel points, color difference values ​​and characteristic values ​​according to the color difference values ​​and brightness curves of the color difference image to obtain replacement pixel values, and then replacing the pixel points with the replacement pixel values, thereby re-coloring the pixel points in the color difference image to obtain a fluorescence coloring image. The present invention performs color conversion processing only on a part of the color difference image, and does not generate image fusion and overlapping deviations, thereby improving the accuracy of image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscope image processing, and in particular to a fluorescence coloring processing method for a single-chip endoscope camera system. Background Art

[0002] A medical endoscope is an optical instrument that is mainly used for the examination, auxiliary treatment and non-open intracavitary surgery of clinical diseases. It is an indispensable medical instrument for hospitals to conduct accurate diagnosis and treatment.

[0003] Prior art CN114972106A discloses an image processing method, device, image processing equipment and endoscope for an endoscope, the method comprising: obtaining a red narrow-band image, a green narrow-band image and a blue narrow-band image respectively collected by at least two narrow-band camera modules of the endoscope; fusing the red narrow-band image, the green narrow-band image and the blue narrow-band image collected by each narrow-band camera module to obtain a color fused image corresponding to each narrow-band camera module; performing image enhancement processing on each color fused image, and outputting a standard display image corresponding to each narrow-band camera module;

[0004] However, the existing technology uses multiple chips (usually 2 or 4) and chips of different levels (size, pixel size) will have a hysteresis effect on the image frame rate and speed during image fusion, registration and other calculation processes. In addition, in the process of secondary fusion of the intercepted fluorescent image and white light, object matching, overlapping offset and other phenomena are likely to occur, thereby reducing the accuracy of the processed image. Summary of the invention

[0005] The purpose of the present invention is to solve the problems in the background technology and to propose a fluorescence coloring processing method for a single-chip endoscope camera system.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A fluorescence coloring processing method for a single-chip endoscope camera system, the method specifically comprising the following steps:

[0008] Step 1: respectively obtain a white light image and an original fluorescence image, first use the white light image as a reference image, set a control area in the original fluorescence image, set a reference area in the reference image according to the position of the control area in the original fluorescence image, then perform image processing on the original fluorescence image, calculate the image information of the processed control area and the image information of the reference area to obtain a coincidence difference, then compare the coincidence difference with a threshold coefficient, and obtain a signal image according to the comparison result, perform pixel comparison of the signal image with the reference image in the entire screen, obtain color difference points according to the comparison result, extract the color difference points, and form a color difference image;

[0009] Step 2: extract the color difference value and brightness curve in the color difference image, obtain the area of ​​the standard image in the reference image through the position of the color difference image in the original fluorescence image, calculate the color difference value between the pixel points in the color difference image and the pixel points in the standard image, and obtain the color difference value of the color difference image, and then use the brightness value in the color difference image as the Y axis and the brightness value in the standard image as the X axis to obtain the brightness curve, and then fit the brightness curve to obtain the linear algebraic expression of the brightness curve;

[0010] Step 3: According to the color difference value and brightness curve of the color difference image, first obtain the characteristic relationship of the brightness curve, then randomly select a pixel point in the color difference image, calculate the RGB value of the pixel point with the color difference value and the characteristic value, obtain the replacement pixel value according to the calculation result, and then replace the pixel point with the replacement pixel value, and then re-color the pixel point in the color difference image to obtain a fluorescent color image;

[0011] Step 4: Replace the original fluorescent image with the fluorescent colored image to obtain the fluorescent processed image.

[0012] As a further solution of the present invention, a method for obtaining the overlap difference is:

[0013] Setting the detection area, selecting several detection areas in the original fluorescence image and marking them as control areas, the area and shape of the detection area are both fixed values;

[0014] Overlapping the original fluorescent image with the reference image, and marking the position in the reference image that overlaps with the control area as the reference area according to the position of the control area in the original fluorescent image;

[0015] The original fluorescence image is processed, and image information of the control area in the processed original fluorescence image is extracted, and the image information includes brightness, contrast and regional RGB value;

[0016] use The overlap difference Cz is obtained, Ld and Dd are the brightness and contrast in the control area d, Lz and Dz are the brightness and contrast in the reference area, rd, gd, and bd are the regional RGB values ​​in the control area d, R, G, and B are the channel values ​​of RGB in the reference area, α1, α2, and α3 are all proportional coefficients, and α1+α2+α3=1.

[0017] As a further solution of the present invention, the regional RGB value is the average value obtained after the RGB value of each pixel in the control area is averaged. The specific method for obtaining the regional RGB value is:

[0018] Traverse the pixels in the control area, extract the R, G, and B channel values ​​of each pixel, then sum the R, G, and B values, and then divide them by the number of pixels in the control area, and use the obtained value as the regional RGB value of the control area.

[0019] As a further solution of the present invention, a method for obtaining a color difference image is as follows:

[0020] Compare the coincidence difference Cz with the coincidence threshold coefficient X1. If the coincidence difference Cz is less than or equal to the coincidence threshold coefficient X1, a termination signal is generated for the image processing process of the original fluorescence image, and the image processing is terminated. At this time, the original fluorescence image after the image processing is completed is marked as a signal image. Otherwise, the image processing of the original fluorescence image will continue until a termination signal is generated.

[0021] Set the RGB difference interval [0, Cq×(1+e%)], where e is the error threshold. Select any pixel point in the signal image as the target point, find the pixel point at the corresponding position in the reference image according to the position of the target point, and mark this pixel point as the reference point;

[0022] use Get the dot distance value Dc, where c represents the pixel in the signal image, rc, gc and bc are the RGB channel values ​​of the pixel c in the signal image, Rc, Gc and Bc are the RGB channel values ​​of the reference point corresponding to the pixel c, and λ1 is the proportional coefficient;

[0023] The point distance value Dc is judged according to the RGB difference interval. When Dc∈[0, Cq×(1+e%)], the target point is marked as a common point. Otherwise, the target point is marked as a color difference point. All the pixel points in the signal image are processed in turn, so that the pixel points in the signal image are divided into common points and color difference points.

[0024] The color difference points in the signal image are traversed, the area where the color difference points are located is extracted, and a color difference image is formed.

[0025] As a further solution of the present invention, a method for obtaining a color difference value in a color difference image includes:

[0026] According to the position of the color difference image in the original fluorescence image, the area at the same position between the reference image and the color difference image is extracted and marked as the standard area;

[0027] Traverse the pixels in the color difference image and calculate the color difference between the color difference image and the standard image. The color difference value of the pixel is obtained, L1, C1 and H1 are the brightness, chromaticity and hue in the color difference image, Lb, Cb and Hb are the brightness, chromaticity and hue in the standard image, and θ is the weight coefficient.

[0028] As a further solution of the present invention, a method for obtaining a brightness curve includes:

[0029] The brightness value of the standard image is taken as the X-axis, the brightness value in the color difference image is taken as the Y-axis, a plane coordinate system is set, and the brightness values ​​in the color difference image and the standard image are marked in the plane coordinate system respectively, and then the brightness curve and its linear algebraic expression La=f(Lba) are obtained, where Lba is the brightness value in the standard image and f(*) is the function expression.

[0030] As a further solution of the present invention, a method for obtaining a fluorescent color image includes:

[0031] According to the linear algebraic expression of the brightness curve, the characteristic relationship of the brightness curve is obtained by using TL=La′, where TL is the result of the first derivation of La;

[0032] Select any pixel point and extract the three channel values ​​of RGB of this pixel point. Get the replacement pixel value (Rt, Gt, Bt), ri, gi, bi are the values ​​of the RGB channels of the pixel i in the color difference image, TL is the characteristic value of the brightness of this pixel, K1, K2 and K3 are the influence coefficients of R, G and B respectively;

[0033] Each pixel is replaced according to the replacement pixel value (Rt, Gt, Bt), and then the pixel in the color difference image is recolored to obtain a fluorescent colored image.

[0034] As a further solution of the present invention, only a single CMOS chip is used in the endoscope camera system to process the image.

[0035] Compared with the existing technology, the advantages of this method are:

[0036] The present invention sets a reference image, compares the reference image with the original fluorescent image, separates the color difference image, then extracts and calculates the color difference value and the brightness curve in the color difference image, assigns color to the pixels in the color difference image, and generates a display color. The color conversion process is only performed on a part of the color difference image, and image fusion and overlapping deviation will not be generated, and the frame rate and rate of the image will not be affected. On the one hand, the accuracy of image processing is improved, and on the other hand, the rate and efficiency of image processing are improved.

[0037] The present invention uses a single chip for processing to unify the size and pixel standards in the image, thereby reducing image errors caused by different standards for the size and pixel in the image due to different chip models when the image is processed by multiple chips. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the method flow structure of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0040] Embodiment 1, refer to Figure 1 , a fluorescence coloring processing method for a single-chip endoscope camera system, the method specifically comprises the following steps:

[0041] Step 1: When the endoscope is detecting in the body, a white light image is first captured as a reference image, and then a tissue injection image irradiated by near-infrared light is obtained, and the tissue injection image is marked as an original fluorescence image. In this embodiment, the tissue injection image is obtained by staining tissue cells with indocyanine green ICG or methylene blue (methylene blue), and then irradiated by near-infrared light in the endoscope to obtain the tissue image and form an original fluorescence image;

[0042] The reference image is compared with the original fluorescence image, and a color difference image is obtained according to the comparison result. The specific method for obtaining the color difference image is as follows:

[0043] S1: setting a detection region, wherein the area and shape of the detection region are fixed values, and the specific shape and area of ​​the detection region are set by a person skilled in the art, and then selecting a number of detection regions in the original fluorescence image and marking them as control regions;

[0044] It should be further explained that the control area is the area in the original fluorescence image that is not fluorescently labeled;

[0045] S2: then overlapping the original fluorescence image with the reference image, that is, overlapping the image frame in the original fluorescence image with the image frame in the reference image in sequence, and at the same time, according to the position of the reference area in the original fluorescence image, marking the position in the reference image that overlaps with the reference area as the reference area;

[0046] S3: Then the original fluorescent image is processed, and the image information of the control area in the processed original fluorescent image is extracted, wherein the image processing includes denoising, contrast enhancement and other processing, and the image information includes brightness, contrast and regional RGB value. It should be further explained that the RBG value here is the mean value obtained by averaging the RGB value of each pixel in the control area, and the mean value is marked as the regional RGB value. The specific method for obtaining the regional RGB value includes: traversing the pixels in the control area, respectively extracting the values ​​of the three channels of R, G, and B of each pixel, and then summing the three values ​​of R, G, and B respectively, and then dividing by the number of pixels in the control area, and then using the obtained value as the regional RGB value of the control area;

[0047] S4: Extract the image information of the reference area in the reference image and use the formula The overlap difference Cz is obtained, Ld and Dd are the brightness and contrast in the control area d, Lz and Dz are the brightness and contrast in the reference area, rd, gd, bd are the regional RGB values ​​in the control area d, R, G, B are the RGB channel values ​​in the reference area, α1, α2 and α3 are all proportional coefficients, and α1+α2+α3=1;

[0048] The overlap difference Cz is first compared with the overlap threshold coefficient X1. If the overlap difference Cz is less than or equal to the overlap threshold coefficient X1, a termination signal is generated for the image processing process of the original fluorescence image, and the image processing is terminated. At this time, the original fluorescence image after the image processing is completed is marked as a signal image. On the contrary, if the overlap difference Cz is greater than the overlap threshold coefficient X1, the image processing of the original fluorescence image will continue until a termination signal is generated. The overlap threshold coefficient X1 is set by a person skilled in the art.

[0049] S5: Full-screen comparison of the signal image and the reference image to extract the color difference area. The specific method of extracting the color difference area includes:

[0050] According to the overlap difference Cz calculated in step S4, the RGB difference interval is set to [0, Cq×(1+e%)], where e is the error threshold. In this embodiment, the difference interval is set to [0, Cq×(1-5%)];

[0051] Select a pixel point in the signal image and mark it as the target point. According to the position of the target point, find the pixel point at the corresponding position in the reference image and mark this pixel point as the reference point. Extract the RGB values ​​of the target point and the reference point respectively, and use the formula The dot distance value Dc is obtained, where c represents a pixel in the signal image, rc, gc and bc are respectively the RGB channel values ​​of the pixel c in the signal image, Rc, Gc and Bc are respectively the RGB channel values ​​of the reference point corresponding to the pixel c, and λ1 is a proportional coefficient, the specific value of which is obtained by the staff in this field through a large number of experiments;

[0052] The point distance value Dc is judged according to the RGB difference interval. When Dc∈[0, Cq×(1+e%)], the target point is marked as a common point. Otherwise, When , the target point is marked as a color difference point, and all the pixel points in the signal image are processed in sequence, so that the pixel points in the signal image are divided into common points and color difference points;

[0053] Traverse the color difference points in the signal image, extract the area where the color difference points are located, and form a color difference image;

[0054] Step 2: Split the color difference image to obtain real-time image information, which includes color difference values ​​and brightness curves. The specific method for splitting the color difference image is as follows:

[0055] SS1: According to the position of the color difference image in the original fluorescence image, the area at the same position in the reference image and the color difference image is extracted and marked as the standard area. The specific standard area is the area composed of the position points of the color difference points in the reference image in step S5. Therefore, the color difference image here completely overlaps with the standard image;

[0056] SS2: Then traverse the pixels in the color difference image, calculate the color difference value between the color difference image and the standard image, and obtain the color difference value of each pixel in each color difference image. The specific color difference value calculation formula is: L1, C1 and H1 are the brightness, chromaticity and hue in the color difference image, respectively; Lb, Cb and Hb are the brightness, chromaticity and hue in the standard image, respectively; θ is a weight coefficient, and its specific value is set by those skilled in the art;

[0057] It should be further explained that the value calculated when extracting the color difference point in step 1 is a value processed by the corresponding parameters in the HSB color model and the RGB color model, and the color difference value here is the corresponding parameter value extracted under the HSB color model, and in this embodiment, the HSB color model and the RGB color model are converted to each other, and the specific conversion method is a prior art, which will not be repeated here;

[0058] SS3: Then, according to the brightness value in the color difference image and the brightness value in the standard image, the brightness value of the standard image is used as the X-axis and the brightness value in the color difference image is used as the Y-axis, a plane coordinate system is set, and the brightness values ​​in the color difference image and the standard image are marked in the plane coordinate system respectively, so as to obtain a brightness curve, and then the brightness curve is calculated using a mathematical model to obtain a linear algebraic expression of the brightness curve La=f(Lba), where Lba is the brightness value in the standard image, and f(*) is a function expression. The mathematical model in this embodiment is calculated using Matlab software;

[0059] Step 3: Then, the color difference image is colorized according to the real-time image information of the color difference image, and a fluorescent color image is generated. The processing method of the fluorescent color image includes:

[0060] According to the linear algebraic expression of the brightness curve, the characteristic relationship of the brightness curve is obtained by using the formula TL=La′, where TL is the result of the first derivation of La;

[0061] Then select any pixel point and extract the three channel values ​​of RGB of this pixel point based on the formula Obtain the replacement pixel value (Rt, Gt, Bt), ri, gi, bi are the values ​​of the RGB channels of the pixel i in the color difference image, TL is the characteristic value of the brightness of the pixel, K1, K2 and K3 are the influence coefficients of R, G, B, respectively, and the specific values ​​of K1, K2 and K3 are set by those skilled in the art;

[0062] Then, each pixel is replaced according to the replacement pixel value (Rt, Gt, Bt), and the pixel in the color difference image is recolored to obtain a fluorescent color image;

[0063] It should be further explained that in order to avoid the color assignment being inconsistent with the normal image color, in the process of separating the color difference image, an energy color signal with a half-width length of 20nm wavelength in the color difference signal is usually selected.

[0064] Step 4: Replace the original fluorescent image with the fluorescent color image to obtain a fluorescent processed image, and then transmit the fluorescent processed image to a display terminal, and a professional technician views the fluorescent processed image in the display terminal;

[0065] Embodiment 2. This embodiment is based on Embodiment 1, and the difference from Embodiment 1 is that this embodiment also includes: This embodiment is based on a single CMOS chip, and adopts an image color comparison method. There is no need to use multiple chips to realize the extraction, coloring and fusion processing of fluorescent images, which can simplify the system architecture and reduce the number of components, thereby making the entire device more compact and lightweight.

[0066] Embodiment 3: This embodiment is based on Embodiment 1 and Embodiment 2, and is used to integrate Embodiment 1 and Embodiment 2 and implement them.

[0067] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A fluorescence coloring processing method for a single-chip endoscope camera system, characterized in that: The method specifically comprises the following steps: Step 1: Obtain a white light image and an original fluorescence image respectively, first use the white light image as a reference image, set a control area in the original fluorescence image, set a reference area in the reference image according to the position of the control area in the original fluorescence image, then perform image processing on the original fluorescence image, calculate the image information of the processed control area and the image information of the reference area, and obtain the overlap difference. The specific method for obtaining the overlap difference is: Setting the detection area, selecting several detection areas in the original fluorescence image and marking them as control areas, the area and shape of the detection area are both fixed values; Overlapping the original fluorescent image with the reference image, and marking the position in the reference image that overlaps with the control area as the reference area according to the position of the control area in the original fluorescent image; The original fluorescence image is processed, and image information of the control area in the processed original fluorescence image is extracted, and the image information includes brightness, contrast and regional RGB value; use The overlap difference Cz is obtained, Ld and Dd are the brightness and contrast in the control area d, Lz and Dz are the brightness and contrast in the reference area, rd, gd, and bd are the regional RGB values ​​in the control area d, R, G, and B are the channel values ​​of RGB in the reference area, α1, α2, and α3 are all proportional coefficients, and α1+α2+α3=1; Then, the overlap difference is compared with the threshold coefficient, and the signal image is obtained according to the comparison result. The signal image and the reference image are compared in full-screen pixels, and the color difference points are obtained according to the comparison result. The color difference points are extracted and the color difference image is formed. The specific method for forming the color difference image is as follows: Compare the coincidence difference Cz with the coincidence threshold coefficient X1. If the coincidence difference Cz is less than or equal to the coincidence threshold coefficient X1, a termination signal is generated for the image processing process of the original fluorescence image, and the image processing is terminated. At this time, the original fluorescence image after the image processing is completed is marked as a signal image. Otherwise, the image processing of the original fluorescence image will continue until a termination signal is generated. Set the RGB difference interval [0, Cq×(1+e%)], where e is the error threshold. Select any pixel point in the signal image as the target point, find the pixel point at the corresponding position in the reference image according to the position of the target point, and mark this pixel point as the reference point; use Get the dot distance value Dc, where c represents the pixel in the signal image, rc, gc and bc are the RGB channel values ​​of the pixel c in the signal image, Rc, Gc and Bc are the RGB channel values ​​of the reference point corresponding to the pixel c, and λ1 is the proportional coefficient; The point distance value Dc is judged according to the RGB difference interval. When Dc∈[0, Cq×(1+e%)], the target point is marked as a common point. Otherwise, the target point is marked as a color difference point. All the pixel points in the signal image are processed in turn, so that the pixel points in the signal image are divided into common points and color difference points. Traverse the color difference points in the signal image, extract the area where the color difference points are located, and form a color difference image; Step 2: extract the color difference value and brightness curve in the color difference image, obtain the area of ​​the standard image in the reference image through the position of the color difference image in the original fluorescent image, calculate the color difference value between the pixel points in the color difference image and the pixel points in the standard image, and obtain the color difference value of the color difference image. Furthermore, the specific method for obtaining the color difference value includes: According to the position of the color difference image in the original fluorescence image, the area at the same position between the reference image and the color difference image is extracted and marked as the standard area; Traverse the pixels in the color difference image and calculate the color difference between the color difference image and the standard image. Get the color difference value of the pixel point, L1, C1 and H1 are the brightness, chroma and hue in the color difference image, Lb, Cb and Hb are the brightness, chroma and hue in the standard image, and θ is the weight coefficient; Then, the brightness value in the color difference image is used as the Y axis, and the brightness value in the standard image is used as the X axis to obtain a brightness curve. Then, the brightness curve is fitted to obtain a linear algebraic expression of the brightness curve. Specifically, the method for obtaining the brightness curve includes: The brightness value of the standard image is taken as the X-axis, and the brightness value in the color difference image is taken as the Y-axis, a plane coordinate system is set, and the brightness values ​​in the color difference image and the standard image are marked in the plane coordinate system respectively, and then the brightness curve and its linear algebraic expression La=f(Lba) are obtained, where Lba is the brightness value in the standard image and f(*) is the function expression; Step 3: According to the color difference value and brightness curve of the color difference image, first obtain the characteristic relationship of the brightness curve, then randomly select a pixel point in the color difference image, calculate the RGB value of the pixel point with the color difference value and the characteristic value, obtain the replacement pixel value according to the calculation result, and then replace the pixel point with the replacement pixel value, and then re-color the pixel point in the color difference image to obtain a fluorescent color image; Step 4: Replace the original fluorescent image with the fluorescent colored image to obtain the fluorescent processed image.

2. The fluorescence coloring processing method of a single-chip endoscope camera system according to claim 1, characterized in that: The regional RGB value is the average value obtained after the RGB value of each pixel in the control area is averaged. The specific method for obtaining the regional RGB value is: Traverse the pixels in the control area, extract the R, G, and B channel values ​​of each pixel, then sum the R, G, and B values, and then divide them by the number of pixels in the control area, and use the obtained value as the regional RGB value of the control area.

3. The fluorescence coloring processing method of a single-chip endoscope camera system according to claim 1, characterized in that: The method for obtaining the fluorescent color image includes: According to the linear algebraic expression of the brightness curve, the characteristic relationship of the brightness curve is obtained by using TL=La, where TL is the result of the first derivation of La; Select any pixel point and extract the three channel values ​​of RGB of this pixel point. Get the replacement pixel value (Rt, Gt, Bt), ri, gi, bi are the values ​​of the RGB channels of the pixel i in the color difference image, TL is the characteristic value of the brightness of this pixel, K1, K2 and K3 are the influence coefficients of R, G and B respectively; Each pixel is replaced according to the replacement pixel value (Rt, Gt, Bt), and then the pixel in the color difference image is recolored to obtain a fluorescent colored image.

4. The fluorescence coloring processing method of a single-chip endoscope camera system according to claim 1, characterized in that: It also includes the use of only a single CMOS chip to process images in the endoscope camera system.

Images