Image correction method for electronic endoscope system
By calculating and storing the correction parameters of the reference endoscope and processor, the processing load problem during color information correction of the electronic endoscope system is solved, and more efficient image correction is achieved.
Patent Information
- Application Number
- CN202380038911.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-07-28
- Filing Date
- 2023-07-03
- Publication Date
- 2024-12-06
AI Technical Summary
The existing electronic endoscope system has too much processing load when correcting color information, requiring a large amount of decimal operations and large-capacity storage, resulting in inefficiency.
By calculating and storing the first and second correction parameters using the reference endoscope and the reference processor, the processing load during diagnosis is reduced.
In the case of combining any electronic endoscope and processor, the processing load during color information correction is significantly reduced and the system efficiency is improved.
Smart Images

Figure CN119095530A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an image correction method of an electronic endoscope system configured to acquire and process a captured image of living tissue. Background Art
[0002] An electronic endoscope system is used to observe and treat living tissue inside the human body. The electronic endoscope system includes an electronic endoscope (electronic endoscope), in which the electronic endoscope uses an imaging element to capture images of living tissue and transmits the captured images to the processor, and the processor (electronic endoscope processor) emits illumination light and processes the signals of the captured images to generate images for display.
[0003] The imaging element and the light source of the processor of an electronic endoscope (electronic endoscope) have individual differences in spectral characteristics. Therefore, when the electronic endoscope and the processor are connected, the color information of the image of the living tissue displayed by them will be biased. In the past, some people have proposed to correct the color information error (machine error) between the individual processors and endoscopes.
[0004] For example, it is described in Japanese Patent Gazette No. 5544219 that, in order to correct the machine errors of the processor and the endoscope, a chromaticity correction table is created for performing chromaticity correction on the image signal and adjusting the image signal to be consistent with a reference specific hue based on the spectral characteristics of the color filter possessed by the image pickup element of the endoscope, wherein the chromaticity correction corresponds to the difference between the spectral characteristics of the standard light predetermined as the illumination light in the processor and the spectral characteristics of the actual illumination light. Summary of the invention
[0005] Technical problem to be solved by the invention
[0006] However, the correction method described in Japanese Patent No. 5544219 needs to calculate conversion coefficients for converting a plurality of spectral characteristics into reference spectral characteristics in order to create a chromaticity correction table, and a large amount of decimal operations (such as floating-point operations) are required to obtain the product of each conversion coefficient, and a large-capacity memory is required to store the data of each spectral characteristic. That is, the correction method described in Japanese Patent No. 5544219 has a problem that the processing load is too large when correcting the color information error between the processor and the individual endoscopes.
[0007] Therefore, an object of the present invention is to reduce the processing load when color information correction is performed when any electronic endoscope and a processor for the electronic endoscope are used in combination.
[0008] Solutions for solving technical problems
[0009] One embodiment of the present invention is a correction method for correcting a camera image in an electronic endoscope system, in which an electronic endoscope that uses a camera element to photograph living tissue and an electronic endoscope processor that performs signal processing on the camera image photographed by the camera element and creates an image for display are connected.
[0010] The correction method includes:
[0011] The step of calculating a first calibration parameter corresponding to the processor to be calibrated, i.e., the processor for the electronic endoscope to be calibrated, by using a reference endoscope, i.e., the electronic endoscope serving as a calibration reference, and a reference processor, i.e., the processor for the electronic endoscope to be calibrated;
[0012] The step of storing the calculated first correction parameter in the processor to be corrected;
[0013] The step of calculating a second calibration parameter corresponding to the endoscope to be calibrated, i.e., the electronic endoscope to be calibrated, by using the reference endoscope and the reference processor;
[0014] The step of storing the calculated second calibration parameters in the endoscope to be calibrated; and
[0015] The step of correcting an image captured by an imaging element of the endoscope to be calibrated by the processor to be calibrated using the first calibration parameter and the second calibration parameter when the endoscope to be calibrated is connected to the processor to be calibrated.
[0016] The step of calculating the first correction parameter may include the step of calculating a parameter for converting the first pixel value into the second pixel value based on the first pixel value and the second pixel value and using the parameter as the first correction parameter.
[0017] The first pixel value is a pixel value of a photographed image obtained when the reference endoscope and the processor to be calibrated are connected and a specific color index is photographed.
[0018] The second pixel value is a pixel value of a captured image obtained when the reference endoscope and the reference processor are connected and the color index is captured.
[0019] The step of calculating the second correction parameter may include the step of calculating a parameter for converting the third pixel value into the fourth pixel value based on the third pixel value and the fourth pixel value and using the parameter as the second correction parameter.
[0020] The third pixel value is a pixel value of a camera image obtained when the endoscope to be calibrated and the reference processor are connected and a specific color index is captured.
[0021] The fourth pixel value is a pixel value of a captured image obtained when the reference endoscope and the reference processor are connected and the color index is captured.
[0022] The correction method may further include the steps of determining a first feature quantity, i.e., a feature quantity obtained from the first pixel value, and a second feature quantity, i.e., a feature quantity obtained from the second pixel value, and calculating a first feature quantity correction parameter for converting the first feature quantity into the second feature quantity;
[0023] A step of determining a third feature quantity, i.e., a feature quantity obtained from the third pixel value, and a fourth feature quantity, i.e., a feature quantity obtained from the fourth pixel value, and calculating a second feature quantity correction parameter for converting the third feature quantity into the fourth feature quantity; and
[0024] When the endoscope to be calibrated is connected to the processor to be calibrated, the processor to be calibrated uses the first feature quantity correction parameter and the second feature quantity correction parameter to correct the feature quantity obtained from the pixel value of the camera image captured by the camera element of the endoscope to be calibrated.
[0025] The color index may be determined from a plurality of color indices based on a feature amount obtained from pixel values of a captured image.
[0026] [Effect of the Invention] According to the above-mentioned endoscope system, when any electronic endoscope and electronic endoscope processor are used in combination, the processing load when color information correction is performed can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a schematic system configuration diagram of an endoscope system according to an embodiment.
[0028] Figure 2 This is a block diagram of the main structure of an endoscope system according to one embodiment.
[0029] Figure 3 The present invention is a schematic diagram of each process of an image correction method of an electronic endoscope system according to an embodiment.
[0030] Figure 4 This is a diagram for explaining a calculation process of a processor matrix in an electronic endoscope system according to an embodiment.
[0031] Figure 5 This is a diagram for explaining a calculation process of a mirror matrix in an electronic endoscope system according to an embodiment.
[0032] Figure 6 The present invention is a flowchart of image processing during diagnosis by an electronic endoscope system according to an embodiment.
[0033] Figure 7 This is a diagram for explaining the calculation process of a processor matrix in an electronic endoscope system according to another embodiment.
[0034] Figure 8 This is a diagram for explaining the calculation process of the mirror matrix in the electronic endoscope system according to another embodiment. DETAILED DESCRIPTION
[0035] (1) First Implementation Method
[0036] Next, an electronic endoscope system and an image correction method thereof according to an embodiment will be described in detail with reference to the drawings.
[0037] First refer to Figure 1 and Figure 2 An electronic endoscope system 1 according to one embodiment will be described. Figure 1 FIG. 1 is a schematic system configuration diagram of an endoscope system 1 according to an embodiment. Figure 2 This is a block diagram showing a main structure of an endoscope system 1 according to one embodiment.
[0038] like Figure 1 As shown, the electronic endoscope system 1 is a system dedicated to medical use, and includes an electronic scope 10 (an example of an electronic endoscope), a processor 20 (an example of a processor for an electronic endoscope), and a monitor 30 .
[0039] like Figure 1 As shown, the front end of the electronic scope 10 is flexible and is provided with an insertion portion 120 for insertion into the human body. A bending portion 104 is provided near the front end of the insertion portion 120, and the bending portion is bent according to the remote control operation from a manual operation portion 122 connected to the base end of the insertion portion 120. The bending mechanism of the bending portion 104 is a known mechanism assembled in a general endoscope. The bending structure is a structure in which the bending portion 104 is bent by pulling an operation wire, and the operation wire is linked to the rotation operation of the bending operation knob provided in the manual operation portion 122. A solid-state imaging element (hereinafter referred to as an imaging element) 14 (refer to Figure 2 ) The imaging area of the electronic scope 10 is moved by changing the direction of the front end portion 102 by the bending action of the bending portion 104 caused by the rotation operation of the bending operation knob.
[0040] An LCB (Light Carrying Bundle) 11 is arranged over the substantially entire length from the connector portion 10 c to the front end portion 102 of the electronic scope 10 (see Figure 2 The LCB 11 is an optical fiber bundle, and guides the irradiation light supplied from the light source device 25 to the front end portion 102 of the electronic scope 10 .
[0041] The processor 20 processes the image signal of the subject image obtained by imaging the subject through the imaging element 14 of the electronic scope 10 and supplies the processed image signal to the monitor 30 (see Figure 2 ) device.
[0042] like Figure 1 As shown, the processor 20 is provided with a connector portion 20c for connecting to the electronic scope 10. The base end of the electronic scope 10 is provided with a connector portion 10c for connecting to the connector portion 20c of the processor 20. By mechanically connecting the connector portion 10c with the connector portion 20c, the electronic scope 10 is electrically connected to the processor 20, and the light source device 25 is optically connected to the electronic scope 10.
[0043] The information processing device 40 is a device such as a computer device, a tablet terminal, or a smartphone that can perform specific calculations and can be electrically connected to the processor 20 via a cable 200 .
[0044] The information processing device 40 calculates parameters for image correction of the electronic mirror 10 and / or the processor 20, and records the calculated parameters in the electronic mirror 10 and / or the processor 20. Figure 1 In the example shown, although the information processing device 40 is not directly connected to the electronic mirror 10 via a cable, when the electronic mirror 10 is electrically connected to the processor 20, the parameters can be written to the electronic mirror 10 via the processor 20. In addition, if the information processing device 40 and the electronic mirror 10 and the processor 20 can respectively communicate wirelessly, the information processing device 40 can also write the parameters to the electronic mirror 10 and the processor 20 without using the cable 200.
[0045] In one embodiment, as described below, the information processing device 40 is configured to calculate a processor matrix and a mirror matrix and use them as parameters, write the processor matrix into the matrix storage unit 28 of the processor 20, and write the mirror matrix into the matrix storage unit 16 of the electronic mirror 10.
[0046] The timing of calculating and writing the processor matrix and / or the mirror matrix is not limited, but is preferably the timing after the electronic mirror 10 and the processor 20 are manufactured and before they are shipped, for example.
[0047] Reference Figure 2 The processor 20 includes a system controller 21 , an image input processing unit 22 , an image memory 23 , an image output processing unit 24 , a light source device 25 , an operation panel 26 , a color correction operation unit 27 , a matrix storage unit 28 and a condenser lens 29 .
[0048] The electronic scope 10 includes an LCB 11 , a light distribution lens 12 , an objective lens 13 , an imaging element 14 , a driver signal processing circuit 15 , and a matrix storage unit 16 .
[0049] The system controller 21 executes various programs and comprehensively controls the entire electronic endoscope system 1. In addition, the system controller 21 is connected to the operation panel 26. The system controller 21 changes each operation of the electronic endoscope system 1 and the parameters used for each operation according to the instructions of the operator input to the operation panel 26. The system controller 21 outputs a clock pulse for adjusting the operation timing of each part to each circuit in the electronic endoscope system 1.
[0050] The light source device 25 emits illumination light L for illuminating a subject such as living tissue in a body cavity. The illumination light L includes white light, pseudo white light, or special light. According to one embodiment, the light source device 25 preferentially selects any one of a mode of always emitting white light or pseudo white light as the illumination light L and a mode of alternately emitting white light or pseudo white light and special light as the illumination light L, and emits white light, pseudo white light, or special light according to the selected mode. White light is light with a flat spectral intensity distribution in the visible light band, and pseudo white light is light with an uneven spectral intensity distribution and a mixture of lights of multiple bands. Special light is light of a narrow band in the visible light band, such as blue or green. Light in the blue or green band is used when enhancing and observing a specific part in living tissue. The illumination light L emitted from the light source device 25 is focused by the focusing lens 29 to the incident end face of the LCB11 and is incident on the LCB11 of the electronic scope 10.
[0051] In addition, Figure 2 In the illustrated example, the light source device 25 is built into the processor 20 , but the light source device 25 may be configured separately from the processor 20 .
[0052] The illumination light L incident on the LCB 11 propagates in the LCB 11 of the electronic scope 10. The illumination light L propagating in the LCB 11 is emitted from the emission end face of the LCB 11 disposed at the front end portion 102 of the electronic scope 10, and is irradiated to the subject via the light distribution lens 12. The return light from the subject illuminated by the illumination light L from the light distribution lens 12 forms an optical image on the light receiving surface of the imaging element 14 via the objective lens 13.
[0053] The imaging element 14 is a single-plate color CCD (Charge Coupled Device) image sensor with a Bayer-type pixel configuration. The imaging element 14 accumulates the optical image formed on each pixel of the light-receiving surface as an electric charge that varies with the amount of light, and generates R (Red), G (Green), and B (Blue) image signals and outputs them. In addition, the imaging element 14 is not limited to a CCD image sensor, and can also be replaced by a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types of imaging devices. The imaging element 14 can also be an element equipped with a complementary color filter.
[0054] The system controller 21 supplies a clock pulse to the driver signal processing circuit 15 of the electronic mirror 10. The driver signal processing circuit 15 drives and controls the image pickup element 14 in synchronization with the frame rate of the image processed on the processor 20 side according to the clock pulse supplied from the system controller 21. The frame rate is, for example, 1 / 30 second. The driver signal processing circuit 15 performs specific processing including A / D conversion on the image signal input from the image pickup element 14, and outputs the image signal to the image input processing unit 22 of the processor 20.
[0055] The image input processing unit 22 performs specific signal processing such as demosaicing processing and matrix operation on the input image signal. The image input processing unit 22 may also perform specific noise reduction processing.
[0056] The image memory 23 is a memory for buffering the image signal (image) processed by the image input processing unit 22 .
[0057] The image output processing unit 24 processes the images in the image memory 23 one by one, generates screen data for monitor display, converts the generated screen data for monitor display into a specific video format signal, and outputs the converted video format signal to the monitor 30. Thus, the image of the subject is displayed on the display screen of the monitor 30.
[0058] The matrix storage unit 28 of the processor 20 and the matrix storage unit 16 of the electronic mirror 10 may be non-volatile memories such as SSD (Solid State Drive) to store the processor matrix and the mirror matrix respectively. As described above, the processor matrix and the mirror matrix are written by the information processing device 40.
[0059] The color correction operation unit 27 of the processor 20 performs color correction operation on the image buffered in the image memory 23 based on the processor matrix and the mirror matrix. The system controller 21 overwrites the image calculated by the color correction operation unit 27 into the image memory 23. Therefore, the image data for monitor display is generated by the image output processing unit 24 based on the color-corrected image.
[0060] Next, we will refer to Figures 3 to 6 An image correction method of an electronic endoscope system according to an embodiment will be described.
[0061] Figure 3 FIG. 1 is a schematic diagram of each process of an image correction method for an electronic endoscope system according to an embodiment. Figure 3 As shown, the image correction method includes steps S1 to S5.
[0062] Steps S1 and S2 are steps to be performed on the processor to be calibrated, and can be performed, for example, after the processor is manufactured and before it is shipped.
[0063] Steps S3 and S4 are steps to be performed on the electronic mirror to be calibrated, and can be performed, for example, after the electronic mirror is manufactured and before it is shipped.
[0064] Step S5 is a step executed for each image in units of frames during diagnosis.
[0065] (Process S1)
[0066] Step S1 is a step of calculating a processor matrix (an example of first calibration parameters) corresponding to the processor to be calibrated 20_Tar, that is, the processor to be calibrated, using the reference electronic mirror 10_Ref, that is, the electronic mirror serving as a calibration reference, and the reference processor 20_Ref, that is, the processor serving as the calibration reference.
[0067] The reference electronic mirror 10_Ref is the main electronic mirror 10 that is the target of color correction. For example, the image sensor included in a general electronic mirror has individual differences in spectral characteristics, and the reference electronic mirror 10_Ref is an electronic mirror 10 including an image sensor 14 whose spectral characteristics are median values.
[0068] The reference processor 20_Ref is the main processor 20 that is the target of color correction. For example, the illumination light of the light source device provided by a general processor has individual differences in spectral characteristics, while the reference processor 20_Ref is a processor 20 provided with a light source device 25 whose spectral characteristics are central values.
[0069] Figure 4 The procedure of step S1 is conceptually shown.
[0070] First, the reference electronic mirror 10_Ref is connected to the processor to be corrected 20_Tar, and then Figure 4 The information processing device 40 not shown is connected to the processor to be corrected 20_Tar. The front end of the reference electronic mirror 10_Ref faces the color index RC as a reference. The color index RC is an index of the color used as a reference when displaying the correction. The color displayed on the color index RC is appropriately determined according to the subject, but when imaging living tissue, it is preferably a reddish color that matches the color of the affected part. A plurality of color indices RC each containing different colors may also be used.
[0071] In step S1 , a captured image IMG1 of a color index RC obtained by combining the reference electronic mirror 10_Ref and the processor to be corrected 20_Tar is transmitted to the information processing device 40 .
[0072] Then, similarly, the captured image IMG2 of the color index RC obtained by the combination of the reference electronic mirror 10_Ref and the reference processor 20_Ref is transmitted to the information processing device 40 .
[0073] The information processing device 40 calculates a processor matrix M1 for assigning the camera image IMG1 to the camera image IMG2. When calculating the processor matrix M1, a matrix for converting the pixel value (RGB value; an example of a first pixel value) of a specific pixel in the camera image IMG1 to the pixel value (an example of a second pixel value) of the corresponding pixel in the camera image IMG2 can be obtained, and a matrix for converting the average pixel value (RGB average value) of pixels in the corresponding range of each camera image can also be obtained.
[0074] (Process S2)
[0075] Step S2 is a step of storing the processor matrix calculated in step S1 in the matrix storage unit 28 of the processor to be corrected 20_Tar.
[0076] In step S2 , the information processing device 40 sends the processor matrix M1 calculated in step S1 to the processor 20_Tar to be corrected.
[0077] Steps S1 and S2 are usually performed at the manufacturing site of the processor 20 before the processor 20 is shipped. That is, at least one reference electronic mirror 10_Ref and one reference processor 20_Ref are prepared at the manufacturing site of the processor 20, and the multiple processors 20 produced are sequentially used as the processors to be calibrated 20_Tar, and the processor matrix M1 is calculated and stored for each processor 20. Figure 4In the embodiment, the camera image IMG2 is obtained by the combination of the reference electronic mirror 10_Ref and the reference processor 20_Ref only once, and does not need to be performed sequentially.
[0078] (Process S3)
[0079] Step S3 is a step of calculating a mirror matrix (an example of a second correction parameter) corresponding to the electronic mirror to be corrected 10_Tar, ie, the electronic mirror to be corrected, using the reference electronic mirror 10_Ref and the reference processor 20_Ref.
[0080] Figure 5 The procedure in step S3 is conceptually shown.
[0081] First, the electronic mirror 10_Tar to be corrected is connected to the reference processor 20_Ref, and then Figure 5 The information processing device 40 not shown in the figure is connected to the reference processor 20_Ref. The front end portion of the electronic mirror 10_Tar to be corrected faces the color index RC as a reference.
[0082] In step S3 , a captured image IMG3 of a color index RC obtained by a combination of the electronic mirror 10_Tar to be calibrated and the reference processor 20_Ref is transmitted to the information processing device 40 .
[0083] Then, similarly, the captured image IMG4 of the color index RC obtained by combining the reference electronic mirror 10_Ref and the reference processor 20_Ref is transmitted to the information processing device 40 .
[0084] The information processing device 40 calculates a mirror matrix M2, which is used to assign the camera image IMG3 to the camera image IMG4. When calculating the mirror matrix M2, a matrix for converting the pixel value (an example of the third pixel value) of a specific pixel in the camera image IMG3 to the pixel value (an example of the fourth pixel value) of the corresponding pixel in the camera image IMG4 can be obtained, and a matrix for converting the average pixel value (the average value of RGB) of pixels in the corresponding range of each camera image can also be obtained.
[0085] (Process S4)
[0086] Step S4 is a step of storing the mirror matrix calculated in step S3 in the matrix storage unit 16 of the electronic mirror 10_Tar to be corrected.
[0087] In step S4 , the information processing device 40 sends the mirror matrix M2 calculated in step S3 to the electronic mirror 10_Tar to be corrected via the reference processor 20_Ref.
[0088] Steps S3 and S4 are usually performed at the manufacturing site of the electronic mirror 10 before the electronic mirror 10 is shipped. That is, at least one reference electronic mirror 10_Ref and a reference processor 20_Ref are prepared at the manufacturing site of the electronic mirror 10, and the multiple electronic mirrors 10 produced are used as the electronic mirrors to be calibrated 10_Tar one by one, and the mirror matrix M2 is calculated and stored for each electronic mirror 10. Figure 5 In the embodiment, the camera image IMG4 is obtained by the combination of the reference electronic mirror 10_Ref and the reference processor 20_Ref only needs to be performed once, and does not need to be performed sequentially.
[0089] The reference electronic mirror 10_Ref and the reference processor 20_Ref used in steps S1 and S2, respectively, and the reference electronic mirror 10_Ref and the reference processor 20_Ref used in steps S3 and S4, respectively, may be different. If steps S1 and S2 are performed at the manufacturing site of the processor 20, and steps S3 and S4 are performed at the manufacturing site of the electronic mirror 10, it is best to use different reference devices (reference electronic mirror 10_Ref, reference processor 20_Ref) respectively. In this case, as long as the common color index RC is used, the calculated processor matrix M1 and mirror matrix M2 are almost unchanged.
[0090] (Process S5)
[0091] Step S5 is a step in which the processor 20_Tar to be corrected corrects the image captured by the imaging element 14 of the electronic scope 10_Tar to be corrected by using the processor matrix M1 and the mirror matrix M2 when the electronic scope 10_Tar to be corrected is connected to the processor 20_Tar to be corrected.
[0092] Unlike steps S1 to S4 , step S5 is executed when diagnosis is performed at a medical site using the electronic endoscope system 1 in which the processor 20 storing the processor matrix M1 and the electronic scope 10 storing the mirror matrix M2 are connected.
[0093] Figure 6 The flowchart shows the processing executed by the processor 20 in step S5.
[0094] First, the processor 20 obtains the image of the current frame from the electronic mirror 10, performs specific image processing, and temporarily stores it in the image memory 23 (step S10). The processor 20 reads the processor matrix M1 from the matrix storage unit 28 (step S12) and reads the mirror matrix M2 from the matrix storage unit 16 of the electronic mirror 10 (step S14).
[0095] Then, the processor 20 performs a color correction operation on the image stored in the image memory 23 in step S10 (step S16). Specifically, the processor 20 obtains a pixel value obtained by converting the pixel value of each pixel of the image by a matrix (M1·M2), wherein the matrix (M1·M2) is the product of the processor matrix M1 and the mirror matrix M2. The processor 20 performs an image output process on the image in which the pixel value of each pixel is converted (step S18). In the image output process, a screen data image for monitor display is generated based on the image, and the generated screen data for monitor display is converted into a specific video format signal and output to the monitor 30.
[0096] As described above, in an image correction method of an electronic endoscope system of an embodiment, the mirror matrix and the processor matrix optimized before diagnosis are stored in the combined electronic mirror 10 and the processor 20 respectively, and the mirror matrix and the processor matrix can be used to correct the color information during diagnosis. Therefore, the processing load of the traditional system when correcting the color information can be greatly reduced. The electronic mirror 10 and the processor 20 only need to store the mirror matrix and the processor matrix respectively, and the required memory space is small.
[0097] In the above-mentioned image correction method of the electronic endoscope system, the processor matrix and the mirror matrix are determined in a state reflecting the spectral reflectance characteristics of the color index RC, wherein the color index RC varies with the subject, so that the pixel values (R value, G value, B value) of the image of each frame can be corrected according to the characteristics of the subject. For example, when each matrix is determined based on a captured image showing a reddish color index RC that matches the color of the affected part, correction corresponding to the spectral reflectance characteristics of the subject (the affected part of the patient) can be performed when making a diagnosis.
[0098] In the above-mentioned image correction method for an electronic endoscope system, since the pre-optimized processor matrix and mirror matrix are stored in the electronic mirror and the processor, there is no need to consider the combination of the electronic mirror and the processor when making a diagnosis. This means that even if any combination of the electronic mirror and the processor is used during diagnosis, the camera image can be optimally corrected.
[0099] (2) Second Embodiment
[0100] Hereinafter, a second embodiment will be described.
[0101] The second embodiment is different from the first embodiment in that a processor matrix and a mirror matrix are calculated in a form including not only an image but also image feature quantities, and image correction is performed based on the processor matrix and the mirror matrix.
[0102] The following mainly discusses the image correction method described in the first embodiment ( Figure 5Steps S1 to S5 of the image correction method of this embodiment are as follows.
[0103] Figure 7 Conceptual representation of the calculation of relative Figure 4 The order of the processor matrix M1a of the feature quantity is also included.
[0104] Reference Figure 7 In step S1, the processor to be corrected 20_Tar determines a feature quantity FTR1 (an example of a first feature quantity) based on a pixel value (an example of a first pixel value) of a specific pixel of a captured image IMG1 of a color index RC obtained by a combination of the reference electronic mirror 10_Ref and the processor to be corrected 20_Tar.
[0105] The reference processor 20_Ref determines a feature quantity FTR2 (an example of a second feature quantity) from pixel values (an example of a second pixel value) of corresponding pixels in the captured image IMG2 of the color index RC obtained by the combination of the reference electronic scope 10_Ref and the reference processor 20_Ref.
[0106] Then, the information processing device 40 calculates a processor matrix M1 a including parameters (one example of first feature amount correction parameters) for converting the feature amount FTR1 into the feature amount FTR2 in addition to parameters for converting the captured image IMG1 into the captured image IMG2 .
[0107] In step S2, the processor matrix M1a is stored in the processor to be calibrated 20_Tar.
[0108] Figure 8 This diagram conceptually shows the calculation of Figure 5 The order of the mirror matrix M2a (which also includes the characteristic quantities).
[0109] Reference Figure 8 In step S3, the reference processor 20_Ref determines a feature quantity FTR3 (an example of a third feature quantity) based on a pixel value (an example of a third pixel value) of a specific pixel of the captured image IMG3 of the color index RC obtained by a combination of the electronic mirror 10_Tar to be corrected and the reference processor 20_Ref.
[0110] The reference processor 20_Ref determines a feature quantity FTR4 (an example of a fourth feature quantity) based on pixel values (an example of a fourth pixel value) of corresponding pixels in the captured image IMG4 of the color index RC obtained by the combination of the reference electronic mirror 10_Ref and the reference processor 20_Ref.
[0111] Then, the information processing device 40 calculates a mirror matrix M2a including, in addition to parameters for converting the captured image IMG3 into the captured image IMG4, parameters for converting the feature quantity FTR3 into the feature quantity FTR4 (an example of a second feature quantity correction parameter).
[0112] In step S4, the mirror matrix M2a is stored in the electronic mirror 10_Tar to be corrected.
[0113] In step S5, when the electronic mirror 10_Tar to be corrected is connected to the processor 20_Tar to be corrected, the processor 20_Tar to be corrected uses the processor matrix M1a and the mirror matrix M2a to correct the image captured by the image sensor 14 of the electronic mirror 10_Tar to be corrected. Thus, the feature value obtained from the pixel value of the image captured by the image sensor 14 of the electronic mirror 10_Tar to be corrected is corrected by the processor 20_Tar to be corrected.
[0114] In steps S1 and S3, a method for calculating the feature value may be appropriately defined. For example, as a value representing the degree of inflammation of the inflammation site of the subject, i.e., the patient, the ratio of the pixel values of corresponding pixels (e.g., the ratio of the G value to the R value) may be used as the feature value.
[0115] In the present embodiment, since the parameters for correcting the feature amount are included in the processor matrix and the mirror matrix respectively, the feature amount can be easily corrected by executing a color correction operation on the image of each frame obtained during diagnosis.
[0116] In addition, in the present embodiment, it is preferable to determine the color index RC used in steps S1 and S3 from a plurality of color indices based on the feature amount obtained from the pixel value of the captured image. By selecting the color index so that the color tone of the subject is clearly reflected by the feature amount of the captured image, the values of the calculated processor matrix M1a and mirror matrix M2a can be made more suitable for correcting the feature amount. For example, as described above, when the ratio of the G value to the R value of the pixel is set as the feature amount, it is preferable to select a color index that includes red or reddish from a plurality of color indices so that the redness of the subject is clearly reflected by the feature amount.
[0117] The present invention relates to a patent application of Japanese Patent Application No. 2022-120393 filed with the Japan Patent Office on July 28, 2022, and the entire contents of the application are incorporated into this specification by reference.
Claims
1. An image correction method for an electronic endoscope system, in, The image correction method is a correction method for correcting a captured image in an electronic endoscope system, in which an electronic endoscope for capturing images of living tissue using an imaging element and an electronic endoscope processor for performing signal processing on the captured image captured by the imaging element and creating an image for display are connected. The image correction method comprises the following steps: The step of calculating a first calibration parameter corresponding to the processor to be calibrated, i.e., the processor for the electronic endoscope to be calibrated, by using a reference endoscope, i.e., the electronic endoscope serving as a calibration reference, and a reference processor, i.e., the processor for the electronic endoscope to be calibrated; The step of storing the calculated first correction parameter in the processor to be corrected; The step of calculating a second calibration parameter corresponding to the endoscope to be calibrated, i.e., the electronic endoscope to be calibrated, by using the reference endoscope and the reference processor; The step of storing the calculated second calibration parameters in the endoscope to be calibrated; and The step of correcting an image captured by an imaging element of the endoscope to be calibrated by the processor to be calibrated using the first calibration parameter and the second calibration parameter when the endoscope to be calibrated is connected to the processor to be calibrated.
2. The image correction method of the electronic endoscope system according to claim 1, in, The step of calculating the first correction parameter includes the step of calculating a parameter for converting the first pixel value into the second pixel value based on the first pixel value and the second pixel value and using the parameter as the first correction parameter. The first pixel value is a pixel value of a photographed image obtained when the reference endoscope and the processor to be calibrated are connected and a specific color index is photographed. The second pixel value is a pixel value of a photographed image obtained when the reference endoscope and the reference processor are connected and the color index is photographed. The step of calculating the second correction parameter includes the step of calculating a parameter for converting the third pixel value into the fourth pixel value based on the third pixel value and the fourth pixel value and using the parameter as the second correction parameter. The third pixel value is a pixel value of a camera image obtained when the endoscope to be calibrated and the reference processor are connected and a specific color index is captured. The fourth pixel value is a pixel value of a captured image obtained when the reference endoscope and the reference processor are connected and the color index is captured.
3. The image correction method of the electronic endoscope system according to claim 2, in, The image correction method further comprises the following steps: A step of determining a first feature quantity, i.e., a feature quantity obtained from the first pixel value, and a second feature quantity, i.e., a feature quantity obtained from the second pixel value, and calculating a first feature quantity correction parameter for converting the first feature quantity into the second feature quantity; A step of determining a third feature quantity, i.e., a feature quantity obtained from the third pixel value, and a fourth feature quantity, i.e., a feature quantity obtained from the fourth pixel value, and calculating a second feature quantity correction parameter for converting the third feature quantity into the fourth feature quantity; and When the endoscope to be calibrated is connected to the processor to be calibrated, the processor to be calibrated uses the first feature quantity correction parameter and the second feature quantity correction parameter to correct the feature quantity obtained from the pixel value of the camera image captured by the camera element of the endoscope to be calibrated.
4. The image correction method of the electronic endoscope system according to claim 3, in, The color index is determined from a plurality of color indices based on a feature amount derived from pixel values of a captured image.
