Endoscope white balance correction method and device, electronic device and storage medium

Abstract

The invention relates to an endoscope white balance correction method and device, an electronic device and a storage medium, and aims to solve the problems in related technologies. The endoscope white balance correction method is applied to the field of medical endoscopes and comprises the steps that whether a dominant light source in a current endoscope is a matched light source of the endoscope or not is recognized; the dominant light source is a light source with the maximum radiation illumination ratio when a target object is irradiated; when it is recognized that the current dominant light source is the matched light source of the endoscope, white balance correction parameters of the endoscope are determined; and processing the currently shot endoscope image according to the white balance correction parameter. According to the method and the device, the problem that the white balance correction effect is poor in the use process of the endoscope is solved, and then the in-vivo scene color restoration effect is improved.

Description

Technical Field

[0001] The present application relates to the field of medical endoscopes, and in particular to an endoscope white balance correction method, device, electronic device and storage medium. Background Art

[0002] An endoscope is a medical tool used to directly observe the surface of internal organs and tissues of the human body, which can help doctors observe the internal conditions of the human body. When using an endoscope for examination or surgery, the quality of the image is crucial to the diagnosis result. Among them, white balance (WB) is an important factor affecting image quality.

[0003] However, there are relatively few white targets in scenes inside the human body, and it is difficult for endoscopes to perform automatic white balance. The currently commonly used method is manual white balance correction. Manual white balance correction is only triggered and calibrated by facing a white target before surgery. If the endoscope or light source is not replaced during subsequent surgeries, there is no need to repeat the calibration. However, in actual applications, manual white balance correction has problems due to the small amount of light from the mirror or the long distance during calibration, which affects the effect of white balance correction, resulting in poor color restoration of scenes inside the body when the endoscope is used.

[0004] With regard to the problem in the related art that the color restoration effect of in vivo scenes during the use of endoscopes is poor, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a method, device, electronic device and storage medium for endoscope white balance correction are provided to solve the problem of poor color restoration of in vivo scenes during the use of the endoscope in the related art.

[0006] In a first aspect, a method for calibrating white balance of an endoscope is provided in this embodiment, and the method comprises:

[0007] Identify whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest radiation illumination when irradiating the target object;

[0008] When it is identified that the current dominant light source is a supporting light source of the endoscope, determining a white balance correction parameter of the endoscope;

[0009] The currently captured endoscopic image is processed according to the white balance correction parameters.

[0010] In some embodiments, the step of identifying whether the dominant light source in the current endoscope is the supporting light source of the endoscope includes:

[0011] Under the same exposure parameters, respectively acquiring a first image taken by the endoscope when the matching light source is turned on, and a second image taken by the endoscope when the matching light source is turned off;

[0012] Calculating a mean normalized brightness difference between the first image and the second image;

[0013] If the normalized brightness difference mean is greater than a preset threshold, it is determined that the dominant light source in the current endoscope is the supporting light source of the endoscope; otherwise, it is determined that the dominant light source in the current endoscope is ambient light.

[0014] In some of the embodiments, the calculating the normalized brightness difference mean between the first image and the second image includes: obtaining normalized brightness difference values ​​of corresponding pixels in the first image and the second image;

[0015] According to the image resolution of the first image and / or the second image, the normalized brightness difference values ​​of the corresponding pixels are averaged to obtain a normalized brightness difference mean value.

[0016] In some embodiments, the method further comprises:

[0017] When it is determined that the dominant light source is ambient light, determining whether the brightness of the supporting light source reaches a preset brightness threshold;

[0018] If the brightness of the matching light source does not reach a preset brightness threshold, controlling to increase the brightness of the matching light source;

[0019] If the brightness of the matching light source reaches a preset brightness threshold, the user is prompted in the display screen of the endoscope to adjust the endoscope closer to the white target in the endoscope image and re-trigger the white balance correction;

[0020] After receiving a signal from the user triggering white balance correction, it is again identified whether the dominant light source in the current endoscope is the supporting light source.

[0021] In some of the embodiments, after the controlling increases the brightness of the matching light source, the control further includes:

[0022] An endoscopic image captured after the brightness of the supporting light source is increased is acquired, and a dominant light source in the endoscope is identified based on the endoscopic image captured after the brightness of the supporting light source is increased.

[0023] In some embodiments, when the current dominant light source is identified as a supporting light source of the endoscope, determining the white balance correction parameters of the endoscope includes:

[0024] Determine a difference image according to brightness differences of corresponding pixels in the first image and the second image;

[0025] Divide the difference image into blocks to obtain a plurality of difference sub-image blocks;

[0026] The difference sub-image blocks whose color attributes are within a preset color attribute range are used as target image blocks; and the white balance correction parameters of the endoscope are determined according to the multiple target image blocks.

[0027] In some embodiments, determining the white balance correction parameters of the endoscope according to the plurality of target image blocks includes:

[0028] Based on a preset color temperature region, determining color temperature region weights of a plurality of target image blocks;

[0029] Determining a color component mean value of the difference image according to the color component values ​​of the plurality of target image blocks and the color temperature region weights;

[0030] The white balance correction parameter is determined according to the color component mean value.

[0031] In a second aspect, an endoscope white balance correction device is provided in this embodiment, the device comprising: a light source identification module, a parameter determination module and an image processing module;

[0032] The light source identification module is used to identify whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest radiation illumination ratio;

[0033] The parameter determination module is used to determine the white balance correction parameters of the endoscope when identifying that the current dominant light source is the supporting light source of the endoscope;

[0034] The image processing module is used to process the currently captured endoscopic image according to the white balance correction parameters.

[0035] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the endoscope white balance correction method described in the first aspect is implemented.

[0036] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the endoscope white balance correction method described in the first aspect is implemented.

[0037] Compared with the related art, the endoscope white balance correction method, device, electronic device and storage medium provided in this embodiment identify the dominant light source in the endoscope to determine whether the dominant light source is the supporting light source of the endoscope. When the dominant light source is identified as the supporting light source, the white balance correction parameters of the endoscope under the supporting light source are obtained, and the currently captured endoscope image is processed according to the white balance correction parameters. By obtaining the white balance correction parameters of the endoscope under the supporting light source, the interference of ambient light is eliminated, which is conducive to improving the white balance correction effect of the endoscope, and then improving the color restoration effect of the scene in the body.

[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 It is a hardware structure block diagram of a terminal of an endoscope white balance correction method provided in an embodiment of the present application;

[0041] Figure 2 is a flow chart of an endoscope white balance correction method provided in an embodiment of the present application;

[0042] Figure 3 is a schematic diagram of a dominant light source identification method provided in an embodiment of the present application;

[0043] Figure 4 is a basic flow chart of the white balance correction method provided in this specific embodiment;

[0044] Figure 5 is a flow chart of the dominant light source identification method provided in this specific embodiment;

[0045] Figure 6 is an example diagram of obtaining images of turning off and on the matching light source in this specific embodiment;

[0046] Figure 7 is a flow chart of a method for calculating white balance correction parameters provided in this specific embodiment;

[0047] Figure 8 is a schematic diagram of the color temperature region provided in this specific embodiment;

[0048] Fig. 9 It is a structural block diagram of an endoscope white balance correction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of an endoscope white balance correction method provided in an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0052] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the endoscope white balance correction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0054] Generally, when an endoscope is used in an in-vivo scene, it is difficult to use automatic white balance because there are relatively few white targets in the scene. The manual white balance correction method is generally used. White targets in in-vivo scenes include gauze, surgical instruments, etc. Manual white balance correction only requires triggering calibration with a white target before surgery. If the endoscope or light source is not replaced during subsequent surgeries, there is no need to repeat the calibration.

[0055] The current manual white balance correction has the problem that the accuracy of white balance correction is greatly affected by ambient light due to the small amount of mirror illumination or the long distance during calibration, which in turn affects the color restoration effect of the in vivo scene. For example, endoscopes with smaller outer diameters, such as otolaryngoscopes and ureteropelvic endoscopes, have a relatively small light output aperture at the front end of the mirror, so the mirror illumination flux is relatively small. When the distance is slightly farther, the proportion of ambient light increases and becomes the dominant light source. If the manual white balance correction is successful in the current in vitro scene, the color restoration effect in the in vivo scene may still be biased. Even if the current ambient light is not the dominant light source, the ambient light will also affect the white balance correction parameters.

[0056] In the existing white balance correction method, the white or gray target on the in-vivo image is identified, and then the target white balance parameters are determined according to the pixel information of the white target. The images in the image sequence taken by the endoscope are white-balanced according to the target white balance parameters. Although this method can eliminate the interference of ambient light, there is a problem that the white balance calibration cannot be completed when there is no white target in the in-vivo scene. In addition, there is also a method of removing color noise before white balance to reduce the interference of white points in white balance detection, and then converting the image to HSI color space, and filtering the areas that are more likely to be white by calculating weights, generating white balance parameters and correcting the image. However, this method does not consider the influence of ambient light on white balance. Even if a white block is found, this white block may be a white block affected by ambient light, which will lead to inaccurate white balance correction.

[0057] Therefore, the current endoscope white balance correction has the above-mentioned problems, and a white balance correction method that can eliminate the interference of ambient light is needed.

[0058] In this embodiment, a method for white balance correction of an endoscope is provided. When there is ambient light interference, a more accurate white balance correction effect is obtained by detecting the dominant light source and eliminating the interference of the ambient light. Figure 2 is a flow chart of the endoscope white balance correction method provided in an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0059] Step S210, identifying whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest proportion of radiation illumination when irradiating the target object.

[0060] In this step, when using an endoscope to perform white balance correction on a white target object outside the body, the processor first needs to identify the dominant light source of the current endoscope, and determine the subsequent white balance correction method based on the identified dominant light source. Exemplarily, the dominant light source is a light source that has a greater impact on image brightness and color. Specifically, the identification standard may include determining that the light source with the largest radiant illumination among multiple light sources irradiating the target object is the dominant light source. Among them, the radiant illumination is the radiation flux per unit area of ​​the irradiated surface. In the endoscope white balance correction, the light source irradiating the target object includes the supporting light source of the endoscope and the ambient light in which the target object is located. By identifying whether the dominant light source is the supporting light source of the endoscope, it is helpful to improve the subsequent elimination of interference from ambient light.

[0061] Step S220, when it is identified that the current dominant light source is the supporting light source of the endoscope, the white balance correction parameters of the endoscope are determined.

[0062] In this step, when the processor recognizes that the current dominant light source is the supporting light source of the endoscope, an endoscope image under the current dominant light source is acquired, and the white balance correction parameters of the endoscope are determined according to the color attributes in the endoscope image.

[0063] Step S230: Process the currently captured endoscope image according to the white balance correction parameters.

[0064] In this step, an endoscopic image is captured under the supporting light source of the endoscope, and the endoscopic image currently captured by the endoscope is processed using the white balance correction parameters obtained from the endoscopic image to obtain an image after white balance correction.

[0065] Through the above steps, the dominant light source in the endoscope is identified to determine whether the dominant light source is the supporting light source of the endoscope. When the dominant light source is identified as the supporting light source, the white balance correction parameters of the endoscope under the supporting light source are obtained, and the currently captured endoscope image is processed according to the white balance correction parameters. By obtaining the white balance correction parameters of the endoscope under the supporting light source, the interference of ambient light is eliminated, which is conducive to improving the white balance correction effect of the endoscope.

[0066] Figure 3 is a schematic diagram of a dominant light source identification method provided in an embodiment of the present application. In some of the embodiments, reference Figure 3 In step S210, identifying whether the dominant light source in the current endoscope is the supporting light source of the endoscope includes:

[0067] Step S211, under the same exposure parameters, respectively acquire a first image taken by the endoscope when the matching light source is turned on, and a second image taken by the endoscope when the matching light source is turned off.

[0068] In this step, the processor maintains the exposure parameters of the endoscope, and turns on and off the matching light source under the exposure parameters, and respectively captures the corresponding first image and second image. The exposure parameters are used to adjust the brightness of the image, and mainly include three elements: aperture, shutter speed, and ISO sensitivity. Exemplarily, the exposure parameters can be obtained based on experience, or can be determined based on the light source and image brightness in the actual shooting scene, and are not specifically limited here.

[0069] Step S212: Calculate the average of the normalized brightness differences between the first image and the second image.

[0070] Among them, calculating the normalized brightness difference mean between the first image and the second image includes: obtaining the normalized brightness difference values ​​of corresponding pixel points in the first image and the second image; and averaging the normalized brightness difference values ​​of corresponding pixel points according to the image resolution of the first image and / or the second image to obtain the normalized brightness difference mean.

[0071] Exemplarily, the image resolutions of the first image and the second image are the same. The first image and the second image include a plurality of corresponding pixels. Since the shooting conditions of the first image and the second image are different, that is, the shooting conditions for controlling the turning on or off of the dominant light source are different, the brightness of the corresponding pixels in the first image and the second image in different channels in the color space is different. According to the different brightness of the corresponding pixels in the first image and the second image, combined with the resolution of the image, the normalized brightness difference mean is calculated.

[0072] Step S213, if the normalized brightness difference mean is greater than a preset threshold, it is determined that the dominant light source in the current endoscope is the supporting light source of the endoscope; otherwise, it is determined that the dominant light source in the current endoscope is ambient light.

[0073] In this step, when the calculated mean value of the normalized brightness difference is greater than the preset threshold, it means that the supporting light source contributes more to the image brightness, and then the dominant light source in the current endoscope is determined to be the supporting light source of the endoscope; when the calculated mean value of the normalized brightness difference is less than the preset threshold, it means that the ambient light contributes more to the image brightness, and then the dominant light source in the current endoscope is determined to be the ambient light.

[0074] Through the above steps, the first image and the second image under the same exposure parameters are obtained by taking the endoscope in the scenes where the dominant light source is turned on and off, and the global normalized brightness difference mean of the image is obtained by comparing the brightness of all corresponding pixels in the first image and the second image, combined with the resolution of the image. Thereafter, according to the global normalized brightness difference mean of the image, it is determined whether the dominant light source of the current endoscope is the supporting light source of the endoscope.

[0075] In one possible embodiment, when it is determined that the dominant light source is ambient light, it is determined whether the brightness of the supporting light source reaches a preset brightness threshold; if the brightness of the supporting light source does not reach the preset brightness threshold, the brightness of the supporting light source is controlled to be increased; after controlling to increase the brightness of the supporting light source, it also includes: acquiring an endoscopic image taken after the brightness of the supporting light source is increased, and identifying the dominant light source in the endoscope based on the endoscopic image taken after the brightness of the supporting light source is increased.

[0076] In this step, when it is determined that the dominant light source of the endoscope is not the supporting light source of the endoscope, for example, when it is determined that the dominant light source of the endoscope is ambient light, it is necessary to further determine the brightness of its light source. Exemplarily, it is determined whether the brightness of the supporting light source of the endoscope is at a maximum value at this time. If the brightness of the supporting light source does not reach a preset brightness threshold, that is, it does not reach the maximum brightness of the supporting light source, the brightness of the supporting light source is controlled to increase to the maximum brightness, and then the dominant light source is re-identified. The above steps are helpful to avoid the situation where the ambient light has too much influence on the image captured by the endoscope due to the brightness of the supporting light source of the endoscope not reaching the maximum value, which is helpful to further eliminate the interference of ambient light and improve the accuracy of subsequent endoscope white balance correction.

[0077] In one possible embodiment, when it is determined that the dominant light source is ambient light, it is determined whether the brightness of the supporting light source reaches a preset brightness threshold; if the brightness of the supporting light source reaches the preset brightness threshold, the user is prompted in the display screen of the endoscope to adjust the endoscope close to the white target in the endoscope image, and the white balance correction is re-triggered; after receiving the signal of the user triggering the white balance correction, it is again identified whether the dominant light source in the current endoscope is the supporting light source. In this step, when it is determined that the dominant light source of the endoscope is not the supporting light source of the endoscope, for example, when it is determined that the dominant light source of the endoscope is ambient light, it is necessary to further determine the brightness of its light source. Exemplarily, it is determined whether the brightness of the supporting light source of the endoscope is the maximum value at this time. If the brightness of the supporting light source has reached the preset brightness threshold, that is, the maximum value of the brightness of the supporting light source, the dominant light source is ambient light at this time. This may be because the front end of the lens is too far away from the white target, or the brightness of the supporting light source is attenuated more. At this time, it is necessary to prompt the user to adjust the lens of the endoscope closer to the white target, and manually trigger the white balance correction again, and then identify the dominant light source again.

[0078] When the user receives the prompt information of adjusting the endoscope to be close to the white target in the endoscope image in the display screen of the endoscope, the endoscope lens is adjusted according to the prompt information so that the endoscope is close to the white target. Further, the prompt information also includes manually triggering the white balance correction again after the lens is adjusted close to the white target. After the user completes the distance adjustment between the endoscope lens and the white target, the user needs to manually trigger the white balance correction of the endoscope again. After the processor receives the signal of the user triggering the white balance correction, it identifies again whether the dominant light source of the current endoscope is a supporting light source. After the user is prompted to adjust the endoscope lens, it identifies again whether the dominant light source is a supporting light source, which is conducive to further eliminating the influence of ambient light and distance on the image captured by the endoscope. Through the above steps, the situation of insufficient brightness of the supporting light source caused by the distance between the endoscope lens and the target object is avoided, and the accuracy of the white balance correction is further improved. At the same time, the interaction with the user in the white balance correction is realized, which is conducive to shortening the time of white balance correction, further ensuring that the white balance setting accurately reflects the true color of the environment in which the endoscope is located, and is conducive to improving the accuracy and efficiency of white balance correction.

[0079] In one possible embodiment, when the current dominant light source is identified as the supporting light source of the endoscope in step S220, the white balance correction parameters of the endoscope are determined, including: determining a difference image based on the brightness difference between corresponding pixels in the first image and the second image; dividing the difference image into blocks to obtain a plurality of difference sub-image blocks; taking the difference sub-image blocks whose color attributes are within a preset color attribute range as target image blocks; and determining the white balance correction parameters of the endoscope based on the plurality of target image blocks.

[0080] Wherein, determining white balance correction parameters in an endoscopic image according to a plurality of target image blocks comprises: determining color temperature area weights of a plurality of target image blocks based on a preset color temperature area; determining a color component mean of a difference image according to color component values ​​and color temperature area weights of a plurality of target image blocks; and determining white balance correction parameters according to the color component mean.

[0081] In this step, when the current dominant light source is the supporting light source of the endoscope, the first image and the second image taken by the endoscope when the supporting light source is turned on and off are obtained respectively, and the brightness difference of all the corresponding pixels in the first image and the second image is determined, and the difference image is obtained according to the brightness difference. Then, the difference image is divided into blocks to obtain a plurality of difference sub-image blocks, and the way of dividing the difference image here can be to divide it equally according to the preset pixels. Then, the values ​​of the color attributes of all pixels in the plurality of difference sub-image blocks are calculated respectively, and the color attributes include brightness and color difference. Then, the difference sub-image blocks whose color attributes are within the preset color attribute range are taken as the target image blocks, and the difference sub-image blocks other than the target image blocks are eliminated. Based on the preset color temperature area, the color temperature area weights of the plurality of target image blocks are determined; and according to the color component values ​​and the color temperature area weights of the plurality of target image blocks, the weighted mean of the color components of the entire image, i.e., the color mean, is calculated, and the white balance correction parameters are determined according to the color mean.

[0082] The present embodiment is described and illustrated by means of specific examples below.

[0083] Figure 4 is a basic flow chart of the white balance correction method provided by this specific embodiment. Figure 4 The white balance correction method includes: step S10 to step S17.

[0084] Step S10, identifying the dominant light source.

[0085] Step S11, determining whether the dominant light source is a matching light source. If yes, executing step S12, if no, executing step S13.

[0086] Step S12, eliminating the interference of ambient light and calculating white balance correction parameters rgain and bgain.

[0087] Step S13, determining whether the light source brightness has reached the maximum value. If yes, proceed to step S16, if no, proceed to step S17.

[0088] Step S14: white balance correction.

[0089] Step S15, prompting the user that "white balance correction is successful".

[0090] Step S16, prompting the user to "please adjust the lens closer to the white target and manually trigger white balance correction again."

[0091] Step S17, adjusting the brightness of the light source to the maximum.

[0092] Exemplarily, the light source brightness in steps S13 and S17 refers to the brightness of the matching light source of the endoscope.

[0093] Specifically, refer to Figure 1 , after the white balance correction starts, step S10 is executed to identify the dominant light source, which is the light source that has a greater impact on the brightness and color of the image. Generally, when multiple light sources illuminate the target object, the light source with the largest radiant illumination is the dominant light source. Step S11 determines the current dominant light source. If the dominant light source is the supporting light source for the endoscope, the white balance correction can be successfully performed, and steps S12 and S14 are processed. Step S14 uses the white balance correction parameters rgain and bgain obtained in step S12 to process each pixel on the image to obtain a new color component value:

[0094] r(i,j)=r(i,j)*rgain

[0095] b(i,j)=b(i,j)*bgain

[0096] Where r(i,j) and b(i,j) are the red and blue color components of the pixel at position (i,j), and g(i,j) is the green color component and is not processed.

[0097] If the dominant light source is not the supporting light source, then it is ambient light, and it is necessary to further determine whether the current supporting light source brightness has been set to the maximum value. If the supporting light source brightness has been set to the maximum value, then it is possible that the front end of the lens is too far away from the white target, or the light source brightness has been attenuated. At this time, the user needs to be prompted to "please adjust the lens closer to the white target", otherwise even if the white balance correction is successful, the accuracy of the white balance correction will be relatively poor. If the light source brightness has not been set to the maximum value, then the light source brightness needs to be adjusted to the maximum, and steps S10, S11 and subsequent steps are performed again, that is, the dominant light source is identified again.

[0098] Figure 5 is a flow chart of the dominant light source identification method provided in this specific embodiment. Figure 5 The white balance correction method includes: step S20 to step S25.

[0099] Step S20: The exposure parameters remain unchanged.

[0100] Step S21, acquiring an image frame with the light source turned on and an image frame with the light source turned off.

[0101] Step S22, calculating the average of the normalized brightness difference between the two frames of images.

[0102] Step S23, determining whether the normalized brightness difference mean is greater than a threshold value. If yes, proceed to step S25, if no, proceed to step S24.

[0103] Step S24, identifying the dominant light source as ambient light.

[0104] Step S25, identifying the dominant light source as a supporting light source.

[0105] Specifically, when the dominant light source is identified, the current exposure parameters are kept unchanged, and then in step S21, an image with one frame of light source turned on and one frame of light source turned off is obtained. Step S22 calculates the brightness difference between the two frames of images obtained in step S21, and calculates the global mean. First, the brightness difference is calculated for each pixel on the image. Here, the method for calculating the brightness difference is to convert the image from the RGB color space to the YCrCb color space. The specific formula is as follows:

[0106] Y diff (i, j) = 0.2126*(r on (i, j)-r off (i,j))+0.7152*(g on (i, j)-g o f f (i,j))+0.0722(b on (i, j)-b off (i, j))

[0107] Among them, r on (i, j), g on (i, j), b on (i, j) is the color component value at position (i, j) on the image when the light source is turned on, r off (i, j), g off (i, j), b off (i, j) is the color component value at position (i, j) on the image when the light source is turned off.

[0108] Normalized brightness difference y norm_diff (i, j) is:

[0109] y norm_diff (i, j) = Y diff (i, j) / y on (i, j)

[0110] y o n(i, j) = 0.2126*r on (i, j)+0.7152*g on (i, j)+0.0722*b on (i, j)

[0111] Then calculate the global normalized brightness difference mean y of the image mean-diff :

[0112]

[0113] Among them, the image resolution is M*N.

[0114] After that, after the normalized brightness difference mean is calculated, step S23 compares the normalized brightness difference mean with the threshold thr, i.e., the preset threshold in the above-mentioned embodiment. When the normalized brightness difference mean is less than the threshold, it means that the supporting light source contributes less to the image brightness, and the dominant light source is identified as ambient light. When the normalized brightness difference mean is greater than the threshold, it means that the supporting light source contributes more to the image brightness, and the dominant light source is identified as the supporting light source. It is generally recommended to set the threshold thr greater than 0.5, so that it is easier to obtain accurate white balance correction results.

[0115] In one specific embodiment, when using an image sensor with rolling shutter exposure, referring to Figure 6 , Figure 6 It is an example diagram of obtaining images with the supporting light source turned off and on in this specific embodiment. Among them, when the supporting light source level is high, it means that the supporting light source is in the on state at this time, for example, when the Sensor is exposed to the Nth frame, the N+4th frame and the N+5th frame, the supporting light source is in the on state; when the supporting light source level is low, it means that the supporting light source is in the off state, for example, when the supporting light source is in the N+2th frame, it is in the off state. When the N+1th and N+3th frames are exposed, the supporting light source will be switched on and off, so there will be differences in the brightness of pixels in different rows on the image, but because it is in the process of white balance correction, the abnormality of the picture has no effect on in vivo use. In this embodiment, the N or N+4 frame image can be obtained as the image when the light source is turned on, and the N+2 frame image can be obtained as the image when the light source is turned off, which is used to identify the dominant light source and subsequently calculate the white balance correction parameters.

[0116] In another possible embodiment, a global exposure method may be adopted, in which the switching time of the matching light source is controlled to be aligned with the sensor exposure start time, so that images of corresponding frames when the matching light source is turned on and off can be directly obtained.

[0117] Figure 7 is a flow chart of the method for calculating white balance correction parameters provided in this specific embodiment. Figure 7 As shown, the white balance correction parameter calculation method includes steps S30 to S35.

[0118] Step S30, calculate the difference between the two frames of images in step S21 to obtain image I. The two frames of images here are the first image and the second image in the aforementioned embodiment, and image I is the difference image in the aforementioned embodiment. Among them, step S30 calculates the difference image I between the two frames of images in step S21 for the subsequent calculation of white balance correction parameters. The image when the light source is turned off can be considered to have only ambient light, and the image when the light source is turned on is the effect of the superposition of ambient light and the matching light source. Therefore, the difference image calculated from the two frames of images can be considered to have only the effect of the matching light source lighting.

[0119] Step S31, performing block-by-block statistics on brightness and color difference of the image I.

[0120] Specifically, the brightness and color difference of image I are divided into blocks and counted. The average brightness of the i-th block is:

[0121] y i_mean =0.2126*r i_mean +0.7152*g i_mean +0.0722*b i_mean

[0122] The mean color difference c of the i-th block b_i and c r_i The calculation method is:

[0123] c b_i = = b i_mean -y i_mean

[0124] c r_i =r i_mean -y i_mean

[0125] where r i_mean , g i_mean , b i_mean is the mean color component of all pixels in the block.

[0126] Step S32, determine whether the brightness and color difference of the blocks are within the specified range, and remove the blocks that are not within the specified range. Here, the image I is divided into blocks to obtain the difference sub-image blocks in the above embodiment.

[0127] Specifically, step S32 determines whether the block brightness and color difference are within the specified range, and removes the blocks that are not within the specified range. The removed blocks are not used for subsequent calculations. For blocks with too high or too low brightness, the proportional relationship of the color components RGB may change, and cannot truly reflect the characteristics of the light source spectrum, so they need to be removed. Blocks with color differences exceeding the specified range may be non-white blocks and also need to be removed, because the calculation of white balance correction parameters only counts white blocks. Assume that the brightness limit range is: [Ymi n , Yma x ], the color difference limit range is: [c b_min , c b_max ]、[c r_min , c r_max ], the brightness and color difference are within the limited range: min <y i <y max , c b_min <c b_i <c b_max , c r_min<c r_i <c r_max .

[0128] Step S33, calculating the color temperature area weight for the blocks that meet the limited ranges of brightness and color difference.

[0129] Specifically, the color temperature area weight is calculated for blocks that meet the brightness and color difference limits. Generally, color temperature areas are calibrated in advance for light sources with different color temperatures. Commonly used light sources include: D75, D65, D50, TL84, CWF, A, etc. The calibrated color temperature areas are as follows: Figure 8 As shown, Figure 8 is a schematic diagram of the color temperature region provided in this specific embodiment. The same color line frames a color temperature region of a light source, where the horizontal axis is g / r and the vertical axis is g / b. Figure 8 , from the upper left to the lower right, the color temperature gradually increases. For each block, its g i_mean / r i_mean and g i_mean / b i_mean , determine whether it falls into the preset color temperature area, if so, its corresponding color temperature area weight value w i =1, expressed as Figure 8 The red dot in the i =0.

[0130] Step S34, calculating the color component mean of the entire image according to the color component value of each block and the color temperature area weight.

[0131] Specifically, step S34 calculates the color component mean of the entire image according to the color component value of each block and the color temperature area weight:

[0132]

[0133]

[0134]

[0135] Among them, m is the total number of blocks, w i Indicates the color temperature area weight value. i , g i , b i Represents the color component value of each block respectively.

[0136] Step S35, obtaining white balance correction parameters rgain and bgain according to the color component averages.

[0137] Specifically, the white balance correction parameters rgain and bgain are obtained according to the average value of the color components:

[0138] rgain=gmean / r mean

[0139] bgain=g mean / b mean

[0140] The endoscope white balance correction method provided in this embodiment first identifies the dominant light source, and when the dominant light source is a matching light source, calculates the white balance correction parameters by eliminating the interference of ambient light; when the dominant light source is ambient light and the light source brightness level has been adjusted to the maximum, the user is prompted to perform the correct operation, otherwise the light source brightness is adjusted to the maximum and the previous process is repeated.

[0141] Among them, the dominant light is identified. By keeping the exposure parameters unchanged, the light source is turned off and on to obtain two frames of images when the light source is turned on and off respectively. If the brightness difference between the two frames of images is small, the ambient light is determined to be the dominant light source. If the difference is large, the matching light source is determined to be the dominant light source.

[0142] Furthermore, to eliminate the interference of ambient light and calculate the white balance correction parameters, a difference image I is obtained based on two frames of images with the light source turned on and off, the brightness and color difference of the difference image are divided into blocks, the blocks whose brightness and color difference are not in the specified range are removed, and the color temperature area weights of the blocks whose brightness and color difference are in the specified range are obtained. Finally, the white balance correction parameters are obtained according to the color component values ​​and the color temperature area weights.

[0143] Through the endoscope white balance correction method of the present application, in the presence of ambient light interference, the interference of ambient light can be eliminated and accurate white balance correction parameters can be calculated. When the user performs irregular correction operations, such as when the distance is far, the irregular operations can be detected and the user can be prompted, which is beneficial to improving the accuracy of the endoscope white balance correction effect and the correction efficiency.

[0144] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a group of computer executable instructions.

[0145] In this embodiment, an endoscope white balance correction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0146] Fig. 9 is a structural block diagram of an endoscope white balance correction device provided in an embodiment of the present application, such as Fig. 9As shown, the device includes: a light source identification module 10 , a parameter determination module 20 and an image processing module 30 .

[0147] The light source identification module 10 is used to identify whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest proportion of radiation illumination.

[0148] The parameter determination module 20 is used to determine the white balance correction parameters of the endoscope when it is identified that the current dominant light source is the supporting light source of the endoscope.

[0149] The image processing module 30 is used to process the currently captured endoscope image according to the white balance correction parameters.

[0150] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0151] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0152] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0153] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0154] S1, identifying whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest proportion of radiation illumination when irradiating the target object.

[0155] S2, when it is identified that the current dominant light source is the supporting light source of the endoscope, determining the white balance correction parameters of the endoscope.

[0156] S3, processing the currently captured endoscope image according to the white balance correction parameters.

[0157] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0158] In addition, in combination with the endoscope white balance correction method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the endoscope white balance correction method. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the endoscope white balance correction methods in the above embodiments is implemented.

[0159] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0160] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0161] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0162] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for calibrating white balance of an endoscope, characterized in that: The method comprises: Identify whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest radiation illumination when irradiating the target object; When it is identified that the current dominant light source is a supporting light source of the endoscope, determining a white balance correction parameter of the endoscope; The currently captured endoscopic image is processed according to the white balance correction parameters.

2. The endoscope white balance correction method according to claim 1, characterized in that: The step of identifying whether the dominant light source in the current endoscope is the supporting light source of the endoscope includes: Under the same exposure parameters, respectively acquiring a first image taken by the endoscope when the matching light source is turned on, and a second image taken by the endoscope when the matching light source is turned off; Calculating a mean normalized brightness difference between the first image and the second image; If the normalized brightness difference mean is greater than a preset threshold, it is determined that the dominant light source in the current endoscope is the supporting light source of the endoscope; otherwise, it is determined that the dominant light source in the current endoscope is ambient light.

3. The endoscope white balance correction method according to claim 2, characterized in that: The calculating the normalized brightness difference mean between the first image and the second image comprises: obtaining normalized brightness difference values ​​of corresponding pixels in the first image and the second image; According to the image resolution of the first image and / or the second image, the normalized brightness difference values ​​of the corresponding pixels are averaged to obtain a normalized brightness difference mean.

4. The endoscope white balance correction method according to claim 2, characterized in that: The method further comprises: When it is determined that the dominant light source is ambient light, determining whether the brightness of the supporting light source reaches a preset brightness threshold; If the brightness of the matching light source does not reach a preset brightness threshold, controlling to increase the brightness of the matching light source; If the brightness of the matching light source reaches a preset brightness threshold, the user is prompted in the display screen of the endoscope to adjust the endoscope closer to the white target in the endoscope image and re-trigger the white balance correction; After receiving a signal from the user triggering white balance correction, it is again identified whether the dominant light source in the current endoscope is the supporting light source.

5. The endoscope white balance correction method according to claim 4, characterized in that: After the control increases the brightness of the matching light source, the method further includes: An endoscopic image captured after the brightness of the supporting light source is increased is acquired, and a dominant light source in the endoscope is identified based on the endoscopic image captured after the brightness of the supporting light source is increased.

6. The endoscope white balance correction method according to claim 2, characterized in that: The step of determining the white balance correction parameters of the endoscope when identifying that the current dominant light source is the supporting light source of the endoscope includes: Determine a difference image according to brightness differences of corresponding pixels in the first image and the second image; Divide the difference image into blocks to obtain a plurality of difference sub-image blocks; The difference sub-image blocks whose color attributes are within a preset color attribute range are used as target image blocks; and the white balance correction parameters of the endoscope are determined according to the multiple target image blocks.

7. The endoscope white balance correction method according to claim 6, characterized in that: The step of determining the white balance correction parameters of the endoscope according to the plurality of target image blocks comprises: Based on a preset color temperature region, determining color temperature region weights of a plurality of target image blocks; Determining a color component mean value of the difference image according to the color component values ​​of the plurality of target image blocks and the color temperature region weights; The white balance correction parameter is determined according to the color component mean value.

8. An endoscope white balance correction device, characterized in that: The device comprises: a light source identification module, a parameter determination module and an image processing module; The light source identification module is used to identify whether the dominant light source in the current endoscope is the supporting light source of the endoscope; the dominant light source is the light source with the largest radiation illumination ratio; The parameter determination module is used to determine the white balance correction parameters of the endoscope when identifying that the current dominant light source is the supporting light source of the endoscope; The image processing module is used to process the currently captured endoscopic image according to the white balance correction parameters.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the endoscope white balance correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the endoscope white balance correction method according to any one of claims 1 to 7 are implemented.

Images