A lighting correction method, apparatus, electronic device and storage medium
By identifying the reference light source and the light source to be calibrated in the endoscopic system, generating a characteristic response ratio model and calculating the adjustment coefficient, the problem of endoscopic spectral inconsistency is solved, and spectral consistency and image quality are improved.
Patent Information
- Application Number
- CN202210762851.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-30
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-06-30
AI Technical Summary
The emission spectrum of multi-wavelength light sources in an endoscope system is inconsistent with the actual illumination spectrum transmitted to the endoscope, leading to problems such as abnormal image color and decreased contrast.
By identifying the reference light source and the light source to be calibrated in a multi-wavelength light source, a response image is acquired using a preset driving signal, a characteristic response ratio model is generated, and an adjustment coefficient is calculated to correct the driving signal of the light source to be calibrated, thereby achieving spectral consistency.
Spectral correction can be directly implemented in existing endoscope systems, reducing complexity, eliminating the need for special equipment, ensuring consistent illumination spectra from different endoscopes, and improving image quality.
Smart Images

Figure CN117377173B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of endoscopic illumination, and in particular to an illumination correction method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] Multi-wavelength illumination devices in endoscopic systems can employ various wavelength light-emitting elements as illumination sources, and combine the light beams through an optical lens group, converging them onto the fiber optic bundle in the endoscope's light guide section to provide illumination to the endoscope lens. However, due to limitations in structural positional tolerances and assembly processes, as well as the aging of light-guiding materials such as the fiber optic bundle, endoscopic systems using multi-wavelength illumination often suffer from inconsistencies between the emitted spectrum and the actual illumination spectrum transmitted to the endoscope. These inconsistencies in the illumination spectra emitted by different endoscopes can ultimately lead to numerous problems, including abnormal endoscopic image colors, decreased contrast, and poor performance of special lighting effects.
[0003] Therefore, improving the consistency of the illumination spectrum emitted from the head end of the endoscope system to the subject is a technical problem that urgently needs to be solved.
[0004] Application content
[0005] The purpose of this application is to provide an illumination correction method, apparatus, electronic device, and computer-readable storage medium that can correct the light source to be corrected based on the proportional relationship between the response characteristics of the light source to be corrected and the response characteristics of the reference light source, thereby effectively reducing the complexity of endoscopic illumination correction.
[0006] To address the aforementioned technical problems, this application provides an illumination correction method applied to an endoscope system, the endoscope system comprising a multi-wavelength light source and an endoscope, the illumination correction method comprising:
[0007] Determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source;
[0008] The reference light source is set to emit light constantly in a specified illumination mode. At the same time, the light source to be calibrated is lit up using multiple preset drive signals, and the response image of the endoscope to the standard object is acquired each time it is lit up.
[0009] A feature response ratio model is generated using the preset driving signal and its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source and the driving signal of the light source to be corrected;
[0010] An adjustment coefficient is determined using the characteristic response ratio model, a preset ratio standard value, and the original drive signal of the light source to be calibrated corresponding to the specified lighting mode. The adjustment coefficient is then used to correct the original drive signal of the light source to be calibrated in each lighting mode.
[0011] Optionally, the standard object is a white balance cap.
[0012] Optionally, determining the reference light source section and the light source section to be calibrated in the multi-wavelength light source includes:
[0013] Determine the reference light source section and any light source section to be corrected in the multi-wavelength light source;
[0014] Then, after determining the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be corrected corresponding to the specified lighting mode, the lighting correction method further includes:
[0015] The next light source to be calibrated in the multi-wavelength light source is determined, and the step of illuminating the light source to be calibrated using multiple preset driving signals is performed until the adjustment coefficients of all the light sources to be calibrated have been determined.
[0016] Optionally, the lighting correction method further includes:
[0017] The target illumination mode is determined, and the original driving signal under the target illumination mode is used to illuminate all the light source parts of the multi-wavelength light source, and the original response image of the endoscope to the standard object is acquired.
[0018] Acquire a standard response image of the standard object captured by a standard endoscope system under the target illumination mode, and determine a first deviation value between the original response image and the standard response image;
[0019] The original driving signal of the corresponding light source unit to be calibrated is corrected using all the aforementioned adjustment coefficients to obtain the calibrated driving signal under the target illumination mode;
[0020] The correction drive signal in the target illumination mode is used to illuminate all the light source parts of the multi-wavelength light source, and at the same time, the correction response image of the endoscope to the standard object is acquired;
[0021] Determine a second deviation value between the corrected response image and the standard response image, and determine whether the second deviation value is less than a first deviation value;
[0022] If so, then save all the aforementioned adjustment coefficients.
[0023] Optionally, generating a feature response ratio model using the preset driving signal and its corresponding response image includes:
[0024] A driving response model is generated using the preset driving signal and the color values of the response image;
[0025] Based on the first response feature corresponding to the reference light source and the second response feature corresponding to the light source to be corrected, the first response function corresponding to the first response feature and the second response function corresponding to the second response feature are generated using the parameters in the driving response model.
[0026] The characteristic response ratio model is generated using the first response function and the second response function.
[0027] Optionally, generating a driving response model using the preset driving signal and the color values of the response image includes:
[0028] The correlation between the preset driving signal and the color values of the response image is fitted using a polynomial fitting method to obtain the driving response model.
[0029] Optionally, the first response feature and the second response feature each correspond to different color channels, and the step of generating the first response function corresponding to the first response feature and the second response function corresponding to the second response feature using the parameters in the driving response model includes:
[0030] The first response function corresponding to the color channel of the first response feature and the second response function corresponding to the color channel of the second response feature are extracted from the driving response model.
[0031] This application also provides an illumination correction device for use in an endoscope system, the endoscope system including a multi-wavelength light source and an endoscope, the illumination correction device comprising:
[0032] An initialization module is used to determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source.
[0033] The driving module is used to set the reference light source to emit light constantly in a specified illumination mode, and at the same time, to illuminate the light source to be calibrated using multiple preset driving signals, and to acquire the response image of the endoscope to the standard object each time it is illuminated.
[0034] The model generation module is used to generate a feature response ratio model using the preset driving signal and its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source and the driving signal of the light source to be corrected.
[0035] The adjustment coefficient calculation module is used to determine the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be corrected corresponding to the specified lighting mode, so as to use the adjustment coefficient to correct the original drive signal of the light source to be corrected in each lighting mode.
[0036] This application also provides an electronic device, including:
[0037] Memory, used to store computer programs;
[0038] A processor for implementing the lighting correction method as described above when executing the computer program.
[0039] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the lighting correction method described above.
[0040] This application provides an illumination correction method applied to an endoscope system. The endoscope system includes a multi-wavelength light source and an endoscope. The illumination correction method includes: determining a reference light source and a light source to be corrected in the multi-wavelength light source; setting the reference light source to emit light constantly in a specified illumination mode, while simultaneously illuminating the light source to be corrected using multiple preset driving signals, and acquiring a response image of a standard object captured by the endoscope each time the light source is illuminated; generating a feature response ratio model using the preset driving signals and their corresponding response images; the feature response ratio model reflects the correspondence between the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source and the driving signals of the light source to be corrected; determining an adjustment coefficient using the feature response ratio model, a preset ratio standard value, and the original driving signal of the light source to be corrected corresponding to the specified illumination mode, so as to correct the original driving signal of the light source to be corrected in each illumination mode using the adjustment coefficient.
[0041] As can be seen, this application first determines the reference light source and the light source to be calibrated in a multi-wavelength light source, and adjusts the reference light source to emit light at a constant rate under a specified illumination mode to ensure that the light output of the reference light source remains constant, which facilitates the adjustment of the relative spectrum between the light sources. Subsequently, this application collects the actual response image data generated by the light source to be calibrated under different driving signal conditions and the reference light source with fixed light output, and generates a characteristic response ratio model using a preset driving signal and its response image. Through this characteristic response ratio model, the correspondence between the ratio of the unique image response features of the light source to be calibrated and the unique image response features of the reference light source and the driving signal of the light source to be calibrated can be established. Then, using this characteristic response ratio model, the target driving signal that can achieve the preset ratio standard value can be calculated, and based on the target driving signal and the original driving signal of the light source to be calibrated corresponding to the specified illumination mode, the adjustment coefficient of the light source to be calibrated can be determined. Finally, based on the adjustment coefficient, the light source to be calibrated can be calibrated, so that the relative spectrum of the illumination light emitted from different endoscopes can be made consistent. Furthermore, the aforementioned illumination correction process can be directly implemented within existing endoscope systems without the need for special fixtures and optical power measurement equipment, thus effectively reducing the complexity of endoscope illumination correction. This application also provides an illumination correction device, electronic device, and computer-readable storage medium, which possess the aforementioned beneficial effects. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0043] Figure 1 This is a schematic diagram of an endoscope system provided in an embodiment of this application;
[0044] Figure 2 A flowchart illustrating a lighting correction method provided in an embodiment of this application;
[0045] Figure 3 A flowchart of a lighting correction control timing provided in an embodiment of this application;
[0046] Figure 4 This is a structural block diagram of a lighting correction device provided in an embodiment of this application;
[0047] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] To facilitate understanding, a brief description of the endoscopic system to which this application applies will be provided first. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of an endoscope system provided in an embodiment of this application. As can be seen, the endoscope system includes a multi-wavelength light source 10, an endoscope 11, and an image processor 12.
[0050] Specifically, the multi-wavelength light source 10 in the endoscope includes two or more light source units. For example, as one example, it may include a first light source unit 101, a second light source unit 102, a third light source unit 103, and a fourth light source unit 104. These light source units can be uniformly controlled by a light source control unit 105, which can also individually control the driving signals of each light source unit. For instance, when a certain imaging mode and a certain image brightness level are selected, the light source control unit 105 can drive the first light source unit 101, the second light source unit 102, the third light source unit 103, and / or the fourth light source unit 104 to emit light according to the original driving signals corresponding to the imaging mode and the image brightness level. The illumination light emitted by these light source units is coupled into the optical fiber (not shown) in the endoscope 11 by the beam combining optical lens group 106, and transmitted to the head end of the endoscope 11 to illuminate the object being photographed (when performing illumination correction, the object being photographed is a standard object with high consistency, such as a white balance cap or gauze) and facilitate the endoscope 11 in acquiring image signals. The image signal can be transmitted from the endoscope 11 to the image processor 12 for processing. In addition, the image processor 12 is connected to the light source control unit 105 via a serial port, which can transmit control signals and other specific information.
[0051] It should be noted that the embodiments of this application do not limit the specific image processor, light source control unit, light source unit, beam combining filter group and endoscope, nor do they limit the specific number of light source units, nor do they limit the communication control process between the above components. For details, please refer to the relevant technology of endoscope system.
[0052] In practical applications, it can be assumed that multi-wavelength light sources have undergone spectral calibration before leaving the factory. Therefore, even different multi-wavelength light sources can emit the same illumination spectrum under the same illumination mode, thus it can be considered that all multi-wavelength light sources have a consistent emission spectrum. However, what is uncontrollable during use is that when different endoscopes are connected to multi-wavelength light sources, due to structural position tolerances and assembly process limitations, the actual spectrum coupled to each endoscope from each light source may shift, leading to inconsistencies between the illumination spectrum emitted by each endoscope and the actual illumination spectrum emitted by the multi-wavelength light source. Furthermore, due to the differences in the transmission characteristics of the fiber bundle for illumination light of different wavelengths and the aging of the fiber bundle, even when the same endoscope is connected to the same multi-wavelength light source, the spectrum emitted by the endoscope may differ from the actual illumination spectrum transmitted to the endoscope. This inconsistency in the illumination spectrum emitted by the endoscope can ultimately lead to numerous problems such as abnormal endoscopic image colors, decreased contrast, and poor special lighting effects.
[0053] In view of this, this application provides an illumination correction method that addresses the problem of spectral shift caused by differences in endoscopes by correcting the relative spectra between each light source unit, thereby ensuring that the illumination spectrum emitted by the endoscope remains consistent.
[0054] Among them, the inventors of this application discovered through research that:
[0055] 1. The relative spectra of each light source remain consistent, which can be reflected in the stable image response ratio (specifically, the color / channel response ratio) between each light source in the image.
[0056] 2. The driving signals of each light source can directly affect the image response of each light source in the image acquired by the endoscope connected to it;
[0057] 3. The main reason for the shift in the illumination spectrum emitted from the endoscope is that the illumination light emitted from each light source has different attenuation in different endoscopes, and this attenuation can be considered fixed and is unrelated to specific illumination requirements.
[0058] Therefore, as long as under any identical lighting requirements (e.g., under the same imaging mode and the same image brightness level), by using appropriate adjustment coefficients to correct the original driving signals of each light source, the image response ratio between each light source in the response image of each endoscope to the standard object is the same as the image response ratio between each light source in the response image of the standard endoscope to the standard object, thus ensuring that the relative spectrum emitted by each endoscope remains consistent.
[0059] Based on the above inventive concept, in the embodiments of this application, a standard value for the image response ratio between each light source can be established in advance based on the response image of the standard endoscope taken by the standard endoscope on a standard object (such as a white balance cap, whiteboard, gauze, etc. with high imaging consistency) under preset illumination requirements.
[0060] To facilitate the establishment of the relative relationship of image responses among the various light sources, one light source can be selected as a reference light source, and the others as light sources to be calibrated. Using the reference light source as a benchmark (i.e., keeping the image response characteristics of the reference light source unchanged), a proportional standard value is pre-established between the image response characteristics of each light source to be calibrated and the image response characteristics of the reference light source. It is understood that, during the pre-setting of the proportional standard value stage, this embodiment does not limit the selection criteria for the reference light source; it can be any light source. In practical applications, a light source with good monochromaticity can be selected as the reference light source to better distinguish its main image response characteristics.
[0061] Specifically, the "preset lighting requirement" refers to the requirement that, in order to achieve a certain imaging mode (preferably an imaging mode including each color light source) and a certain image brightness level (e.g., medium brightness level), each light source emits light under a specified drive signal corresponding to the aforementioned imaging mode and image brightness level. For example, the "preset lighting requirement" can be: making Figure 1 The first light source 101 (designated as a reference light source) emits light under a specified driving signal I1, the second light source 102 (designated as a light source to be calibrated) emits light under a specified driving signal I2, the third light source 103 (designated as a light source to be calibrated) emits light under a specified driving signal I3, and the fourth light source 104 (designated as a light source to be calibrated) emits light under a specified driving signal I4. The specified driving signals I1 to I4 are related driving signals, and the specified driving signals I1 to I4 may be different from each other.
[0062] Furthermore, in practical applications, when illumination calibration is required for a particular endoscope to be calibrated, the illumination calibration method provided in the embodiments of this application can be used to determine the adjustment coefficients of each light source unit associated with the endoscope to be calibrated.
[0063] For details, please refer to Figure 2 , Figure 2 A flowchart of an illumination correction method provided in this application embodiment is shown. This method is applied to the aforementioned endoscope system, which includes a multi-wavelength light source and an endoscope. The method may include, but is not limited to:
[0064] S201. Determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source.
[0065] As described above, the light source unit to be calibrated is the light source unit that needs to be calibrated and adjusted, while the reference light source unit serves as the calibration benchmark for each light source unit to be calibrated. Before the calibration of all light source units is completed, the reference light source unit should remain fixed, that is, the reference light source unit cannot be replaced, and its corresponding drive signal should remain fixed, and the light output state should remain constant. This ensures that each light source unit to be calibrated has a unified comparison benchmark, which facilitates the adjustment of the relative spectrum between each light source.
[0066] It should be noted that the embodiments of this application do not limit the number of light sources to be calibrated that can be calibrated in a single calibration; there can be one or more. To reduce computational complexity and the complexity of setting related driving signals, a single calibration may only process one light source to be calibrated.
[0067] S202. Set the reference light source to emit light constantly in a specified illumination mode. At the same time, use multiple preset drive signals to light up the light source to be calibrated, and acquire the response image of the endoscope taken of the standard object each time it is lit.
[0068] It should be noted that the lighting mode here corresponds to the "preset lighting requirements" mentioned above. That is, the lighting mode refers to the illumination of each light source unit under a specified drive signal corresponding to the aforementioned imaging mode and image brightness level in order to achieve a certain imaging mode and image brightness level. In other words, the reference light source unit should be driven constantly using the drive signal corresponding to the lighting mode to achieve constant light output. It should be noted that the embodiments of this application do not limit the specific type of drive signal for each light source unit, such as voltage, current, control words, etc., and can be selected according to actual application requirements. The control word is a control quantity in serial port control word communication, used to control the light-emitting element of the built-in control board. Furthermore, in the embodiments of this application, the response image is the image information captured by the photosensitive element on the standard object when the light source unit is illuminated using a specific drive signal and the standard object is illuminated. In other words, this application uses image data to record the illumination status of the light source unit. Furthermore, this application acquires response images corresponding to multiple different preset driving signals. These images not only record the specific response changes of the light source to be calibrated under different driving conditions, but also, since the light source to be calibrated and the reference light source operate simultaneously when acquiring the response images, the images can further record the changes in the response ratio between the light source to be calibrated and the reference light source under different driving conditions, for subsequent fitting. It should be noted that the embodiments of this application do not limit the specific value and number of preset driving signals, which can be set according to actual application requirements. For ease of control, the preset driving signals can be pre-written into a sequence, and the light source to be calibrated can be sequentially illuminated using this sequence.
[0069] Furthermore, the embodiments of this application do not limit the specific type of the standard object, and can be a standard object with high consistency, such as a white balance cap or gauze. Since the white balance cap is a routine accessory for endoscopes and can effectively ensure the consistency of imaging, this application uses the white balance cap as the standard object.
[0070] S203. Generate a feature response ratio model using a preset driving signal and its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be calibrated and the image response features corresponding to the reference light source and the driving signal of the light source to be calibrated.
[0071] In this embodiment, the feature response ratio model is a model obtained by fitting the aforementioned preset driving signal and response image. It reflects the correspondence between the ratio of the image response features corresponding to the light source to be calibrated and the image response features corresponding to the reference light source and the driving signal of the light source to be calibrated. The purpose of setting this model is to determine the specific driving signal condition required when the energy ratio between the image response features corresponding to the light source to be calibrated and the image response features corresponding to the reference light source reaches a specific value. This is because the consistency of the illumination spectrum can be reflected in the stable image response ratio between each light source in the image. In other words, only by adjusting the energy ratio between the aforementioned image response features to a standard value can the aforementioned specific driving signal condition be determined, thus achieving the calibration effect on the light source to be calibrated. It should be noted that this embodiment does not limit the method used to fit the feature response ratio model; for example, linear fitting, polynomial fitting, piecewise linear fitting, etc., can be used. Furthermore, during the fitting process, the relationship between the preset driving signal and the response image can be fitted first to obtain an initial driving response model; then, based on the aforementioned two image response features, relevant parameters are extracted from the driving response model to construct the feature response ratio model, which can be adjusted according to actual application requirements.
[0072] S204. Using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be calibrated corresponding to the specified lighting mode, the adjustment coefficient is determined, so as to use the adjustment coefficient to correct the original drive signal of the light source to be calibrated in each lighting mode.
[0073] It should be noted that the preset ratio standard value is the standard energy ratio between the image response characteristics of the light source to be calibrated and the image response characteristics of the reference light source under the specified lighting mode. Since the driving signal of the light source to be calibrated is the independent variable in this model, while the ratio between the image response characteristics of the light source to be calibrated and the image response characteristics of the reference light source under different driving signal conditions is the dependent variable, the target driving signal of the light source to be calibrated can be calculated using the feature response ratio model and the preset ratio standard value. Then, the adjustment coefficient can be determined using the target driving signal and the original driving signal of the light source to be calibrated corresponding to the specified lighting mode.
[0074] Specifically, the process of determining the adjustment coefficient can be expressed as follows:
[0075] I_fix=F -1 (Std_response)
[0076] k = I_fix / I_now
[0077] Where I_fix represents the target drive signal, Std_response represents the preset proportional standard value, and F -1 (·) represents the inverse function of the characteristic response proportional model, I_now represents the original driving signal, and k represents the adjustment coefficient. It should be noted that the embodiments of this application do not limit the method of determining the inverse function; relevant technical content can be consulted.
[0078] Of course, if a single calibration can simultaneously calibrate multiple light sources to be calibrated, then the process of determining the above adjustment coefficient can be expressed as follows:
[0079] response = F(I1, I2, I3...I m )
[0080] I_FIX=[1_fix1, I_fix2, I_fix3...I_fix m ] = F -1 (Std_response)
[0081] K = I_FIX / I_now
[0082] Where I_FIX represents the target driving signal sequence, and response (i.e., F(·)) represents the characteristic response scaling model.
[0083] Furthermore, the shift in the illumination spectrum emitted from the endoscope lens is primarily due to the varying attenuation of the illumination light emitted from different light sources within different endoscopes. This attenuation can be considered fixed and independent of the specific illumination mode. Therefore, after obtaining the adjustment coefficient, this coefficient can be used to correct the original drive signal of the light source to be calibrated under each illumination mode. It is evident that since the calculation of the adjustment coefficient does not require special fixtures or optical power measurement equipment, and the illumination calibration process can be directly implemented within the existing endoscope system, the complexity of endoscope illumination calibration can be effectively reduced. Of course, after obtaining the adjustment coefficient, it can also be stored in the endoscope's memory. When the endoscope is reconnected to a multi-wavelength light source, the multi-wavelength light source automatically reads the adjustment coefficient and updates the drive signal of the corresponding light source to be calibrated under each illumination mode. This can be set according to actual application requirements.
[0084] Furthermore, it can be understood that in this embodiment, in order to simplify the amount of calculation, only one light source part to be calibrated is processed in a single calibration. In order to achieve calibration of each light source part of the endoscope, after determining the adjustment coefficient of the current light source part to be calibrated, the next light source part can be selected from the multi-wavelength light sources to enter the calibration process until the calculation and processing of the adjustment coefficients of other light source parts except the reference light source part are completed.
[0085] In one possible scenario, step S201 specifically involves: determining the reference light source section and any light source section to be calibrated in the multi-wavelength light source. Then, after determining the adjustment coefficient using the characteristic response scaling model, the preset scaling standard value, and the original drive signal of the light source section to be calibrated, the process may further include:
[0086] Step 21: Determine the next light source to be calibrated among the multi-wavelength light sources, and proceed to the step of illuminating the light source to be calibrated using multiple preset drive signals, until the adjustment coefficients of all light sources to be calibrated have been determined.
[0087] Furthermore, to ensure the reliability and effectiveness of the adjustment coefficient, an effectiveness detection process can be set up to detect the effectiveness of the adjustment coefficient. This application embodiment does not limit the specific process of effectiveness detection. For example, after calibration and adjustment, the deviation value between the response images acquired by the endoscope system to be calibrated and the standard endoscope system under the condition that all light source parts are illuminated through the same illumination mode can be detected, and the reliability of the adjustment coefficient can be determined based on whether the deviation value is less than a preset threshold. Alternatively, a first deviation value can be determined before calibration between the response images acquired by the endoscope and the standard endoscope under the condition that all light source parts are illuminated through the same illumination mode, and a second deviation value can be determined after calibration between the response images acquired by the endoscope and the standard endoscope under the condition that all light source parts are illuminated through the same illumination mode. The reliability of the adjustment coefficient can then be determined by determining whether the second deviation value is less than the first deviation value. In this application embodiment, to avoid the influence of improper preset threshold setting on effectiveness detection, the latter method can be adopted to perform effectiveness detection on the adjustment coefficient.
[0088] In one possible scenario, the lighting correction method may further include:
[0089] Step 31: Determine the target illumination mode, and simultaneously illuminate all light sources in the multi-wavelength light source using the original drive signal under the target illumination mode, and acquire the original response image of the endoscope on the standard object.
[0090] Step 32: Acquire a standard response image of a standard object taken by a standard endoscope system in target illumination mode, and determine the first deviation value between the original response image and the standard response image.
[0091] It should be noted that the target illumination mode is preferably an illumination mode that can simultaneously illuminate all light source parts in the multi-wavelength light source. The embodiments of this application do not limit the driving signals corresponding to each light source part in the target illumination mode, and can be set according to actual application requirements. Furthermore, if the multi-wavelength light source does not have an illumination mode capable of illuminating all light source parts, its multiple illumination modes can be combined, as long as it can ensure that the combined illumination mode covers all light source parts in the multi-wavelength light source.
[0092] Step 33: Use all adjustment coefficients to correct the original drive signal of the corresponding light source unit to be corrected, and obtain the corrected drive signal in the target illumination mode;
[0093] Step 34: Use the correction drive signal in the target illumination mode to illuminate all the light sources in the multi-wavelength light source, and at the same time acquire the correction response image of the endoscope on the standard object;
[0094] Step 35: Determine the second deviation value between the calibrated response image and the standard response image, and determine whether the second deviation value is less than the first deviation value; if yes, proceed to step 36; if no, determine that the calibration has failed and prompt a failure message.
[0095] Step 36: If yes, save all adjustment coefficients.
[0096] It should be noted that the standard endoscope system is a calibrated endoscope system, which can be used as a benchmark for evaluating the calibration of this endoscope. This application does not limit the indicators selected for calculating the deviation value; these indicators may include one or more indicators such as channel response ratio and color evaluation coefficients for each color space. If all adjustment coefficients are verified to be effective, all adjustment coefficients are stored in the endoscope's memory for automatic reading by multi-wavelength light sources.
[0097] For easier understanding, please refer to Figure 3 , Figure 3 This is a flowchart of a lighting correction control timing provided in an embodiment of this application.
[0098] Based on the above embodiments, this application can first determine the reference light source and the light source to be calibrated in the multi-wavelength light source, and adjust the reference light source to emit light at a constant rate under a specified illumination mode to ensure that the light output of the reference light source remains constant, which facilitates the adjustment of the relative spectrum between the light sources. Subsequently, this application will collect the actual response image data generated by the light source to be calibrated under different driving signal conditions and the reference light source with fixed light output, and generate a characteristic response ratio model using a preset driving signal and its response image. Through this characteristic response ratio model, the correspondence between the ratio of the unique image response features of the light source to be calibrated and the unique image response features of the reference light source and the driving signal of the light source to be calibrated can be established. Then, using this characteristic response ratio model, the target driving signal that can achieve the preset ratio standard value can be calculated. Based on the target driving signal and the original driving signal of the light source to be calibrated, which corresponds to the driving signal of the reference light source, the adjustment coefficient of the light source to be calibrated can be determined. Finally, based on the adjustment coefficient, the light source to be calibrated can be calibrated, so that the relative spectrum of the illumination light emitted from different endoscopes can be made consistent. Furthermore, the aforementioned illumination correction process can be directly implemented in existing endoscope systems without the need for special fixtures and optical power measurement equipment, thus effectively reducing the complexity of endoscope illumination correction.
[0099] Based on the above embodiments, the specific construction process of the feature response ratio model will be described in detail below. In one possible case, generating the feature response ratio model using a preset driving signal and its corresponding response image may include:
[0100] S301. Generate a driving response model using the color values of the preset driving signal and its corresponding response image.
[0101] In this embodiment, the driving response model reflects the correspondence between the driving signal of the light source to be corrected and the image response value. This model can be fitted using linear fitting, polynomial fitting, piecewise linear fitting, or other methods. Since polynomial fitting is a commonly used method, this embodiment employs this method for model fitting.
[0102] In one possible scenario, generating a driving response model using a preset driving signal and the color values of its corresponding response image can include:
[0103] The correlation between the color values of the preset driving signal and the response image is fitted using a polynomial fitting method to obtain the driving response model.
[0104] It should be noted that the embodiments of this application do not limit the color space corresponding to the response image. For example, it can be the RGB color space (Red, Green, Blue), or other color spaces such as the YUV color space (Y represents luminance, U represents hue, and V represents saturation). Therefore, the aforementioned color values can be RGB colors, YUV colors, or other types of colors. The following is a specific example illustrating the above fitting process. Multiple preset driving signals can be represented using the following sequence:
[0105] [I1 I2 I3 … I m ];
[0106] Where I i ,i∈[1,m] represents the i-th driving signal. The RGB response values of the corresponding response image of the driving signal can be represented by the image response sequence matrix:
[0107]
[0108] Among them, R i G i and B i This represents the RGB value corresponding to the i-th response image. The driving response model obtained by polynomial fitting of the above-mentioned preset driving signal and response sequence matrix can be expressed as:
[0109]
[0110] Among them, f R (I), f G (I), f B (I) represents the response functions of the three channels R, G, and B, where P refers to the series of the polynomial, kr p kg p ,kb pThese are the coefficients of each term, whose values are determined by fitting the actual measured values mentioned above. Among them, kr0, kg0, and kb0 are the channel values sensed by the image when only the reference light source is used for illumination.
[0111] S302. Based on the first response feature corresponding to the reference light source and the second response feature corresponding to the light source to be corrected, generate the first response function corresponding to the first response feature and the second response function corresponding to the second response feature using the parameters in the driving response model.
[0112] In this embodiment, the first response feature is an image response feature unique to the reference light source, and the second response feature is an image response feature unique to the light source to be calibrated. In other words, the response signals generated by the reference light source and the light source to be calibrated can be significantly distinguished according to the first and second response features. It should be noted that this embodiment does not limit the types of the first and second response features. For example, they can be color channel features or color features, where the color channels in the RGB color space are the R, G, and B color channels, and the color is the result of mixing these channels. Since each channel has an independent response function in the driving response model, when the first and second response features correspond to different color channels, the response function of the corresponding channel can be directly extracted from the above model as the specific response function corresponding to the response feature. It should be noted that this embodiment does not limit the specific way of selecting the first and second response features. For example, when the above response feature is a color channel feature, the main response channel of the reference light source and the light source to be calibrated can be selected; when the two main response channels overlap, the channel with the second highest response can be selected, and so on.
[0113] In one possible scenario, the first response feature and the second response feature each correspond to different color channels. Generating the first response function corresponding to the first response feature and the second response function corresponding to the second response feature using parameters from the driving response model can include:
[0114] Extract the first response function corresponding to the color channel of the first response feature and the second response function corresponding to the color channel of the second response feature from the driving response model.
[0115] S303. Generate a characteristic response ratio model using the first response function and the second response function.
[0116] Taking the scenario where the reference light source emits green light and the light source to be calibrated emits blue light as an example, in this case, it is necessary to divide the response functions of the blue and green channels of the image to obtain the driving sequence-image blue-green channel response ratio model (BG_RATIO, i.e., the feature response ratio model), which can be expressed as:
[0117]
[0118] Of course, if the first response feature and the second response feature correspond to other color channels, or if the first response feature and the second response feature correspond to different color features, the above feature response ratio model may have other forms and can be adjusted according to actual application needs.
[0119] Based on the above embodiments, the embodiments of this application can use a fitting method to determine the correlation between the preset driving signal and the response image, and then use the parameters in the corresponding driving response model to construct a first response function corresponding to the first response feature and a second response function corresponding to the second response feature. Finally, the first response function and the second response function can be used to construct a feature response ratio model to effectively fit the actual feature response ratio change of the light source to be corrected under different driving signal conditions and the reference light source.
[0120] The lighting correction device, electronic device, and computer-readable storage medium provided in the embodiments of this application are described below. The lighting correction device, electronic device, and computer-readable storage medium described below can be referred to in correspondence with the lighting correction method described above.
[0121] Please refer to Figure 4 , Figure 4 This is a structural block diagram of an illumination correction device provided in an embodiment of this application. The device is applied to an endoscope system, which includes a multi-wavelength light source and an endoscope. The device may include:
[0122] Initialization module 401 is used to determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source;
[0123] The drive module 402 is used to set the reference light source to emit light constantly in a specified illumination mode, and at the same time, to use multiple preset drive signals to light up the light source to be calibrated, and to acquire the response image of the endoscope to the standard object each time it is lit.
[0124] The model generation module 403 is used to generate a feature response ratio model using a preset driving signal and its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source and the driving signal of the light source to be corrected.
[0125] The adjustment coefficient calculation module 404 is used to determine the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be calibrated corresponding to the specified lighting mode, so as to use the adjustment coefficient to calibrate the original drive signal of the light source to be calibrated in each lighting mode.
[0126] Optionally, the standard object is a white balance cap.
[0127] Optionally, the initialization module 401 is specifically used to determine the reference light source section and any light source section to be calibrated in the multi-wavelength light source; the device may further include:
[0128] The traversal module is used to determine the next light source to be calibrated among the multi-wavelength light sources after the adjustment coefficient calculation module 404 determines the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be calibrated corresponding to the specified lighting mode. This next light source is then entered into the drive module 402 and the adjustment coefficient calculation module 404 for processing until the adjustment coefficients of all light sources to be calibrated have been determined.
[0129] Optionally, the device may further include:
[0130] The raw response image acquisition module is used to determine the target illumination mode, and at the same time use the raw drive signal under the target illumination mode to illuminate all the light source parts of the multi-wavelength light source, and acquire the raw response image of the endoscope on the standard object.
[0131] The first deviation value calculation module is used to acquire the standard response image of the standard object captured by the standard endoscope system in the target illumination mode, and to determine the first deviation value between the original response image and the standard response image.
[0132] The correction module is used to correct the original drive signal of the corresponding light source unit to be corrected using all adjustment coefficients, so as to obtain the corrected drive signal under the target illumination mode.
[0133] The calibration response image acquisition module is used to illuminate all the light sources of the endoscope using the calibration drive signal in the target illumination mode, and at the same time acquire the calibration response image of the endoscope on the standard object.
[0134] The second deviation value calculation and validity judgment module is used to determine the second deviation value between the corrected response image and the standard response image, and to determine whether the second deviation value is less than the first deviation value; if so, all adjustment coefficients are saved.
[0135] Optionally, the model generation module 403 may include:
[0136] The drive response model generation unit is used to generate a drive response model using a preset drive signal and the color values of its corresponding response image.
[0137] The response function generation unit is used to generate a first response function corresponding to a first response feature and a second response function corresponding to a second response feature using the parameters in the driving response model.
[0138] The feature response scaling model generation unit is used to generate a feature response scaling model based on the first response feature corresponding to the reference light source and the second response feature corresponding to the light source to be corrected, using the first response function and the second response function.
[0139] Optionally, the driving response model generation unit may include:
[0140] The driving response model generation sub-unit is used to fit the correlation between the color values of the preset driving signal and the response image using a polynomial fitting method to obtain the driving response model.
[0141] Optionally, the response function generation unit may include:
[0142] The extraction sub-unit is used to extract the first response function corresponding to the color channel of the first response feature and the second response function corresponding to the color channel of the second response feature from the driving response model.
[0143] See Figure 5 As shown in the figure, this application discloses an electronic device 20, including a processor 21 and a memory 22; wherein, the memory 22 is used to store a computer program; the processor 21 is used to execute the lighting correction method disclosed in the foregoing embodiment when executing the computer program.
[0144] The specific process of the above-mentioned lighting correction method can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.
[0145] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, and the storage method can be temporary storage or permanent storage.
[0146] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26; wherein, the power supply 23 is used to provide operating voltage for the various hardware devices on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0147] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the lighting correction method disclosed in the foregoing embodiments.
[0148] The specific process of the above-mentioned lighting correction method can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.
[0149] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0150] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0151] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0152] The above provides a detailed description of the lighting correction method, apparatus, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.
Claims
1. An illumination correction method applied to an endoscope system, the endoscope system comprising a multi-wavelength light source and an endoscope, characterized in that, The lighting correction method includes: Determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source; The reference light source is set to emit light constantly in a specified illumination mode. At the same time, the light source to be calibrated is lit up using multiple preset drive signals, and the response image of the endoscope to the standard object is acquired each time it is lit up. A feature response ratio model is generated using the preset driving signal and the color values of its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be calibrated and the image response features corresponding to the reference light source and the driving signal of the light source to be calibrated; the ratio of the image response features corresponding to the light source to be calibrated and the image response features corresponding to the reference light source is the color ratio or channel response ratio between the light source to be calibrated and the reference light source. An adjustment coefficient is determined using the characteristic response ratio model, a preset ratio standard value, and the original drive signal of the light source to be calibrated corresponding to the specified lighting mode. The adjustment coefficient is then used to correct the original drive signal of the light source to be calibrated in each lighting mode.
2. The lighting correction method according to claim 1, characterized in that, The standard object is a white balance cap.
3. The lighting correction method according to claim 1, characterized in that, Determining the reference light source section and the light source section to be calibrated in the multi-wavelength light source includes: Determine the reference light source section and any light source section to be corrected in the multi-wavelength light source; Then, after determining the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be corrected corresponding to the specified lighting mode, the lighting correction method further includes: The next light source to be calibrated in the multi-wavelength light source is determined, and the step of illuminating the light source to be calibrated using multiple preset driving signals is performed until the adjustment coefficients of all the light sources to be calibrated have been determined.
4. The lighting correction method according to claim 3, characterized in that, The lighting correction method further includes: The target illumination mode is determined, and the original driving signal under the target illumination mode is used to illuminate all the light source parts of the multi-wavelength light source, and the original response image of the endoscope to the standard object is acquired. Acquire a standard response image of the standard object captured by a standard endoscope system under the target illumination mode, and determine a first deviation value between the original response image and the standard response image; The original driving signal of the corresponding light source unit to be calibrated is corrected using all the aforementioned adjustment coefficients to obtain the calibrated driving signal under the target illumination mode; The correction drive signal in the target illumination mode is used to illuminate all the light source parts of the multi-wavelength light source, and at the same time, the correction response image of the endoscope to the standard object is acquired; Determine a second deviation value between the corrected response image and the standard response image, and determine whether the second deviation value is less than a first deviation value; If so, then save all the aforementioned adjustment coefficients.
5. The lighting correction method according to any one of claims 1 to 4, characterized in that, The step of generating a feature response ratio model using the preset driving signal and the color values of its corresponding response image includes: A driving response model is generated using the preset driving signal and the color values of the response image; Based on the first response feature corresponding to the reference light source and the second response feature corresponding to the light source to be corrected, the first response function corresponding to the first response feature and the second response function corresponding to the second response feature are generated using the parameters in the driving response model. The characteristic response ratio model is generated using the first response function and the second response function.
6. The lighting correction method according to claim 5, characterized in that, The step of generating a driving response model using the preset driving signal and the color values of the response image includes: The correlation between the preset driving signal and the color values of the response image is fitted using a polynomial fitting method to obtain the driving response model.
7. The lighting correction method according to claim 5, characterized in that, The first response feature and the second response feature each correspond to different color channels. The step of generating the first response function corresponding to the first response feature and the second response function corresponding to the second response feature using the parameters in the driving response model includes: The first response function corresponding to the color channel of the first response feature and the second response function corresponding to the color channel of the second response feature are extracted from the driving response model.
8. An illumination correction device for use in an endoscope system, the endoscope system comprising a multi-wavelength light source and an endoscope, characterized in that, The lighting correction device includes: An initialization module is used to determine the reference light source section and the light source section to be calibrated in the multi-wavelength light source. The driving module is used to set the reference light source to emit light constantly in a specified illumination mode, and at the same time, to illuminate the light source to be calibrated using multiple preset driving signals, and to acquire the response image of the endoscope to the standard object each time it is illuminated. The model generation module is used to generate a feature response ratio model using the preset driving signal and the color values of its corresponding response image; the feature response ratio model is used to reflect the correspondence between the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source and the driving signal of the light source to be corrected; the ratio of the image response features corresponding to the light source to be corrected and the image response features corresponding to the reference light source is the color ratio or channel response ratio between the light source to be corrected and the reference light source; The adjustment coefficient calculation module is used to determine the adjustment coefficient using the characteristic response ratio model, the preset ratio standard value, and the original drive signal of the light source to be corrected corresponding to the specified lighting mode, so as to use the adjustment coefficient to correct the original drive signal of the light source to be corrected in each lighting mode.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the lighting correction method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the lighting correction method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Optical scanning-type endoscope device
CN107072469A
Endoscope apparatus
CN108014424A
