Color calibration method, apparatus and equipment for medical endoscopes
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- RONGFENG (JIANGSU) MEDICAL TECHNOLOGY CO LTD
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-26
Smart Images
Figure CN122074882A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical image processing technology, and in particular to a color calibration method, apparatus and device for medical endoscope equipment. Background Technology
[0002] In medical endoscopic examinations and diagnoses, the color accuracy of images is directly related to the doctor's judgment of lesion sites. For example, the identification of mucosal lesions and abnormal blood vessel distribution all depend on the true and accurate color presentation of endoscopic images. To ensure the consistency of image colors across different medical endoscopic devices and under different usage environments, color calibration algorithms are required to calibrate the images acquired by the devices.
[0003] Traditional medical endoscope color calibration methods typically optimize by minimizing color differences when solving calibration parameters. While this method can improve color consistency, the calibrated images may have problems such as blurred boundaries between lesions and normal tissues and low detail recognition. This can seriously affect doctors' observation of image details and make it difficult to provide doctors with reliable and clear visual evidence. For example, it may cause small lesions to be masked by noise or difficult to identify due to insufficient contrast, thus interfering with clinical diagnosis and even causing diagnostic errors. Summary of the Invention
[0004] In view of this, the present application provides a color calibration method, apparatus, and device for medical endoscopes to solve at least one problem existing in the background art.
[0005] In a first aspect, embodiments of this application provide a color calibration method for a medical endoscope device, the method comprising: A target function containing a first component and a second component that are negatively correlated is constructed for the target medical endoscope device. The first component is used to characterize the color difference between the calibration color chart image and the reference color chart image, and the second component is used to compensate for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope device through a color calibration matrix. The objective function is iteratively optimized using the color calibration matrix as the optimization variable to solve for the optimal color calibration matrix; The original medical images acquired by the target medical endoscope are color-calibrated using the optimal color calibration matrix.
[0006] In some embodiments, the objective function is iteratively optimized using the color calibration matrix as the optimization variable to solve for the optimal color calibration matrix, including: Initialize a color calibration matrix as the current color calibration matrix; The original color chart image is color-transformed using the current color calibration matrix to generate the current calibration color chart image; Based on the current calibration color chart image and the reference color chart image, calculate the current value of the first component in the objective function, and based on the color data of multiple color patches in the current calibration color chart image, calculate the current value of the second component in the objective function; The current value of the objective function is calculated based on the current values of the first component and the second component. Update the current color calibration matrix based on the current value of the objective function; The process of generating the current calibration color chart image and updating the current color calibration matrix is repeated until a preset optimization termination condition is met. The color calibration matrix that meets the optimization termination condition is then taken as the optimal color calibration matrix.
[0007] In some embodiments, calculating the current value of the second component in the objective function based on the color data of multiple color patches in the current calibration color chart image includes: Based on the color data of multiple color patches in the current calibration color chart image, calculate the average color value of each color patch in at least one color channel; Based on the color mean, the color distribution dispersion corresponding to each color channel is calculated; The current value of the second component is calculated based on the color distribution dispersion corresponding to each color channel.
[0008] In some embodiments, the color distribution dispersion is the variance or standard deviation of the color mean of each color channel, and the current value of the second component is the average variance or average standard deviation of the plurality of color channels.
[0009] In some embodiments, the second component is introduced into the objective function by the reciprocal of the second component or by a decreasing function with respect to the second component; the value of the objective function is positively correlated with the value of the first component and negatively correlated with the value of the second component.
[0010] In some embodiments, constructing a target function for the target medical endoscope device, comprising a first component and a second component that are negatively correlated, includes: Based on the selected clinical observation mode of the target medical endoscope, corresponding weight coefficients are assigned to the first component and the second component in the objective function, and the first component and the second component are weighted and calculated based on the configured weight coefficients to construct the objective function. The clinical observation modes include tissue detail enhancement mode, color fidelity mode, and standard balance mode; In the organization detail enhancement mode, the weight coefficient assigned to the second component is higher than the weight coefficient assigned to the second component in the color fidelity mode, and / or, the weight coefficient assigned to the first component is lower than the weight coefficient assigned to the first component in the color fidelity mode. In the standard balancing mode, the first component and the second component are assigned equal weighting coefficients.
[0011] In some embodiments, the step of color calibration of the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix includes: Based on the target clinical observation mode selected for the original medical image, the optimal color calibration matrix corresponding to the target clinical observation mode is invoked to perform color transformation on the original medical image.
[0012] In some embodiments, the optimization algorithm used in the iterative optimization is gradient descent, least squares, or genetic algorithm.
[0013] In some embodiments, the method further includes: The original medical image is preprocessed to obtain a preprocessed image as an image to be calibrated. The preprocessing includes: enhancing the pixel values of pixels in the original medical image that meet preset conditions, and keeping the pixel values of pixels that do not meet the preset conditions unchanged. The preset conditions include: the pixel value of a pixel is higher than the mean of its local neighborhood, and the difference between the maximum and minimum pixel values in its local neighborhood is higher than a preset screening threshold. The step of performing color calibration on the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix includes: The optimal color calibration matrix is used to perform color calibration on the image to be calibrated to obtain the calibrated image.
[0014] Secondly, embodiments of this application provide a color calibration device for medical endoscope equipment, the device comprising: A construction module is used to construct an objective function for a target medical endoscope device, which includes a first component and a second component that are negatively correlated. The first component is used to characterize the color difference between the calibration color chart image and the reference color chart image, and the second component is used to compensate for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope device through a color calibration matrix. An optimization module is used to iteratively optimize the objective function with the color calibration matrix as the optimization variable in order to solve for the optimal color calibration matrix; The calibration module is used to perform color calibration on the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix.
[0015] Thirdly, embodiments of this application provide a medical endoscope device, including a processor, the processor being configured to invoke instructions to cause the medical endoscope device to perform the color calibration method for a medical endoscope device as described in any of the first aspects.
[0016] Fourthly, embodiments of this application provide a storage medium having an executable program stored thereon, wherein the executable program, when executed by a processor, implements the color calibration method for a medical endoscope device as described in any of the first aspects.
[0017] This application provides a color calibration method, apparatus, and device for medical endoscopes. It constructs an objective function for the target medical endoscope, comprising a first component and a second component that are negatively correlated. The first component represents the color difference between a calibration color chart image and a reference color chart image, while the second component compensates for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope using a color calibration matrix. The objective function is iteratively optimized using the color calibration matrix as an optimization variable to solve for the optimal color calibration matrix. Because the two components are negatively correlated, this iterative optimization process reduces color error while simultaneously improving color contrast. This ensures that the endoscope image calibrated using the optimal color calibration matrix effectively improves the color contrast between different tissue regions (such as lesions and normal tissues) while maintaining color consistency with the real scene (i.e., meeting the color accuracy requirements for medical diagnosis). This results in clearer boundary contours and enhanced visibility of minute details, effectively alleviating the image blurring and reduced detail recognition problems caused by traditional color calibration methods that solely pursue minimizing color differences. Attached Figure Description
[0018] Figure 1 This is a schematic flowchart of a color calibration method for a medical endoscope device provided in an embodiment of this application.
[0019] Figure 2 This is a schematic flowchart of a color calibration method for a medical endoscope device provided in another embodiment of this application.
[0020] Figure 3 This is a schematic flowchart of a color calibration method for a medical endoscope device provided in another embodiment of this application.
[0021] Figure 4 This is a schematic diagram of the structure of a color calibration device for medical endoscope equipment provided in an embodiment of this application. Detailed Implementation
[0022] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.
[0023] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.
[0024] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.
[0025] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0026] Figure 1 This is a schematic flowchart illustrating a color calibration method for a medical endoscope device according to an embodiment of this application. Figure 1 As shown, the color calibration method includes steps S101 to S103.
[0027] S101: Construct an objective function for the target medical endoscope device that includes a first component and a second component that are negatively correlated. The first component is used to characterize the color difference between the calibration color chart image and the reference color chart image, and the second component is used to compensate for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope device through a color calibration matrix.
[0028] In this embodiment, the entity executing the color calibration method can be a processing unit integrated into the target medical endoscope device itself, or a terminal device connected to the target medical endoscope device via wired or wireless connection. The terminal device can be, for example, a computer with image processing capabilities, an embedded processor, or a cloud server.
[0029] The target medical endoscope can be any type of medical endoscope, such as an electronic gastroscope, colonoscope, laparoscope, or bronchoscope.
[0030] The original color chart image is the original image obtained by the image sensor of the target medical endoscope device capturing a standard color chart (e.g., a standard 24-color chart or other standard color chart specifications).
[0031] In some examples, to reduce noise interference not related to color calibration, the original color chart image acquired by the target medical endoscope device can be preprocessed, such as removing lens distortion, white balance correction, and dark current noise elimination. The preprocessed original color chart image is then used for iterative optimization of the objective function.
[0032] The calibration color chart image refers to the image obtained by color transformation of the original color chart image using the color calibration matrix of the current iteration in each optimization iteration.
[0033] The reference color chart image represents an ideal image of color reproduction, for example, an image of the same standard color chart captured under ideal conditions by a rigorously calibrated reference imaging system.
[0034] The first component is used to quantify the color difference between the calibration color chart image and the reference color chart image. For example, the CIE 1976 LAB color difference formula in the CIE LAB color space can be used to calculate the color difference between each color patch in the calibration color chart image (e.g., the 24 color patches in a standard 24-color chart) and the corresponding color patch in the reference color chart image, and the average value of the color differences of all color patches is taken as the current value of the first component.
[0035] The second component is used to characterize the color contrast between different color patches in the same calibration color chart image, i.e., color discrimination. Color contrast reflects the degree of dispersion between different color regions in the calibration color chart image. The larger the value, the more obvious the color distinction in the image and the more prominent the details.
[0036] In this embodiment, the first component and the second component in the objective function are negatively correlated, which can guide the iterative optimization process of the objective function to take into account both reducing color difference and enhancing color contrast when solving for the optimal color calibration matrix.
[0037] In some examples, it is possible to mathematically construct a second component that contributes to the function value in a way that is opposite to that of the first component.
[0038] In some examples, the second component is introduced into the objective function by either the reciprocal of the second component or a decreasing function with respect to the second component; the value of the objective function is positively correlated with the value of the first component and negatively correlated with the value of the second component.
[0039] In a specific example, the second component is introduced into the objective function in the form of its reciprocal, for example, the objective function can be constructed as: Where J represents the value of the objective function, C represents the first component (i.e., color difference), and D represents the second component (i.e., color contrast). and The positive coefficients used to balance the contributions of the first and second components to the objective function (where, It can be set to 1). In this objective function, minimizing J will drive C to decrease (more accurate colors) and simultaneously drive D to increase (enhanced contrast).
[0040] In a specific example, the second component is introduced into the objective function as a decreasing function; for example, the objective function can be constructed as: or ;in, v is a positive coefficient, and e is a natural constant.
[0041] S102: Iteratively optimize the objective function using the color calibration matrix as the optimization variable to solve for the optimal color calibration matrix.
[0042] The color calibration matrix can be, for example, a 3×3 matrix, used to perform a linear transformation on the RGB pixel values of the original image. The optimization variables are the values of each element in this matrix.
[0043] In some examples, the optimization algorithm used may be gradient descent, least squares, or genetic algorithm, etc.
[0044] For example, when using gradient descent, after initializing a color calibration matrix, in each iteration, the original color chart image is calibrated using the current color calibration matrix to generate a calibration color chart image. The current values of the first and second components are calculated using the calibration and reference color chart images, and then substituted into the objective function to obtain the objective function value. Then, based on the objective function value, the matrix elements of the current color calibration matrix are updated along the inverse direction of the gradient according to a set learning rate. Through multiple iterations, the objective function value converges to its minimum value or satisfies a preset termination condition; the corresponding color calibration matrix at this point is the optimal color calibration matrix.
[0045] S103: Use the optimal color calibration matrix to perform color calibration on the raw medical images acquired by the target medical endoscope.
[0046] The optimal color calibration matrix can be used to linearly transform the RGB values of each pixel in the original medical image to output the calibrated image.
[0047] For example, by constructing a vector from the RGB values of each pixel in the original medical image and multiplying it with the optimal color calibration matrix, the calibrated pixel values can be obtained, outputting the final color-calibrated image. This calibration process can be represented as: ,in, The optimal color calibration matrix is 3×3. These are the original pixel values. These are the calibrated pixel values.
[0048] Understandably, after obtaining the optimal color calibration matrix through the aforementioned steps S101 to S102, it can be stored for subsequent use. Thus, when performing color calibration on the raw medical images subsequently acquired by the target medical endoscope, the stored optimal color calibration matrix can be directly called for processing, without needing to recalculate it before each color calibration.
[0049] In the aforementioned color calibration method for medical endoscopes, an objective function containing a negatively correlated first component and a second component is constructed for the target medical endoscope. The first component represents the color difference between the calibration color chart image and the reference color chart image, while the second component compensates for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope using a color calibration matrix. The objective function is iteratively optimized using the color calibration matrix as the optimization variable to solve for the optimal color calibration matrix. Since the two components are negatively correlated, this iterative optimization process reduces color error while also improving color contrast. This allows the endoscope image calibrated using the optimal color calibration matrix to effectively improve the color contrast between different tissue regions (such as lesions and normal tissues) while ensuring color consistency between the calibrated image and the real scene (i.e., meeting the color accuracy requirements for medical diagnosis). This results in clearer boundary contours and enhanced visibility of minute details, effectively alleviating the problems of image blurring and decreased detail recognition caused by the traditional color calibration method's sole pursuit of minimizing color differences.
[0050] In some embodiments, such as Figure 2 As shown, the step S102 above, which uses the color calibration matrix as the optimization variable to iteratively optimize the objective function in order to solve for the optimal color calibration matrix, may include steps S201 to S207.
[0051] S201: Initialize a color calibration matrix as the current color calibration matrix.
[0052] The initialized color calibration matrix can be an identity matrix, an empirically preset matrix, or a randomly generated numerical matrix. For example, a 3×3 identity matrix can be initialized. As the current color calibration matrix.
[0053] S202: Perform color transformation on the original color chart image using the current color calibration matrix to generate the current calibration color chart image. The color calibration matrix M of the current iteration (M is the initial matrix M0 in the first iteration) can be used to perform color transformation on the original color chart image acquired by the target medical endoscope. For each pixel in the original color chart image, its RGB value vector [R, G, B] is... T Multiply by matrix M on the left to obtain the transformed RGB values, which is the current calibration color chart image.
[0054] S203: Based on the current calibration color chart image and the reference color chart image, calculate the current value of the first component in the objective function, and based on the color data of multiple color patches in the current calibration color chart image, calculate the current value of the second component in the objective function.
[0055] The calculation process for the current value of the first component may include: extracting the pixel values of N color patches in the current calibration color chart image (e.g., 24 color patches in the current calibration color chart image of a 24-color chart), and the reference color values of the corresponding N color patches in the reference color chart image; and using the CIE 1976 LAB color difference formula to calculate the color difference of each color patch in the current calibration color chart image and the reference color chart image. The average value of the color difference of the N color blocks is used as the current value of the first component.
[0056] The calculation of the current value of the second component may include: calculating the pixel average of N color patches in the current calibration color chart image across the R, G, and B channels respectively. For each color channel, the variance (or standard deviation) of the sequence formed by these N averages is calculated. For example, the variance of the R channel is calculated from the R mean of the N color patches. The average of the variances of the three channels can be used as the current value of the second component. The larger this current value, the higher the color contrast of the image. In other examples, the second component can also be calculated based on the statistical properties of other color spaces (such as the a and b components of the Lab space), as long as it can effectively reflect the dispersion of color values.
[0057] S204: Calculate the current value of the objective function based on the current values of the first component and the second component.
[0058] Substitute the current values of the first component and the second component obtained in step S203 into the pre-constructed objective function to calculate the objective function value for the current iteration.
[0059] S205: Update the current calibration matrix based on the current value of the objective function.
[0060] Based on the current value of the objective function, the parameters of the current color calibration matrix are adjusted to generate a new color calibration matrix that minimizes the objective function value. Since the optimization objective is to minimize the objective function value, the current value of the objective function directly represents the degree to which the current color calibration matrix deviates from the ideal optimal solution.
[0061] To generate a new matrix that reduces the objective function value, thus driving the iterative process closer to the optimal solution, parameter updates are performed according to the chosen optimization algorithm. For example, using gradient descent, the gradient of the objective function with respect to each element of the current color calibration matrix is calculated, and the elements of the current color calibration matrix are updated in the reverse direction of the gradient with a predetermined step size (learning rate). Using a genetic algorithm, the current color calibration matrix can be encoded as an individual, its fitness calculated based on the current value of the objective function, and new candidate matrix populations generated through selection, crossover, and mutation. Using least squares, more efficient parameter update strategies can be constructed using first- or second-order information about the objective function with respect to matrix elements, accelerating convergence or avoiding local optima.
[0062] S206: Determine whether the preset optimization termination condition is met. If not, return to steps S202 to S205. If met, proceed to step S207.
[0063] The optimization termination condition may include at least one of the following: the current change in the objective function value (e.g., the change in the current value compared to the previous value) is less than a preset positive number (e.g., 10). -6 The current iteration count has reached the preset maximum iteration count; the current value of the objective function has fallen below the expected performance threshold.
[0064] S207: The color calibration matrix that meets the optimization termination condition is taken as the optimal color calibration matrix.
[0065] In this embodiment, during the iterative optimization of the objective function, a calibration image is generated based on the current color calibration matrix in each iteration, and its color difference and contrast are calculated to evaluate the objective function. Then, the color calibration matrix is updated according to the objective function value, so that the color calibration matrix is continuously optimized in the direction of reducing color difference and improving contrast, thereby efficiently and reliably converging to the optimal color calibration matrix that meets the termination condition.
[0066] In some embodiments, such as Figure 3 As shown, step S203 above, which calculates the current value of the second component in the objective function based on the color data of multiple color patches in the current calibration color chart image, may include the following steps: S301: Based on the color data of multiple color patches in the current calibration color chart image, calculate the average color value of each color patch in at least one color channel; S302: Calculate the color distribution dispersion for each color channel based on the color mean; S303: Calculate the current value of the second component based on the color distribution dispersion corresponding to each color channel.
[0067] For example, for each of the N (N is a positive integer) specified color patches in the current calibration color chart image (e.g., corresponding to 18 color patch areas in a standard 24-color chart), the central image region of each patch can be selected as the region of interest (e.g., a 100×100 pixel region surrounding the center of the patch). The mean pixel value of all pixels within this region is calculated for each color channel in at least one color channel to obtain the color mean. The at least one color channel may include the R channel, G channel, and B channel.
[0068] The color distribution dispersion corresponding to each color channel can characterize the degree of dispersion of the average color distribution of N color blocks in each color channel.
[0069] In some embodiments, the color distribution dispersion is the variance or standard deviation of the color mean for each color channel, and the current value of the second component is the average variance or average standard deviation of the multiple color channels.
[0070] For example, for the i-th color patch, calculate the average pixel value of all pixels in its region of interest across the R, G, and B color channels, denoted as . , , From this, we can obtain the mean sequence of N color patches in the R channel { , ,..., }, in the mean sequence of channel G { , ,..., }, in the mean sequence of channel B { , ,..., Based on the mean color of N color patches in each color channel of the current calibration color chart image, calculate the variance of the mean sequence for each channel.
[0071] For example, taking the R channel as an example, the variance of the mean sequence of this channel... It can be calculated using the following formula: ;in, It is the overall color average of the N color patches in the R channel. Alternatively, the standard deviation can be used as a measure of the dispersion of the color distribution. , , ;in , , These are the standard deviations for the R, G, and B color channels, respectively.
[0072] The larger the variance or standard deviation of any color channel, the greater the difference in color values between different color blocks on that color channel, meaning that the color channel contributes a higher color contrast.
[0073] In this embodiment, the current value of the second component is calculated by statistically analyzing the dispersion of the average color values of different color patches in each color channel of the current calibration color chart image. This ensures that the current value of the second component accurately reflects whether there is good distinguishability between different color regions in the image after processing by the current color calibration matrix. Introducing the value of the second component, calculated in this way, into the objective function that is negatively correlated with the first component representing color difference guides the optimization algorithm to actively seek a color calibration matrix that improves the overall color contrast of the image while enhancing color reproduction accuracy. This achieves a balance between color realism and detail prominence in the final calibrated medical image, which is beneficial for enhancing the visual separability between diseased and normal tissues.
[0074] In some embodiments, step S101 above, constructing a target function for the target medical endoscope device that includes a first component and a second component that are negatively correlated, may include: Based on the clinical observation mode selected by the target medical endoscope, corresponding weight coefficients are assigned to the first and second components of the objective function, and the first and second components are weighted and calculated based on the configured weight coefficients to construct the objective function.
[0075] The clinical observation modes include tissue detail enhancement mode, color fidelity mode, and standard balance mode.
[0076] In the organization detail enhancement mode, the weight coefficient assigned to the second component is higher than that assigned to the second component in the color fidelity mode, and / or, the weight coefficient assigned to the first component is lower than that assigned to the first component in the color fidelity mode; in the standard balance mode, the first component and the second component are assigned equal weight coefficients.
[0077] In this embodiment, each clinical observation mode corresponds to a different weighting strategy to adapt to the different emphases of image calibration in specific clinical application scenarios. In the standard balanced mode, the first component and the second component are assigned equal or similar weight coefficients. In the color fidelity mode, a weighting strategy opposite to that of the tissue detail enhancement mode is adopted. The optimization process in the color fidelity mode focuses on minimizing color differences to ensure that the color of the calibrated image is highly consistent with the true color.
[0078] For example, if the objective function is constructed as In standard balance mode, the balance coefficients α and β can be set to be equal or their ratio close to 1. This mode achieves a balance between color accuracy and detail enhancement, and the optimized matrix generates an image with reliable overall performance. In tissue detail enhancement mode, the optimized matrix generates an image with more prominent contrast and sharper details, which helps in the detection of minute lesions.
[0079] Compared to the color fidelity mode, the weighting coefficients related to the second component in the objective function can be increased in the tissue detail enhancement mode. The value, or increase The ratio of the weighting coefficient α related to the first component to the overall weighting coefficient β in the objective function makes the optimization process more inclined to improve the color contrast of the image. For example, it allows for a slight relaxation of the requirements for color reproduction accuracy within an acceptable range in exchange for a more significant contrast improvement. In color fidelity mode, the weighting coefficient assigned to the first component can be higher than that assigned to the second component by increasing the value of the weighting coefficient α related to the first component in the objective function, or by increasing the ratio of α to the weighting coefficient β related to the second component.
[0080] In this embodiment, by introducing a target function construction mechanism that allows for flexible configuration of weight coefficients based on clinical observation modes, the color calibration method possesses high scene adaptability. Users or the system can select the appropriate mode according to the specific diagnostic task or observation needs, thereby generating a targeted optimal color calibration matrix.
[0081] In some embodiments, step S103 above, which involves color calibration of the original medical image acquired by the target medical endoscope using an optimal color calibration matrix, may include: Based on the target clinical observation mode selected for the original medical image, the optimal color calibration matrix corresponding to the target clinical observation mode is invoked to perform color transformation on the original medical image.
[0082] The target clinical observation mode includes one of the following: tissue detail enhancement mode, color fidelity mode, and standard balance mode.
[0083] For the same endoscopic device, the optimal color calibration matrix can be pre-stored for different clinical observation modes.
[0084] For example, from a plurality of pre-stored optimal color calibration matrices, the optimal color calibration matrix corresponding to the target clinical observation mode is called, and the original medical image is color transformed using the called optimal color calibration matrix to generate and output the calibrated medical image; wherein, each of the optimal color calibration matrices is obtained by optimizing and iterating the objective function according to the weight coefficient configuration matched with each clinical observation mode.
[0085] In this embodiment, by binding the optimal color calibration matrix to the specific clinical observation mode of the target medical endoscope, when the target medical endoscope is used for medical image calibration, the optimal color calibration matrix matching the specific observation mode can be invoked to perform color calibration of the medical image. This ensures that the output image best meets the needs of specific diagnostic scenarios, such as prioritizing boundary contrast in tissue detail enhancement mode or prioritizing color fidelity in color fidelity mode. This effectively improves the specificity and clinical applicability of image color calibration.
[0086] In some embodiments, prior to performing step 103 above, the method may further include: The original medical image is preprocessed to obtain a preprocessed image as the image to be calibrated. The preprocessing includes: enhancing the pixel values of pixels in the original medical image that meet preset conditions, while keeping the pixel values of pixels that do not meet the preset conditions unchanged. The preset conditions include: the pixel value of a pixel is higher than the mean of its local neighborhood, and the difference between the maximum and minimum pixel values in its local neighborhood is higher than a preset screening threshold.
[0087] Step 103 above uses the optimal color calibration matrix to perform color calibration on the original medical image acquired by the target medical endoscope device, which may include: using the optimal color calibration matrix to perform color calibration on the image to be calibrated to obtain the calibrated image.
[0088] The inventors discovered that the human visual system is extremely sensitive to high-frequency signals (such as edges, textures, and other details) in images, while its perception of low-frequency signals (such as gently changing backgrounds) is relatively weak. Therefore, if the original medical image is directly color-calibrated using the optimal color calibration matrix to improve color reproduction and color contrast, the low-frequency components in the image may interfere with the high-frequency signals, reducing the visual saliency of key details. Therefore, in this embodiment, targeted image preprocessing is performed before color calibration of the original medical image acquired by the target medical endoscope device. This effectively enhances high-frequency details and suppresses low-frequency interference, providing a higher-quality input image for subsequent color calibration. As a result, a medical image with realistic colors and clearer details can be obtained after color calibration, making it easier for doctors to detect minute lesions that are difficult to detect with the naked eye.
[0089] In this embodiment, for each pixel in the original medical image, its local neighborhood refers to a rectangular pixel window (e.g., 5×5 pixels) centered on that pixel. The pixel values of all pixels within its local neighborhood are calculated, along with the mean pixel value of the local neighborhood and the difference between the maximum and minimum pixel values within the local neighborhood.
[0090] To select pixels that need enhancement from the entire image, each pixel needs to be evaluated to determine if it meets preset conditions: whether its pixel value is higher than the average pixel value of its local neighborhood, and whether the difference between the maximum and minimum pixel values in its local neighborhood is higher than a preset filtering threshold. The filtering threshold can be the value obtained by multiplying the average difference between the maximum and minimum pixel values in all local neighborhoods of the original medical image by a preset adjustment coefficient (e.g., 1.1).
[0091] The enhancement of pixel values of pixels meeting preset conditions in the original medical image may include: for each pixel meeting the preset conditions, amplifying the difference between the pixel value of the target pixel and the mean of its local neighborhood based on the magnification factor corresponding to the target pixel, and adding the amplified result to the original pixel value of the target pixel to obtain the enhanced pixel value. The magnification factor of a pixel can be negatively correlated with the standard deviation of the pixel values in its local neighborhood. For example, the ratio of a preset factor to the standard deviation of the pixel values in its local neighborhood can be used as the magnification factor of the pixel. The preset factor is a positive number greater than 0 and less than 1. In this way, if the pixel is located in an area with very rich details, the enhancement amplitude will be moderately suppressed, thereby preventing excessive enhancement that leads to information distortion. In practical applications, to avoid the magnification factor being too large, it is usually limited to a reasonable preset range.
[0092] Next, the color calibration method for medical endoscopes provided in this application will be further explained with specific examples.
[0093] This application provides a color calibration algorithm for medical endoscopes, which can solve the problem of insufficient color contrast that may occur after traditional color calibration algorithms calibrate medical endoscope images. This algorithm improves the color accuracy and color contrast of medical endoscope images, providing more reliable image data for clinical diagnosis.
[0094] The color calibration method may include the following steps: Step S11: Calibration preparation.
[0095] Obtain and configure the equipment and materials required to perform color calibration.
[0096] The medical endoscope to be calibrated (such as a gastroscope, colonoscope, or laparoscope); the standard color chart used is a standard 24-color chart (compliant with ISO 17321-1 standard, and the true RGB and LAB values of each color block on the color chart have been pre-calibrated); the main body for executing the method is an image acquisition and processing system, which may include an image sensor connected to the endoscope, a data transmission module, and a computer platform for running algorithm software (e.g., an environment based on MATLAB or Python OpenCV).
[0097] Step S12: Image preprocessing.
[0098] Place the standard 24-color chart within the field of view of the medical endoscope, adjust the endoscope's focus and light source intensity to ensure the color chart image is clear and unobstructed; then acquire the original image of the 24-color chart using the image acquisition system. The original image is preprocessed (such as removing lens distortion, white balance correction, dark current noise elimination, etc., to avoid noise interference not related to color calibration).
[0099] Step S13: Optimize the construction of the objective function.
[0100] Construct the objective function, for example, the objective function expression is: Where J is the total optimization cost of the objective function; The color difference term in the objective function (i.e., the first component in the above embodiment) is used to measure the calibrated 24-color chart image. Image of a real 24-color chart The color difference between them can be expressed using the CIE 1976 LAB color difference formula. Or, calculations based on metrics such as Euclidean distance in the RGB color space; The color contrast compensation term (i.e., the second component in the above embodiments) is used to measure the calibrated 24-color chart image. Color contrast. Image color contrast can be defined as the standard deviation between the color means of 18 color patches in a 24-color chart image after color calibration. For example, first calculate the R, G, and B color means of the 18 color patches, then calculate the variance (or standard deviation) of the color means of each color channel on the 18 color patches, and finally calculate the average variance (or average standard deviation) of the standard deviations of the three channels as the evaluation index of color contrast. Alternatively, other objective evaluation indexes that can assess color contrast can also be used.
[0101] The weighting factor for the image contrast compensation term (value range: 0 < <1), used to balance the priority of color calibration accuracy and contrast control; for example, when When the color value approaches 1, the algorithm focuses more on minimizing color differences; when... When the value approaches 0, the algorithm focuses more on enhancing color contrast (it is necessary to avoid excessive suppression that could cause color calibration failure).
[0102] Step S14: Solving for the color calibration matrix. Using the newly constructed composite objective function J as the optimization objective, the optimal 3×3 color calibration matrix is obtained by employing optimization algorithms such as gradient descent, least squares, or genetic algorithm.
[0103] For example, with To achieve the objective, gradient descent is used to iteratively optimize the elements of a 3×3 calibration matrix M: in each iteration, the element values of matrix M are adjusted, and the value of the objective function J is updated; when the number of iterations reaches a preset threshold (e.g., 1000 times), or the change in the objective function J is less than 10... -6 When the convergence condition is met, the iteration stops and the optimal 3×3 calibration matrix is obtained.
[0104] Step S15: Medical endoscope image calibration application.
[0105] The obtained 3×3 color calibration matrix is applied to the original medical images acquired by the medical endoscope, and a linear transformation is performed on the RGB color values of each pixel in the original medical image (i.e., Where M is a 3×3 calibration matrix, These are the original pixel values. (This refers to the calibrated pixel values), completing the image color calibration. However, due to the contrast compensation, the color differences in the calibrated image become greater.
[0106] In summary, the color calibration method for medical endoscopes provided in this application has at least the following beneficial effects: 1. Balancing color calibration accuracy and color contrast control: By introducing a color contrast compensation term into the objective function, the limitations of traditional algorithms that "simply pursue the minimization of color differences" are broken. While ensuring the consistency of the calibrated image with the real scene color (meeting the requirements of medical diagnosis for color accuracy), the color contrast of the image is effectively improved, clearly highlighting tissue details and lesion boundaries.
[0107] 2. Color contrast intensity can be flexibly adjusted: By adjusting the weighting factor of the color contrast compensation item, the color contrast of the calibrated image can be flexibly controlled according to the needs of different clinical examination scenarios (such as increasing the contrast to highlight details when observing small lesions, and appropriately relaxing the contrast limit to ensure natural colors when observing large areas of tissue), making it more applicable.
[0108] 3. Strong compatibility and easy implementation: This invention only optimizes the objective function of the traditional algorithm. The core process of solving the 3×3 calibration matrix is compatible with the traditional algorithm. There is no need to modify the hardware structure of medical endoscope equipment. It can be implemented only through software algorithm upgrade. The development cost is low, the promotion difficulty is small, and it can be quickly applied to existing medical endoscope equipment.
[0109] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
[0110] In the various embodiments of the specification, some or all of the steps and their optional implementations can be arbitrarily combined with some or all of the steps in other embodiments, or arbitrarily combined with the optional implementations in other embodiments.
[0111] This application provides a color calibration device for medical endoscope equipment, such as... Figure 4 As shown, the color calibration device 100 includes: The construction module 101 is used to construct a target function for the target medical endoscope device, which includes a first component and a second component that are negatively correlated. The first component is used to characterize the color difference between the calibration color card image and the reference color card image, and the second component is used to compensate for the color contrast of the calibration color card image. The calibration color card image is obtained by calibrating the original color card image acquired by the target medical endoscope device through a color calibration matrix. The optimization module 102 is used to iteratively optimize the objective function with the color calibration matrix as the optimization variable in order to solve for the optimal color calibration matrix; The calibration module 103 is used to perform color calibration on the original medical images acquired by the target medical endoscope using the optimal color calibration matrix.
[0112] In some embodiments, the optimization module 102 is specifically used for: Initialize a color calibration matrix as the current color calibration matrix; The original color chart image is color-transformed using the current color calibration matrix to generate the current calibration color chart image; Based on the current calibration color chart image and the reference color chart image, calculate the current value of the first component in the objective function, and based on the color data of multiple color patches in the current calibration color chart image, calculate the current value of the second component in the objective function; The current value of the objective function is calculated based on the current values of the first component and the second component. Update the current color calibration matrix based on the current value of the objective function; The process of generating the current calibration color chart image and updating the current color calibration matrix is repeated until a preset optimization termination condition is met. The color calibration matrix that meets the optimization termination condition is then taken as the optimal color calibration matrix.
[0113] In some embodiments, the optimization module 102 is specifically used for: Based on the color data of multiple color patches in the current calibration color chart image, calculate the average color value of each color patch in at least one color channel; Based on the color mean, the color distribution dispersion corresponding to each color channel is calculated; The current value of the second component is calculated based on the color distribution dispersion corresponding to each color channel.
[0114] In some embodiments, the color distribution dispersion is the variance or standard deviation of the color mean of each color channel, and the current value of the second component is the average variance or average standard deviation of the plurality of color channels.
[0115] In some embodiments, the second component is introduced into the objective function by the reciprocal of the second component or by a decreasing function with respect to the second component; the value of the objective function is positively correlated with the value of the first component and negatively correlated with the value of the second component.
[0116] In some embodiments, the construction module 101 is specifically used for: Based on the selected clinical observation mode of the target medical endoscope, corresponding weight coefficients are assigned to the first component and the second component in the objective function, and the first component and the second component are weighted and calculated based on the configured weight coefficients to construct the objective function. The clinical observation modes include tissue detail enhancement mode, color fidelity mode, and standard balance mode; In the organization detail enhancement mode, the weight coefficient assigned to the second component is higher than the weight coefficient assigned to the second component in the color fidelity mode, and / or, the weight coefficient assigned to the first component is lower than the weight coefficient assigned to the first component in the color fidelity mode. In the standard balancing mode, the first component and the second component are assigned equal weighting coefficients.
[0117] In some embodiments, the calibration module 103 is specifically used for: Based on the target clinical observation mode selected for the original medical image, the optimal color calibration matrix corresponding to the target clinical observation mode is invoked to perform color transformation on the original medical image.
[0118] In some embodiments, the optimization algorithm used in the iterative optimization is gradient descent, least squares, or genetic algorithm.
[0119] In some embodiments, the apparatus may further include: The preprocessing module is used to preprocess the original medical image to obtain a preprocessed image as the image to be calibrated. The preprocessing includes: enhancing the pixel values of pixels in the original medical image that meet preset conditions, and keeping the pixel values of pixels that do not meet the preset conditions unchanged. The preset conditions include: the pixel value of a pixel is higher than the mean of its local neighborhood, and the difference between the maximum and minimum pixel values in its local neighborhood is higher than a preset screening threshold.
[0120] The calibration module 103 is specifically used for: The optimal color calibration matrix is used to perform color calibration on the image to be calibrated, resulting in the calibrated image.
[0121] This application also provides a medical endoscope device, including a processor, which is configured to invoke instructions to cause the medical endoscope device to perform the steps of the color calibration method for a medical endoscope device as provided in any of the foregoing embodiments.
[0122] This application also provides a storage medium storing an executable program thereon, which, when executed by a processor, implements the steps of the color calibration method for medical endoscope devices as provided in any of the foregoing embodiments.
[0123] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconstructable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconstructable hardware circuit, the processor loads a configuration document, implementing a cyclical process of hardware circuit configuration. This can be understood as the processor loading instructions to implement the functions of some or all of the above units or modules in a cyclical process. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.
[0124] The computer-readable storage medium provided in this embodiment can execute the color calibration method for medical endoscope equipment described in the above embodiments. Its implementation principle and technical effects are similar to those in the above embodiments, and will not be repeated here.
[0125] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0126] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in a medical endoscope or main control device.
[0127] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0128] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
[0129] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A color calibration method for a medical endoscope device, characterized by, The method comprises: constructing a target function comprising a first component and a second component in a negative correlation relationship for a target medical endoscope device, the first component being used to represent color difference between a calibration color card image and a reference color card image, and the second component being used to compensate for color contrast of the calibration color card image, the calibration color card image being obtained by calibrating a raw color card image collected by the target medical endoscope device by a color calibration matrix; iteratively optimizing the target function with the color calibration matrix as an optimization variable to solve an optimal color calibration matrix; calibrating a raw medical image collected by the target medical endoscope device by using the optimal color calibration matrix.
2. The color calibration method of claim 1, wherein, iteratively optimizing the target function with the color calibration matrix as an optimization variable to solve an optimal color calibration matrix, comprising: initializing a color calibration matrix as a current color calibration matrix; generating a current calibration color card image by performing color transformation on the raw color card image by using the current color calibration matrix; calculating a current value of the first component in the target function based on the current calibration color card image and the reference color card image, and calculating a current value of the second component in the target function based on color data of a plurality of color blocks in the current calibration color card image; calculating a current value of the target function according to the current value of the first component and the current value of the second component; updating the current color calibration matrix according to the current value of the target function; repeating the process of generating the current calibration color card image to updating the current color calibration matrix until a preset optimization termination condition is met, and taking the color calibration matrix meeting the optimization termination condition as the optimal color calibration matrix.
3. The color calibration method of claim 2, wherein, calculating the current value of the second component in the target function based on the color data of the plurality of color blocks in the current calibration color card image, comprising: calculating color mean values of the plurality of color blocks in each color channel based on the color data of the plurality of color blocks in the current calibration color card image; calculating color distribution dispersion corresponding to each color channel based on the color mean values; calculating the current value of the second component based on the color distribution dispersion corresponding to each color channel.
4. The color calibration method of claim 3, wherein, The color distribution dispersion is variance or standard deviation of the color mean values of each color channel, and the current value of the second component is average value of variances or average value of standard deviations of the plurality of color channels.
5. The color calibration method of claim 1, wherein, The second component is introduced into the target function through an inverse of the second component or a decreasing function with respect to the second component; the value of the target function is positively correlated with the value of the first component and is negatively correlated with the value of the second component.
6. The color calibration method according to any one of claims 1 to 5, characterized in that, The method for constructing the target function comprising the first component and the second component in the negative correlation relationship for the target medical endoscope device, comprising: Based on the selected clinical observation mode of the target medical endoscope, corresponding weight coefficients are assigned to the first component and the second component in the objective function, and the first component and the second component are weighted and calculated based on the configured weight coefficients to construct the objective function. The clinical observation modes include tissue detail enhancement mode, color fidelity mode, and standard balance mode; In the organization detail enhancement mode, the weight coefficient assigned to the second component is higher than the weight coefficient assigned to the second component in the color fidelity mode, and / or, the weight coefficient assigned to the first component is lower than the weight coefficient assigned to the first component in the color fidelity mode. In the standard balancing mode, the first component and the second component are assigned equal weighting coefficients.
7. The color calibration method of claim 6, wherein, The step of color calibration of the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix includes: Based on the target clinical observation mode selected for the original medical image, the optimal color calibration matrix corresponding to the target clinical observation mode is invoked to perform color transformation on the original medical image.
8. The color calibration method of claim 1, wherein, The method further includes: The original medical image is preprocessed to obtain a preprocessed image as an image to be calibrated. The preprocessing includes: enhancing the pixel values of pixels in the original medical image that meet preset conditions, and keeping the pixel values of pixels that do not meet the preset conditions unchanged. The preset conditions include: the pixel value of a pixel is higher than the mean of its local neighborhood, and the difference between the maximum and minimum pixel values in its local neighborhood is higher than a preset screening threshold. The step of performing color calibration on the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix includes: The optimal color calibration matrix is used to perform color calibration on the image to be calibrated to obtain the calibrated image.
9. A color calibration device for a medical endoscope apparatus, characterized by, The device includes: A construction module is used to construct an objective function for a target medical endoscope device, which includes a first component and a second component that are negatively correlated. The first component is used to characterize the color difference between the calibration color chart image and the reference color chart image, and the second component is used to compensate for the color contrast of the calibration color chart image. The calibration color chart image is obtained by calibrating the original color chart image acquired by the target medical endoscope device through a color calibration matrix. An optimization module is used to iteratively optimize the objective function with the color calibration matrix as the optimization variable in order to solve for the optimal color calibration matrix; The calibration module is used to perform color calibration on the raw medical images acquired by the target medical endoscope using the optimal color calibration matrix.
10. A medical endoscope apparatus characterized by comprising: The device includes a processor configured to invoke instructions to cause the medical endoscope to perform the color calibration method for a medical endoscope as described in any one of claims 1 to 8.