Image color style adjustment method and device based on isp

Abstract

The application discloses an image color style adjustment method and device based on an ISP. A source image is acquired from a first imaging system, and first probability density cumulative distribution data of the source image is determined; second probability density cumulative distribution data of a target image is acquired in advance, or the target image is acquired from a second imaging system, and the second probability density cumulative distribution data of the target image is determined; a gray scale mapping table of each component channel of the source image and the target image is obtained according to the first and second probability density cumulative distribution data; a third gray scale value of each pixel after adjustment is determined according to a first gray scale value of each pixel of each component channel of the source image and a corresponding gray scale mapping table, and an output image after color style adjustment is obtained. The color difference of different devices can be eliminated, an image with consistent color style is obtained, and the method has strong universality.

Description

Technical Field

[0001] This invention relates to the field of image signal processing technology, and in particular to an image color style adjustment method and apparatus based on ISP. Background Technology

[0002] With the rapid development of minimally invasive surgery and endoscopic diagnostic technologies, the image quality of medical electronic endoscopes directly affects the accuracy of clinical diagnosis, lesion detection rate, and surgical safety. The image signal processor (ISP), as the core module of the endoscopic imaging system, is responsible for real-time processing of the raw image signals output from the sensor, including key processes such as white balance, color correction, gamma correction, and detail enhancement. Significant differences exist in the ISP processing algorithms and color adjustment strategies of different manufacturers, resulting in varying image presentation styles (hue, saturation, contrast, and overall visual atmosphere), making it difficult to achieve a unified industry standard.

[0003] In clinical practice, doctors develop specific color vision habits after using an endoscope system from a particular manufacturer for a long time. When switching to products from other manufacturers, the significant differences in image color style often require doctors to readjust for a considerable period of time. This not only increases the cognitive burden but may also affect the efficiency of identifying lesions and the smoothness of surgical procedures, and even potentially affect diagnostic consistency.

[0004] To mitigate the negative impact of inconsistent color styles in images captured by different devices, industry practice commonly employs color mapping algorithms. Currently, the mainstream color mapping algorithms include:

[0005] 1) Color Correction Matrix (CCM): A 3×3 or 3×4 color correction matrix is ​​used to linearly transform the RGB channels of an image. The color correction matrix is ​​a linear mapping and is most commonly used in the front end of an image signal processor (ISP) for color correction to compensate for the difference between the sensor's spectral response and the human eye. This method can only handle linear relationships and has poor adaptability to complex nonlinear color styles.

[0006] 2) Lookup Table Mapping: The most representative example is 3DLUT (3D Lookup Table, or 3D Cube Mapping Table). 3D LUTs create a lookup table in RGB 3D cube space, using linear interpolation. Only the corrected RGB values ​​are output through the lookup table. It is commonly used for ISP color mapping and preset style adjustments. Storage usage increases with precision, interpolation may introduce slight artifacts, and its ability to handle extreme colors is limited.

[0007] 3) Curve / Tone Curve Mapping: Common curves include S-curves, Gamma curves, and piecewise linear curves. Usually, Hue, Saturation, and Value are adjusted in HSV / HSL space to adjust the color of the image. This method involves a lot of manual parameter adjustment and is complicated to operate.

[0008] In summary, traditional color mapping algorithms mostly use global statistical features or static mapping functions. Their core bottleneck lies in the difficulty of simultaneously achieving real-time performance, realism, and stability. Furthermore, since the color styles of devices from different manufacturers are not entirely the same, static parameters cannot eliminate the differences in color styles across hardware manufacturers and cannot flexibly adapt to the color style adjustment needs of different manufacturers.

[0009] Furthermore, patent application CN103379346A, entitled "A Method, Apparatus and System for Processing Chromaticity Information of YUV Format Images," discloses a chromaticity enhancement scheme based on empirical intervals and local weights. Patent application CN111147857A, entitled "An Image Processing Method, Image Processing Apparatus, Electronic Device and Storage Medium," discloses a chromaticity processing scheme based on adjusting the luminance Y using a shared chromaticity component UV as a basis. These schemes rely on preset empirical intervals and local weights or UV-based downsampling strategies, focusing on specific hue enhancement. However, they have weak global statistical adjustment capabilities, low automation, and poor adaptability to accurately match the overall distribution of any reference image. Summary of the Invention

[0010] In view of the above problems, the present invention is proposed to provide an image color style adjustment method and apparatus based on ISP that overcomes or at least partially solves the above problems.

[0011] This invention provides an image color style adjustment method based on ISP, comprising:

[0012] The source image is acquired from the first imaging system and converted to a specified color space. Based on the image data in the specified color space, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain the first histogram distribution data. Based on the first histogram distribution data, the probability density of each first gray level is determined, and the first probability density cumulative distribution data is obtained based on the probability density of each first gray level.

[0013] Acquire the second probability density cumulative distribution data of the pre-configured target image; or, acquire the target image from the second imaging system and convert the target image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain the second histogram distribution data; determine the probability density of each second gray level based on the second histogram distribution data, and obtain the second probability density cumulative distribution data based on the probability density of each second gray level.

[0014] Based on the first probability density cumulative distribution data and the second probability density cumulative distribution data, a gray level mapping table is obtained for each component channel of the source image to the target image; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image.

[0015] Based on the first grayscale value of each pixel in each component channel of the source image and the corresponding grayscale level mapping table, the adjusted third grayscale value of each pixel is determined to obtain the output image after color style adjustment.

[0016] In some optional embodiments, the first imaging system and the second imaging system are two different endoscope systems, and the conversion of the source image to the specified color space includes: matching a pre-set conversion formula according to the original color space and the converted color space of the source image, and converting the source image to the specified color space according to the conversion formula;

[0017] The step of converting the target image to a specified color space includes: matching a pre-defined conversion formula with the original color space and the converted color space of the target image, and converting the target image to the specified color space according to the conversion formula.

[0018] In some optional embodiments, the original color space of the source image and the target image is the RGB color space. After converting the source image and the target image to a specified color space, if the specified color space after conversion of the source image and the target image is the YUV color space, then the Y, U, and V channels of the source image and the target image respectively satisfy the following relationships:

[0019]

[0020]

[0021]

[0022] In the formula, R, G, and B are the red, green, and blue component channels of the original color space; a1, b1, and c1 are preset conversion coefficients, which respectively satisfy the specified range, and the sum of the three is 1.

[0023] a2, b2, and c2 are preset conversion coefficients, each satisfying a specified range, and the sum of the three is 0;

[0024] a3, b3, and c3 are preset conversion coefficients that satisfy the specified range, and the sum of the three is 0.

[0025] In some optional embodiments, the first gray level is set according to the number of bits of the source image, the second gray level is set according to the number of bits of the target image, each first gray level includes one or several adjacent gray values, and each second gray level includes one or several adjacent gray values.

[0026] The first histogram distribution data and the second histogram distribution data include at least one of the following: uniformly distributed image histogram, Gaussian distributed image histogram, log-normal distributed image histogram, exponentially distributed image histogram, and image histogram of a custom shape.

[0027] In some optional embodiments, determining the probability density of each first gray level based on the first histogram distribution data, and obtaining the first probability density cumulative distribution data based on the probability density of each first gray level, includes:

[0028] For each first gray level k1 of each component channel: divide the number of pixels appearing at the first gray level by the total number of pixels in the source image to obtain the probability density of the first gray level; sum all probability densities from 0 to the first gray level k1 to obtain the cumulative probability density value of the first gray level; the cumulative probability density values ​​of all first gray levels constitute the first probability density cumulative distribution data.

[0029] Based on the second histogram distribution data, the probability density of each second gray level is determined, and the cumulative distribution data of the second probability density is obtained based on the probability density of each second gray level, including:

[0030] For each second gray level k2 of each component channel: divide the number of pixels appearing at the second gray level by the total number of pixels in the target image to obtain the probability density of the second gray level; sum all the probability densities from 0 to the second gray level k2 to obtain the cumulative probability density value of the second gray level; the cumulative probability density values ​​of all second gray levels constitute the second probability density cumulative distribution data.

[0031] In some optional embodiments, obtaining the grayscale level mapping table of each component channel from the source image to the target image based on the first probability density cumulative distribution data and the second probability density cumulative distribution data includes:

[0032] Cumulative probability density for each first gray level of each component channel of the source image:

[0033] Find the position in the second probability density cumulative distribution data that is closest to the cumulative probability density value, and establish a mapping relationship between the second gray level at that position and the corresponding first gray level.

[0034] After establishing a mapping relationship for each first gray level, a gray level mapping table for each component channel of the source image and the target image is obtained.

[0035] In some optional embodiments, determining the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table to obtain the color style adjusted output image includes:

[0036] For each pixel in each component channel of the source image: determine the corresponding second gray level according to the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel, and use a gray value in the determined second gray level as the adjusted third gray value of the pixel;

[0037] The color-style adjusted output image is obtained based on the third grayscale value after adjusting all pixels in each component channel.

[0038] In some optional embodiments, after converting the source image to a specified color space, the method further includes:

[0039] The source image is divided into multiple first image blocks; correspondingly...

[0040] The method of statistically analyzing the number of pixels appearing at each preset first gray level in each component channel of the source image to obtain the first histogram distribution data includes: for each first image block in the source image, statistically analyzing the number of pixels appearing at each preset first gray level in each component channel of the first image block to obtain the first histogram distribution data of each first image block.

[0041] The step of obtaining the first probability density cumulative distribution data includes: obtaining the first probability density cumulative distribution data of each first image block in the source image;

[0042] After converting the target image to the specified color space, the process also includes:

[0043] The target image is divided into multiple second image blocks, and each second image block in the target image corresponds one-to-one with a first image block in the source image; accordingly.

[0044] The method of statistically analyzing the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain second histogram distribution data includes: for each second image block in the target image, statistically analyzing the number of pixels appearing at each preset second gray level in each component channel of each second image block to obtain second histogram distribution data for each second image block.

[0045] The step of obtaining the second probability density cumulative distribution data includes: obtaining the second probability density cumulative distribution data for each second image block in the target image;

[0046] Accordingly, obtaining the grayscale mapping table of each component channel from the source image to the target image includes: obtaining the grayscale mapping table of each component channel of each first image block in the source image and the corresponding second image block in the target image.

[0047] In some optional embodiments, determining the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table to obtain the color style adjusted output image includes:

[0048] For each pixel of each component channel in the source image:

[0049] Based on the first image block to which the pixel belongs, determine the multiple adjacent first image blocks to which it belongs;

[0050] Based on the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel of each adjacent first image block, determine the corresponding multiple candidate third gray values ​​of the pixel.

[0051] Based on the distance between the pixel and the center point of the adjacent first image block, determine the interpolation weight for the corresponding candidate third gray value of each adjacent first image block;

[0052] Based on the interpolation weights, multiple candidate third gray values ​​are weighted and calculated to obtain the adjusted third gray value of the pixel.

[0053] The color-style adjusted output image is obtained based on the third grayscale value after adjusting all pixels in each component channel.

[0054] In some optional embodiments, the number of pixels appearing at each preset third gray level in each component channel of the output image is counted to obtain third histogram distribution data; the similarity between the third histogram distribution data and the second histogram distribution data is greater than a set similarity threshold.

[0055] In some optional embodiments, the similarity ρ between the third histogram distribution data and the second histogram distribution data is determined using the following formula:

[0056] ;

[0057] In the formula, This represents the number of pixels with grayscale value i in the output image. Let i be the number of pixels with grayscale value i in the target image. This is the average grayscale value of all pixels in the output image. L is the average grayscale value of all pixels in the target image, and L is the number of discrete levels for each channel.

[0058] In some optional embodiments, the above method further includes:

[0059] At least one of the first histogram distribution data, the second histogram distribution data, the third histogram distribution data, the first probability density cumulative distribution data, and the second probability density cumulative distribution data is smoothed, and the smoothing process includes at least one of Gaussian filtering, median filtering, and mean filtering.

[0060] In some alternative embodiments, the method is implemented on an FPGA, CPU, or GPU; wherein:

[0061] The steps of obtaining the first and second histogram distribution data, obtaining the first and second probability density cumulative distribution data, obtaining the gray level mapping table of each component channel of the source image and the target image, and determining the adjusted third gray value of each pixel are run synchronously and in parallel.

[0062] By combining random access memory (BRAM), it supports single-cycle read and write of multiple probability density cumulative distribution data and multiple gray level mapping tables, enabling parallel processing of multiple component channels.

[0063] FPGAs, CPUs, or GPUs can be embedded in the ISP pipeline to complete processing within the same clock domain as the data stream in the specified color space, achieving on-chip integration with zero copy and zero bus transmission latency.

[0064] This invention provides an image color style adjustment device based on ISP, comprising:

[0065] The image acquisition module is used to acquire a target image from the second imaging system and a source image from the first imaging system;

[0066] The data acquisition module is used to convert the source image to a specified color space; based on the image data in the specified color space, it counts the number of pixels appearing at each preset first gray level in each component channel of the source image to obtain the first histogram distribution data; based on the first histogram distribution data, it determines the probability density of each first gray level, and obtains the first probability density cumulative distribution data based on the probability density of each first gray level.

[0067] The data acquisition module is further configured to acquire pre-configured second probability density cumulative distribution data; or, convert the source image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the source image to obtain second histogram distribution data; determine the probability density of each second gray level based on the second histogram distribution data, and obtain the second probability density cumulative distribution data based on the probability density of each second gray level.

[0068] The mapping relationship construction module is used to obtain a gray level mapping table from the source image to each component channel of the target image based on the first probability density cumulative distribution data and the second probability density cumulative distribution data; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image;

[0069] The style adjustment module is used to determine the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, so as to obtain the output image after color style adjustment.

[0070] This invention provides a computer storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned ISP-based image color style adjustment method.

[0071] This invention provides a color style adjustment device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described ISP-based image color style adjustment method.

[0072] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:

[0073] In a specified color space, such as the YUV color space, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain first histogram distribution data. Based on the first histogram distribution data, the first probability density cumulative distribution data of the source image is determined. The second probability density cumulative distribution data of the configured target image is obtained, or the number of pixels appearing at each preset second gray level in each component channel of the target image is counted to obtain second histogram distribution data. Based on the second histogram distribution data, the second probability density cumulative distribution data of the target image is determined. Based on the first and second probability density cumulative distribution data, a gray level mapping table for each component channel of the source and target images is obtained. Based on the source gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, the gray values ​​of each component channel of the source image are adjusted to optimize the components, resulting in a color-style adjusted output image while ensuring color fidelity, thus achieving high color fidelity in the processed image. This method can eliminate color differences between different devices, obtain images with consistent color styles, provide clinicians with a consistent and stable visual experience, thereby improving diagnostic efficiency and surgical safety. This method is applicable to the processing of source images captured by various devices, and has strong versatility; it can match source and target images in real time, the algorithm is simple and easy to implement, has low hardware requirements, and is easy to configure on devices.

[0074] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0075] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0076] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0077] Figure 1 This is a flowchart of an image color style adjustment method based on ISP in an embodiment of the present invention.

[0078] Figure 2 This is a schematic diagram of the structure of the image color style adjustment device based on ISP in an embodiment of the present invention.

[0079] Figure 3 This is an example image of the source image in an embodiment of the present invention.

[0080] Figure 4 As described in the embodiments of the present invention Figure 3 The histogram of the source image.

[0081] Figure 5 This is an example image of the target image in an embodiment of the present invention.

[0082] Figure 6 As described in the embodiments of the present invention Figure 5 Histogram of the target image.

[0083] Figure 7 This is an example image of the image after histogram matching in an embodiment of the present invention.

[0084] Figure 8 As described in the embodiments of the present invention Figure 7 The histogram of the matched image.

[0085] Figure 9 This is an example diagram of image block matching in an embodiment of the present invention. Detailed Implementation

[0086] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0087] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the serial numbers used for components or terms herein, such as "first," "second," etc., are only used to distinguish the described objects and have no sequential or technical meaning, nor should they be construed as indicating or implying relative importance.

[0088] To address the aforementioned issues, this invention proposes a rapid color style adjustment method based on an image signal processor (ISP). This method aims to achieve rapid, unified, and controllable transfer of color styles from endoscope images from different manufacturers, while preserving structural details, texture information, and medical realism, thus providing doctors with a consistent visual experience. This method is applicable to medical endoscopic image signal processing, including, but not limited to, color style control for medical endoscopic systems such as gastroscopes, colonoscopes, ureteroscopes, and bronchoscopes.

[0089] To address the color style differences between images captured by devices from different manufacturers, the following technical issues need to be resolved:

[0090] How to achieve accurate and intuitive histogram matching in a specified color space while avoiding color distortion.

[0091] How to reduce computational complexity to suit hardware implementation and real-time algorithm implementation.

[0092] How to solve the problems of local artifacts and noise sensitivity in global image matching.

[0093] How to achieve color normalization while maintaining the naturalness of brightness.

[0094] Ultimately, a standardized process for color style transfer was developed to solve the problem of inconsistent color styles in images captured by devices from different manufacturers.

[0095] This invention provides an image color style adjustment method based on an image signal processor (ISP). For two imaging systems, such as a first imaging system and a second imaging system, since the color styles of the images they acquire differ, the color style of the source image acquired by the first imaging system can be adjusted to be similar to the color style of the target image acquired by the second imaging system. Specifically, using the target image acquired by the second imaging system as a reference, the style of the source image acquired by the first imaging system is adjusted to match the color style of the target image. This process includes, but is not limited to, first acquiring the source and target images using an image acquisition card, then copying the acquired source and target images to a computer or image processing host for processing, and finally storing the resulting grayscale mapping table from the source image to the target image's component channels in the image processing host of the first imaging system. The image processing host can be configured with an ISP. The first and second imaging systems can be, for example, two different endoscope systems. The grayscale mapping table from the source image to the target image's component channels includes a grayscale mapping table for three component channels. The flowchart of this ISP-based image color style adjustment method is as follows: Figure 1 As shown, it includes the following steps:

[0096] Step S11: Acquire a source image from the first imaging system and convert the source image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing in each preset first gray level in each component channel of the source image to obtain the first histogram distribution data; determine the probability density of the image at each first gray level based on the first histogram distribution data, and obtain the first probability density cumulative distribution data based on the probability density of each first gray level.

[0097] Step S12: Obtain the second probability density cumulative distribution data of the pre-configured target image; or obtain the target image from the second imaging system and convert the target image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain the second histogram distribution data; determine the probability density of each second gray level according to the second histogram distribution data, and obtain the second probability density cumulative distribution data according to the probability density of each second gray level.

[0098] Step S13: Based on the first probability density cumulative distribution data and the second probability density cumulative distribution data, obtain the gray level mapping table of each component channel from the source image to the target image; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image.

[0099] Step S14: Based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, determine the adjusted third gray value of each pixel to obtain the output image after color style adjustment.

[0100] The method and apparatus described in this invention, in a specified color space, such as the YUV color space, count the number of pixels appearing at each preset gray level in each component channel of the target image and the source image; based on the number of pixels appearing at each gray level in each component channel, determine the cumulative probability density distribution data of each gray level in the target image and the source image; based on the cumulative probability density distribution data of the target image and the source image, obtain a gray level mapping table from the source image to each component channel of the target image; based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, adjust the gray values ​​of each component channel of the source image to optimize the components, obtain an output image with color style adjustment, and ensure color fidelity, resulting in high color fidelity of the processed image. This method can eliminate color differences between different devices, obtain images with consistent color styles, provide clinicians with a consistent and stable visual experience, thereby improving diagnostic efficiency and surgical safety. This method is applicable to the processing of source images captured by various devices, has strong versatility, can match source and target images in real time, has a simple algorithm, is easy to implement, has low hardware requirements, and is easy to configure on devices.

[0101] Step S11 described above acquires a source image from the first imaging system and determines the first probability density cumulative distribution data of the source image. One or more source images (Spic) can be acquired from the first imaging system. After acquiring the source image, it is processed to obtain the first probability density cumulative distribution data. An example of the acquired source image can be found here. Figure 3 As shown.

[0102] The process of determining the first probability density cumulative distribution data of the source image includes:

[0103] Step S111: Convert the source image to the specified color space.

[0104] Step S112: Based on the image data in the specified color space, count the number of pixels appearing at each preset first gray level in each component channel of the source image to obtain the first histogram distribution data.

[0105] Step S113: Based on the first histogram distribution data, determine the probability density of each first gray level, and obtain the first probability density cumulative distribution data based on the probability density of each first gray level.

[0106] In step S111 above, converting the source image to a specified color space includes: matching a pre-defined conversion formula based on the original color space and the converted color space of the source image; and converting the source image to the specified color space according to the conversion formula to obtain the specified color space data of the source image. The specified color space includes color spaces such as YUV, RGB, Lab, or HSV. It can be selected as needed. In the description of this embodiment, taking the original color space of the source image as RGB and the converted specified color space as YUV as an example, after converting the RGB image to the specified YUV color space, the YUV channels are separated to obtain the YUV data of the source image. After converting the source image to the specified color space, that is, after converting the RGB image to the specified YUV color space, the Y, U, and V channels of the source image respectively satisfy the following formulas:

[0107]

[0108]

[0109]

[0110] In the formula, R, G, and B are the red, green, and blue component channels of the original color space;

[0111] a1, b1, and c1 are preset conversion coefficients that satisfy the specified range intervals, and the sum of the three is 1; optionally, a1, b1, and c1 satisfy the range intervals of [0.25, 0.35], [0.55, 0.65], and [0.05, 0.15], respectively.

[0112] a2, b2, and c2 are preset conversion coefficients that satisfy the specified range intervals, and the sum of the three is 0; optionally, a2, b2, and c2 satisfy the range intervals of [-0.18, -0.12], [-0.32, -0.25], and [0.40, 0.48], respectively.

[0113] a3, b3, and c3 are preset conversion coefficients that satisfy the specified range, and the sum of the three is 0.

[0114] Optionally, a3, b3, and c3 satisfy the ranges [0.58, 0.65], [-0.55, -0.48], and [-0.12, -0.06], respectively.

[0115] In this step, the conversion of each channel component uses a weighted linear combination method, and the coefficients of each channel are selected within a predetermined range. This step obtains the YUV data of the source image: Ys, Us, and Vs.

[0116] In step S112 above, based on the image data in the specified color space, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain the first histogram distribution data.

[0117] The first gray level is set based on the number of bits in the source image. Each first gray level includes one or several adjacent gray values. It supports 8-bit (L=256) and higher bit depths (such as 10-bit, 12-bit, or 16-bit) images. Taking an 8-bit image as an example, with L=256 discrete levels per channel, the total number of first gray levels can be 256 (i.e., first gray level k1=0,1,2,3…L-1); it can also be less than 256, such as 128 (two gray values ​​per first gray level); or 64 (four gray values ​​per first gray level); and the number of gray values ​​included in each first gray level can also vary. The specific settings can be customized as needed. For a 10-bit image, L=1024, and the first gray level can be 1024 or other values. Using histograms provides fast calculation speed, making it suitable for real-time or embedded applications.

[0118] Perform histogram statistics for each component channel of the source image in a specified color space, such as YUV histogram statistics. That is, for each first gray level k1 (k1 is a positive integer), count the number of pixels appearing at each first gray level k1 in the Y, U, and V component channels of the image. In other words, the first histogram distribution data is obtained by counting the total number of pixels in the image whose grayscale values ​​are included in the first grayscale level. See [example of first histogram distribution data for source image]. Figure 4 As shown, the horizontal axis represents the first gray level, and the vertical axis represents the number of pixels corresponding to each first gray level. The number of pixels appearing at each first gray level k1 can be counted based on the first gray value of each pixel in the source image.

[0119] The first histogram distribution data includes at least one of the following: uniformly distributed image histograms, Gaussian distributed image histograms, log-normal distributed image histograms, exponentially distributed image histograms, and image histograms of custom shapes.

[0120] In step S113 above, when determining the probability density of the source image at each first gray level based on the obtained first histogram distribution data, that is, the number of pixels appearing at each first gray level in each component channel, for each first gray level k1 of each component channel:

[0121] 1) Divide the number of pixels appearing at the first gray level by the total number of pixels in the source image to obtain the probability density of the first gray level. Calculate the probability density of the first gray level k1. for: N1 is the total number of pixels in the source image.

[0122] 2) Accumulate all probability densities from 0 to gray level k1 to obtain the cumulative probability density value of the first gray level; the cumulative probability density values ​​of all first gray levels constitute the first probability density cumulative distribution data of the source image. The calculated cumulative probability density value of the first gray level k1 is:

[0123] ;

[0124] In the formula, i and k1 represent gray levels, and the value of i ranges from 0 to k1.

[0125] 3) Obtain the cumulative distribution data of the first probability density based on the probability density of each first gray level.

[0126] This step involves obtaining the first probability density cumulative distribution data (CDFs) of the source image. This data can be a cumulative distribution function or a curve.

[0127] Step S12 above acquires the second probability density cumulative distribution data of a pre-configured target image; or acquires a target image from a second imaging system and determines the second probability density cumulative distribution data of the target image. One or more target images (Tpic) can be acquired from the second imaging system. In this step, the second probability density cumulative distribution data can be acquired in real time as the target reference data for mapping, or pre-configured target reference data can be used without real-time acquisition. This method allows for flexible changes to the matching target, thus making it more adaptable to various devices and processing images captured by different devices. It is highly flexible and versatile. An example of the acquired target image can be found in [link to example]. Figure 5 As shown.

[0128] The process of determining the second probability density cumulative distribution data of the target image includes:

[0129] Step S121: Convert the target image to the specified color space.

[0130] Step S122: Based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain the second histogram distribution data.

[0131] Step S123: Based on the second histogram distribution data, determine the probability density of each second gray level, and obtain the cumulative distribution data of the second probability density based on the probability density of each second gray level.

[0132] Step S121: Convert the target image to a specified color space, including: matching a pre-set conversion formula based on the original color space and the converted color space of the target image; converting the target image to the specified color space according to the conversion formula to obtain the specified color space data of the target image. The specified color space includes color space domains such as YUV, RGB, Lab, or HSV. It can be selected as needed. In the description of this application embodiment, taking the original color space of the target image as RGB color space and the converted specified color space as YUV color space as an example, after converting the RGB image to the specified YUV color space, the YUV channels are separated to obtain the YUV data of the target image. After converting the target image to the specified color space, that is, after converting the RGB image to the specified YUV color space, the Y, U, and V channels of the target image respectively satisfy the following formulas:

[0133]

[0134]

[0135]

[0136] In the formula, R, G, and B are the red, green, and blue component channels of the original color space;

[0137] a1, b1, and c1 are preset conversion coefficients that satisfy the specified range intervals, and the sum of the three is 1; optionally, a1, b1, and c1 satisfy the range intervals of [0.25, 0.35], [0.55, 0.65], and [0.05, 0.15], respectively.

[0138] a2, b2, and c2 are preset conversion coefficients that satisfy the specified range intervals, and the sum of the three is 0; optionally, a2, b2, and c2 satisfy the range intervals of [-0.18, -0.12], [-0.32, -0.25], and [0.40, 0.48], respectively.

[0139] a3, b3, and c3 are preset conversion coefficients that satisfy the specified range, and the sum of the three is 0.

[0140] Optionally, a3, b3, and c3 satisfy the ranges [0.58, 0.65], [-0.55, -0.48], and [-0.12, -0.06], respectively.

[0141] This step involves obtaining the YUV data of the target image: Yt, Ut, and Vt.

[0142] Step S122 above: Based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain the second histogram distribution data.

[0143] The second grayscale level is set according to the number of bits in the target image. Each second grayscale level includes one or several adjacent grayscale values. It supports 8-bit (L=256) and higher bit depths (such as 10-bit, 12-bit, or 16-bit) images. Taking an 8-bit image as an example, with L=256 discrete levels per channel, the total number of second grayscale levels can be 256, i.e., second grayscale level k2=0,1,2,3…L-1; or less than 256, such as 128, i.e., 2 grayscale values ​​per second grayscale level; or 64, i.e., 4 grayscale values ​​per second grayscale level; of course, the number of grayscale values ​​included in each second grayscale level can also be different; the specific setting can be customized as needed. Taking a 10-bit image as an example, L=1024, and the second grayscale level can be 1024 or other values. Histograms are used, resulting in fast calculation speed, making it suitable for real-time or embedded applications. Optionally, the number of second grayscale levels in the target image is the same as the number of first grayscale levels in the source image.

[0144] Perform histogram statistics for each component channel of the target image in a specified color space, such as YUV histogram statistics. That is, for each second gray level k2 (k2 is a positive integer), count the number of pixels appearing at each gray level k2 in the Y, U, and V component channels of the image. In other words, the second histogram distribution data is obtained by counting the total number of pixels in the image whose grayscale values ​​are included in the second grayscale level. An example of the second histogram distribution data for a target image can be found here. Figure 6 As shown, the horizontal axis represents the second gray level, and the vertical axis represents the number of pixels corresponding to each second gray level. The number of pixels appearing at each second gray level k2 can be counted based on the second gray value of each pixel in the target image.

[0145] The second histogram distribution data includes at least one of the following: uniformly distributed image histograms, Gaussian distributed image histograms, log-normal distributed image histograms, exponentially distributed image histograms, and image histograms of custom shapes.

[0146] In step S123 above, when determining the probability density of the target image at each second gray level based on the obtained second histogram distribution data, that is, the number of pixels appearing at each second gray level in each component channel, for each second gray level k2 of each component channel:

[0147] 1) Divide the number of pixels appearing at the second gray level by the total number of pixels in the target image to obtain the probability density of the second gray level. Calculate the probability density of the second gray level 2k. for: N2 is the total number of pixels in the target image.

[0148] 2) Accumulate all probability densities from 0 to the second gray level k2 to obtain the cumulative probability density value for that second gray level; the cumulative probability densities of all second gray levels constitute the second probability density cumulative distribution data. The calculated cumulative probability density value for the second gray level k2 is:

[0149] ;

[0150] In the formula, i and k2 represent gray levels, and the value of i ranges from 0 to k2.

[0151] 3) Obtain the cumulative distribution data of the second probability density of the target image based on the probability density of each second gray level.

[0152] This step involves obtaining the second probability density cumulative distribution data CDFt of the target image. This data can be a cumulative distribution function or a curve.

[0153] Steps S11 and S12 convert the target image and source image to a specified color space. Based on the target image data and source image data in the specified color space, the number of pixels appearing at each preset gray level in each component channel of the target image and source image is counted. Based on the number of pixels appearing at each gray level in each component channel, the probability density of the target image and source image at each gray level is determined. Based on the probability density of each gray level, the first probability density cumulative distribution data of the source image and the second probability density cumulative distribution data of the target image are obtained. In other words, the first probability density cumulative distribution data of the source image and the second probability density cumulative distribution data of the target image are counted separately and then processed according to a specified color channel or quantization method.

[0154] Steps S11 and S12 can also be used to obtain the first probability density cumulative distribution data of the statistical source image, and use the second data probability density cumulative distribution data of the pre-configured target image as the target reference.

[0155] In other words, in this application, the subsequent step S13 can either match the first probability density cumulative distribution data of the source image with the second probability density cumulative distribution data of the target image acquired in real time to construct a mapping table, or match the source image with the second probability density cumulative distribution data of the pre-configured target image to construct a mapping table. Any matching target can be specified, which is highly flexible.

[0156] In step S13 above, a gray level mapping table for each component channel of the source image to the target image is obtained based on the first probability density cumulative distribution data and the second probability density cumulative distribution data, including:

[0157] Cumulative probability density for each first gray level of each component channel of the source image:

[0158] Find the position in the second probability density cumulative distribution data that is closest to the cumulative probability density value, and establish a mapping relationship between the second gray level at that position and the corresponding first gray level.

[0159] After establishing a mapping relationship for each first gray level, a gray level mapping table for each component channel of the source image and the target image is obtained.

[0160] This step generates a grayscale mapping table, which is essentially a color lookup table (LUT) that reflects the grayscale mapping relationship between the source image and the target image used as a reference. For each first grayscale level k1 of the source image, the position z in the second probability density cumulative distribution data CDFt that is closest to the first probability density cumulative distribution data CDFs is found. Let Map(k) = z, and iterate through all grayscale levels to obtain a Map table, which is the color mapping LUT.

[0161] In step S14 above, based on the first grayscale value of each pixel in each component channel of the source image and the corresponding grayscale level mapping table, the adjusted third grayscale value of each pixel is determined to obtain the color style adjusted output image, including:

[0162] For each pixel in each component channel of the source image: determine the corresponding second gray level according to the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel, and use a gray value in the determined second gray level as the adjusted third gray value of the pixel;

[0163] Based on the adjusted third grayscale values ​​of all pixels in each component channel, the color-style adjusted output image is obtained. Furthermore, after obtaining the output image, the third histogram distribution data of the output image can be statistically analyzed. The number of pixels at each third grayscale level can be counted according to a preset third grayscale level to obtain the third histogram distribution data of the output image. A single grayscale value can be considered a grayscale level, or a grayscale level can include more than one grayscale value. When a grayscale level includes more than one grayscale value, any one of the grayscale values ​​from the determined second grayscale level is used as the adjusted grayscale value for that pixel.

[0164] This step performs color mapping on the source image, replacing the grayscale value k of each pixel in the source image with Map(k), resulting in an output image that matches the histogram shape of the target image. An example of the output image after histogram matching of the source image can be found here. Figure 7 As shown, the third histogram distribution data of the output image can be found in [reference needed]. Figure 8 As shown, the horizontal axis represents the third gray level, and the vertical axis represents the number of pixels corresponding to each third gray level. It can be seen that the histogram of the mapped output image has the same shape as the histogram of the target image. This method can perform a uniform transformation on the entire image, resulting in natural color / brightness style transitions and high global consistency.

[0165] This step implements pixel mapping. Based on the determined grayscale mapping relationship, each pixel of the source image is transformed one by one to generate an output image after histogram matching. The mapping relationship or the generated output image satisfies at least one of the following constraints: maintaining the relative order of grayscale values, incorporating local statistical information (block processing), or making the histogram of the output image highly similar to the histogram of the reference image.

[0166] Optionally, after adjusting the source image using the method of this application, the method further includes counting the number of pixels appearing at each preset third gray level in each component channel of the output image to obtain third histogram distribution data; the similarity between the third histogram distribution data and the second histogram distribution data is greater than a set similarity threshold. The similarity can be calculated using a pre-selected similarity calculation algorithm and a corresponding formula, for example, but not limited to, using the following formula to determine the similarity ρ between the third histogram distribution data and the second histogram distribution data used as a reference:

[0167] ;

[0168] In the formula, This refers to the number of pixels with grayscale value i in the adjusted source image (output image). Let i be the number of pixels with grayscale value i in the target image. The average grayscale value of all pixels in the adjusted source image (output image). L is the average grayscale value of all pixels in the target image, and L is the number of discrete levels for each channel.

[0169] The above formula quantifies the similarity between the third histogram distribution data of the output image and the second histogram distribution data of the target image (e.g., the histograms themselves). The similarity ρ is less than or equal to 1, and the closer ρ is to 1, the more similar the third histogram distribution data of the adjusted output image and the reference second histogram distribution data are (e.g., the more similar the shapes of the histograms). A preset similarity threshold can be used. When the similarity ρ is greater than the preset threshold, the third histogram distribution data of the adjusted output image and the reference target image are considered to satisfy a high similarity constraint and are thus considered highly similar.

[0170] In a specified color space, such as the YUV color space, histogram statistics are performed independently on the luminance component Y and the chrominance components U and V, and then matched with the corresponding reference histogram (the second probability density cumulative distribution data of the target image) to generate their respective mapping transformation functions, which are then applied to the corresponding channels in sequence to achieve rapid standardization adjustment of color style and form a complete process standardization method.

[0171] In some optional embodiments, the method further includes smoothing at least one of the first histogram distribution data, the second histogram distribution data, the third histogram distribution data, the first probability density cumulative distribution data, and the second probability density cumulative distribution data. The smoothing process includes at least one of Gaussian filtering, median filtering, and mean filtering. To prevent artifacts in the output image obtained after histogram matching, smoothing mechanisms such as Gaussian filtering, median filtering, and mean filtering can be used to smooth the histogram or CDF curve, reducing peaks at certain gray levels in the histogram, making the distribution more continuous, avoiding abrupt changes, and thus preventing artifacts.

[0172] In some optional embodiments, the above method further includes constructing a grayscale level mapping table for the source and target images by dividing them into blocks. During mapping matching, the mapping is performed block by block, and smoothing is applied based on adjacent blocks. In this optional embodiment, the image is divided into multiple small blocks, and histogram matching is performed independently within each block. Taking the YUV color space as an example, the luminance (Y), chromaticity (U), and chromaticity (V) components are divided into blocks, and histograms are calculated independently within each sub-block, with histogram matching limited to a specific contrast—that is, local histogram matching is performed. When constructing the mapping table by block, the grayscale level mapping table is constructed in conjunction with... Figure 1 The difference in the processes shown is:

[0173] After converting the source image to the specified color space, the process also includes dividing the source image into multiple first image blocks.

[0174] Correspondingly, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain the first histogram distribution data, including: for each first image block in the source image, the number of pixels appearing at each preset first gray level in each component channel of each first image block is counted to obtain the first histogram distribution data of each first image block.

[0175] Accordingly, the first probability density cumulative distribution data is obtained, including obtaining the first probability density cumulative distribution data for each first image block in the source image.

[0176] After converting the target image to the specified color space, the process also includes: dividing the target image into multiple second image blocks; the target image and the source image can be divided into blocks in the same way, and the second image blocks in the target image correspond one-to-one with the first image blocks in the source image.

[0177] Correspondingly, the number of pixels appearing at each preset second gray level in each component channel of the target image is counted to obtain the second histogram distribution data, including: for each second image block in the target image, the number of pixels appearing at each preset second gray level in each component channel of each second image block is counted to obtain the second histogram distribution data of each second image block.

[0178] Accordingly, the second probability density cumulative distribution data is obtained, including: obtaining the second probability density cumulative distribution data for each second image patch in the target image;

[0179] Accordingly, a grayscale mapping table for each component channel from the source image to the target image is obtained, including: obtaining a grayscale mapping table for each component channel of each first image block in the source image and the corresponding second image block in the target image.

[0180] In other words, for both the source and target images, a grayscale level mapping table is established for each component channel of each image block. Correspondingly, after establishing the mapping table, bilinear interpolation is used to process the mapping results during mapping matching. Based on the source grayscale value of each pixel in each component channel of the source image and the corresponding grayscale level mapping table, the adjusted grayscale value of each pixel is determined, resulting in a color-style adjusted output image, including:

[0181] 1) For each pixel p(x, y) of each component channel in the source image.

[0182] 2) Determine the multiple adjacent first image blocks to which the pixel belongs based on the first image block to which it belongs.

[0183] See Figure 9 As shown, the first image block to which P(x, y) belongs is A1. Located to the lower right of A1, within the dashed frame B1, the adjacent first image blocks of p(x, y) are the four image blocks A1, A2, A3, and A4 involved in the dashed frame B1. If P(x, y) is to the upper left of A1, within the dashed frame B2, the adjacent first image blocks of p(x, y) are the four image blocks A1, A5, A6, and A7 involved in the dashed frame B2.

[0184] 3) Based on the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel of each adjacent first image block, determine the corresponding multiple candidate third gray values ​​of the pixel.

[0185] See Figure 9 As shown, the mapping function of adjacent blocks is weighted and fused by bilinear interpolation: For a pixel p(x,y) in the image, the mapping tables Map1, Map2, Map3, and Map4 of its four adjacent first image blocks A1, A2, A3, and A4 are found, and the four candidate third gray values ​​Map1[k], Map2[k], Map3[k], and Map4[k] after mapping and matching based on the four mapping tables are determined respectively.

[0186] 4) Determine the interpolation weights for the candidate third grayscale value corresponding to each adjacent first image block based on the distance between the pixel and the center point of the adjacent first image block. Calculate the bilinear interpolation weights w1, w2, w3, and w4 based on the distances from the pixel to the center points of the four adjacent first image blocks.

[0187] 5) Based on the interpolation weight, multiple candidate third gray values ​​are weighted and calculated to obtain the adjusted third gray value of the pixel.

[0188] The candidate third grayscale values ​​are weighted and calculated to obtain the final third grayscale value Map(x, y), thus avoiding block boundary artifacts. Compared with global histogram matching, this method can better preserve local image details and significantly enhance local contrast. Specifically:

[0189] .

[0190] 6) Based on the third grayscale value after adjusting all pixels in each component channel, the output image after color style adjustment is obtained.

[0191] By using local histogram matching after image segmentation and bilinear interpolation for image mapping adjustment, local contrast can be further improved.

[0192] In some optional embodiments, the above method can also use histogram matching as a preprocessing stage, followed by deep neural network for feature refinement, that is, the adjusted image output from step S14 is refined using a deep neural network.

[0193] Using histogram matching as a preprocessing stage, followed by deep neural network for feature refinement, specifically includes the following steps:

[0194] 1) Obtain the input image to be enhanced;

[0195] 2) Perform color space conversion on the input image (e.g., but not limited to RGB to YUV);

[0196] 3) Perform histogram matching on each component channel (Y, U, V channels) to obtain the preliminary enhanced luminance component;

[0197] 4) Input the image after histogram matching preprocessing (i.e., the output image of step S14) into the deep neural network model;

[0198] 5) Deep neural networks perform end-to-end image retouching, including but not limited to: detail enhancement, noise suppression, artifact removal, local contrast optimization, and color correction at least one of the following:

[0199] 6) Output the final enhanced image.

[0200] Based on the same inventive concept, embodiments of the present invention also provide an image color style adjustment device based on ISP, such as... Figure 2 As shown, it includes:

[0201] Image acquisition module 21 is used to acquire target images from the second imaging system and source images from the first imaging system;

[0202] The data acquisition module 22 is used to convert the source image to a specified color space; based on the image data in the specified color space, it counts the number of pixels appearing at each preset first gray level in each component channel of the source image to obtain the first histogram distribution data; based on the first histogram distribution data, it determines the probability density of each first gray level, and obtains the first probability density cumulative distribution data based on the probability density of each first gray level.

[0203] The data acquisition module 22 is further configured to acquire pre-configured second probability density cumulative distribution data; or convert the source image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the source image to obtain second histogram distribution data; determine the probability density of each second gray level according to the second histogram distribution data, and obtain the second probability density cumulative distribution data according to the probability density of each second gray level.

[0204] The mapping relationship construction module 23 is used to obtain a gray level mapping table from the source image to each component channel of the target image based on the first probability density cumulative distribution data and the second probability density cumulative distribution data; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image;

[0205] The style adjustment module 24 is used to determine the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, so as to obtain the output image after color style adjustment.

[0206] This invention provides a computer storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described ISP-based image color style adjustment method.

[0207] This invention provides a color style adjustment device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described ISP-based image color style adjustment method.

[0208] The purpose of this invention is to provide a rapid color style adjustment method based on an image signal processor (ISP). Taking the YUV color space as an example, it utilizes the decoupling characteristic of Y (luminance) and UV (chromatic difference) in the YUV color space to perform precise histogram matching (CDF mapping) of the YUV channels, achieving unified control of luminance contrast optimization and color fidelity. Furthermore, it establishes a standardized color style transfer process to address the significant color style differences between endoscopic imaging systems from different manufacturers, providing clinicians with a consistent and stable visual experience, thereby improving diagnostic efficiency and surgical safety.

[0209] The above method can be implemented on FPGA, CPU, or GPU; taking FPGA implementation as an example. The hardware pipeline architecture of the above method can synchronously and parallelly run the four stages of statistics, CDF calculation, LUT generation, and lookup mapping. That is, the steps of obtaining the first and second histogram distribution data, obtaining the first and second probability density cumulative distribution data, obtaining the grayscale level mapping table of each component channel from the source image to the target image, and determining the adjusted third grayscale value of each pixel can all run synchronously and parallelly. Combined with Block Random Access Memory (BRAM), it supports single-cycle read and write of multiple probability density cumulative distribution data and multiple grayscale level mapping tables, realizing parallel processing of multiple component channels. For example, it supports single-cycle read and write of 256 histograms and LUT tables, achieving true parallel processing of the Y, U, and V channels. Simultaneously, FPGA, CPU, or GPU can be directly embedded in the ISP pipeline, completing processing within the same clock domain as the data stream of the specified color space, such as processing within the same clock domain as the YUV domain data stream, achieving zero-copy, zero-bus transmission latency on-chip integration.

[0210] Unless otherwise specifically stated, terms such as processing, calculation, operation, determination, display, etc., may refer to the actions and / or processes of one or more processing or computing systems or similar devices that represent the manipulation and conversion of data representing physical (e.g., electronic) quantities within the registers or memory of the processing system into other data similarly representing physical quantities within the memory, registers, or other such information storage, transmission, or display devices of the processing system. Information and signals can be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof.

[0211] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0212] In the detailed description above, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features in a single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, with each claim representing a separate preferred embodiment of the invention.

[0213] Those skilled in the art will also understand that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in alternative ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.

[0214] The steps of the methods or algorithms described in conjunction with the embodiments herein can be directly embodied in hardware, software modules executed by a processor, or a combination thereof. The software modules can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in a user terminal. Alternatively, the processor and storage medium can exist as discrete components in the user terminal.

[0215] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. This software code can be stored in memory units and executed by a processor. The memory units can be implemented within the processor or outside the processor; in the latter case, they are communicatively coupled to the processor via various means, as is well known in the art.

[0216] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

Claims

1. A method for adjusting image color style based on ISP, characterized in that, include: Acquire the source image from the first imaging system and convert the source image to the specified color space; Based on image data in a specified color space, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain the first histogram distribution data; based on the first histogram distribution data, the probability density of each first gray level is determined, and the first probability density cumulative distribution data is obtained based on the probability density of each first gray level. Acquire the second probability density cumulative distribution data of the pre-configured target image; or, acquire the target image from the second imaging system and convert the target image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain the second histogram distribution data; determine the probability density of each second gray level based on the second histogram distribution data, and obtain the second probability density cumulative distribution data based on the probability density of each second gray level. Based on the first probability density cumulative distribution data and the second probability density cumulative distribution data, a gray level mapping table is obtained for each component channel of the source image to the target image; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image. Based on the first grayscale value of each pixel in each component channel of the source image and the corresponding grayscale level mapping table, the adjusted third grayscale value of each pixel is determined to obtain the output image after color style adjustment.

2. The method as described in claim 1, characterized in that, The first imaging system and the second imaging system are two different endoscope systems. The conversion of the source image to the specified color space includes: matching a pre-set conversion formula according to the original color space and the converted color space of the source image, and converting the source image to the specified color space according to the conversion formula. The step of converting the target image to a specified color space includes: matching a pre-defined conversion formula with the original color space and the converted color space of the target image, and converting the target image to the specified color space according to the conversion formula.

3. The method as described in claim 1, characterized in that, The original color space of the source image and the original color space of the target image are both RGB color spaces. After converting the source image and the target image to a specified color space, if the specified color space after conversion is a YUV color space, then the Y, U, and V channels of the source image and the target image respectively satisfy the following relationships: In the formula, R, G, and B are the red, green, and blue component channels of the original color space; a1, b1, and c1 are preset conversion coefficients, which respectively satisfy the specified range, and the sum of the three is 1. a2, b2, and c2 are preset conversion coefficients, each satisfying a specified range, and the sum of the three is 0; a3, b3, and c3 are preset conversion coefficients that satisfy the specified range, and the sum of the three is 0.

4. The method as described in claim 1, characterized in that, The first gray level is set according to the number of bits in the source image, and the second gray level is set according to the number of bits in the target image. Each first gray level includes one or several adjacent gray values, and each second gray level includes one or several adjacent gray values. The first histogram distribution data and the second histogram distribution data include at least one of the following: uniformly distributed image histogram, Gaussian distributed image histogram, log-normal distributed image histogram, exponentially distributed image histogram, and image histogram of a custom shape.

5. The method as described in claim 1, characterized in that, The step of determining the probability density of each first gray level based on the first histogram distribution data, and obtaining the first probability density cumulative distribution data based on the probability density of each first gray level, includes: For each first gray level k1 of each component channel: divide the number of pixels appearing at the first gray level by the total number of pixels in the source image to obtain the probability density of the first gray level; sum all probability densities from 0 to the first gray level k1 to obtain the cumulative probability density value of the first gray level; the cumulative probability density values ​​of all first gray levels constitute the first probability density cumulative distribution data. Based on the second histogram distribution data, the probability density of each second gray level is determined, and the cumulative distribution data of the second probability density is obtained based on the probability density of each second gray level, including: For each second gray level k2 of each component channel: divide the number of pixels appearing at the second gray level by the total number of pixels in the target image to obtain the probability density of the second gray level; sum all the probability densities from 0 to the second gray level k2 to obtain the cumulative probability density value of the second gray level; the cumulative probability density values ​​of all second gray levels constitute the second probability density cumulative distribution data.

6. The method as described in claim 1, characterized in that, The step of obtaining the grayscale level mapping table for each component channel of the source image and the target image based on the first probability density cumulative distribution data and the second probability density cumulative distribution data includes: Cumulative probability density for each first gray level of each component channel of the source image: Find the position in the second probability density cumulative distribution data that is closest to the cumulative probability density value, and establish a mapping relationship between the second gray level at that position and the corresponding first gray level. After establishing a mapping relationship for each first gray level, a gray level mapping table for each component channel of the source image and the target image is obtained.

7. The method as described in claim 6, characterized in that, The step of determining the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table to obtain the color style adjusted output image includes: For each pixel in each component channel of the source image: determine the corresponding second gray level according to the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel, and use a gray value in the determined second gray level as the adjusted third gray value of the pixel; The color-style adjusted output image is obtained based on the third grayscale value after adjusting all pixels in each component channel.

8. The method as described in claim 1, characterized in that, After converting the source image to the specified color space, the process also includes: The source image is divided into multiple first image blocks; correspondingly... The method of statistically analyzing the number of pixels appearing at each preset first gray level in each component channel of the source image to obtain the first histogram distribution data includes: for each first image block in the source image, statistically analyzing the number of pixels appearing at each preset first gray level in each component channel of the first image block to obtain the first histogram distribution data of each first image block. The step of obtaining the first probability density cumulative distribution data includes: obtaining the first probability density cumulative distribution data of each first image block in the source image; After converting the target image to the specified color space, the process also includes: The target image is divided into multiple second image blocks, and each second image block in the target image corresponds one-to-one with a first image block in the source image; accordingly. The method of statistically analyzing the number of pixels appearing at each preset second gray level in each component channel of the target image to obtain second histogram distribution data includes: for each second image block in the target image, statistically analyzing the number of pixels appearing at each preset second gray level in each component channel of each second image block to obtain second histogram distribution data for each second image block. The step of obtaining the second probability density cumulative distribution data includes: obtaining the second probability density cumulative distribution data for each second image block in the target image; Accordingly, obtaining the grayscale mapping table of each component channel from the source image to the target image includes: obtaining the grayscale mapping table of each component channel of each first image block in the source image and the corresponding second image block in the target image.

9. The method as described in claim 8, characterized in that, The step of determining the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table to obtain the color style adjusted output image includes: For each pixel of each component channel in the source image: Based on the first image block to which the pixel belongs, determine the multiple adjacent first image blocks to which it belongs; Based on the first gray level corresponding to the first gray value of the pixel and the gray level mapping table of the component channel of each adjacent first image block, determine the corresponding multiple candidate third gray values ​​of the pixel. Based on the distance between the pixel and the center point of the adjacent first image block, determine the interpolation weight for the corresponding candidate third gray value of each adjacent first image block; Based on the interpolation weights, multiple candidate third gray values ​​are weighted and calculated to obtain the adjusted third gray value of the pixel. The color-style adjusted output image is obtained based on the third grayscale value after adjusting all pixels in each component channel.

10. The method as described in claim 1, characterized in that, Also includes: The number of pixels appearing at each preset third gray level in each component channel of the output image is counted to obtain the third histogram distribution data; The similarity between the third histogram distribution data and the second histogram distribution data is greater than the set similarity threshold.

11. The method as described in claim 10, characterized in that, The similarity ρ between the third histogram distribution data and the second histogram distribution data is determined using the following formula: ； In the formula, This represents the number of pixels with grayscale value i in the output image. Let i be the number of pixels with grayscale value i in the target image. This is the average grayscale value of all pixels in the output image. L is the average grayscale value of all pixels in the target image, and L is the number of discrete levels for each channel.

12. The method as described in claim 10, characterized in that, Also includes: At least one of the first histogram distribution data, the second histogram distribution data, the third histogram distribution data, the first probability density cumulative distribution data, and the second probability density cumulative distribution data is smoothed, and the smoothing process includes at least one of Gaussian filtering, median filtering, and mean filtering.

13. The method as described in claim 1, characterized in that, This method can be implemented on an FPGA, CPU, or GPU; where: The steps of obtaining the first and second histogram distribution data, obtaining the first and second probability density cumulative distribution data, obtaining the gray level mapping table of each component channel of the source image and the target image, and determining the adjusted third gray value of each pixel are run synchronously and in parallel. By combining random access memory (BRAM), it supports single-cycle read and write of multiple probability density cumulative distribution data and multiple gray level mapping tables, and realizes parallel processing of multiple component channels. FPGAs, CPUs, or GPUs can be embedded in the ISP pipeline to complete processing within the same clock domain as the data stream in the specified color space, achieving on-chip integration with zero copy and zero bus transmission latency.

14. An image color style adjustment device based on ISP, characterized in that, include: The image acquisition module is used to acquire a target image from the second imaging system and a source image from the first imaging system; The data acquisition module is used to convert the source image to a specified color space; Based on image data in a specified color space, the number of pixels appearing at each preset first gray level in each component channel of the source image is counted to obtain the first histogram distribution data; based on the first histogram distribution data, the probability density of each first gray level is determined, and the first probability density cumulative distribution data is obtained based on the probability density of each first gray level. The data acquisition module is further configured to acquire pre-configured second probability density cumulative distribution data; or, convert the source image to a specified color space; based on the image data in the specified color space, count the number of pixels appearing at each preset second gray level in each component channel of the source image to obtain second histogram distribution data; determine the probability density of each second gray level based on the second histogram distribution data, and obtain the second probability density cumulative distribution data based on the probability density of each second gray level. The mapping relationship construction module is used to obtain a gray level mapping table from the source image to each component channel of the target image based on the first probability density cumulative distribution data and the second probability density cumulative distribution data; the gray level mapping table includes the second gray level in the target image corresponding to each first gray level of the source image; The style adjustment module is used to determine the adjusted third gray value of each pixel based on the first gray value of each pixel in each component channel of the source image and the corresponding gray level mapping table, so as to obtain the output image after color style adjustment.

15. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which, when executed by a processor, implement the ISP-based image color style adjustment method according to any one of claims 1-13.

16. A color style adjustment device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the ISP-based image color style adjustment method according to any one of claims 1-13.

Images