Image processing method, apparatus, device, storage medium and program product

Abstract

The present disclosure relates to an image processing method, device, equipment, storage medium and program product. According to an example of the present disclosure, the method comprises: in response to determining a target image style, acquiring a to-be-processed image, the target image style comprising a currently selected desired image style; in response to acquiring the to-be-processed image, processing the to-be-processed image based on a target lookup table to obtain a style conversion image of the target image style; wherein the target lookup table comprises a pre-stored lookup table corresponding to the target image style, and the target lookup table is generated based on a difference between the target image style and an initial image style and a basic lookup table. The present disclosure can improve the efficiency of generating a target lookup table, reduce the research and development cost, improve the efficiency of image style conversion, save system computing resources, and thus improve the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, and in particular to an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] As consumers' aesthetic appreciation of images improves, terminal devices such as smartphones that support camera shooting functions are gradually shifting from pursuing hard indicators such as hardware specifications and DxOMark scores to the aesthetic level of image stylization. Image style has become the main image label of current terminal devices.

[0003] In related technologies, image stylization is typically implemented at the software level, such as by integrating a pre-defined post-processing algorithm after the processing flow of the Image Signal Processor (ISP) platform. This approach requires style conversion of the image first on the ISP platform and then in the post-processing algorithm, resulting in low efficiency of image style conversion, consuming significant system computing resources, and consequently impacting user experience. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, the present disclosure provides an image processing method, apparatus, device, storage medium, and program product to solve the defects in the related technologies.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, the method comprising:

[0006] In response to determining the target image style, an image to be processed is obtained, wherein the target image style includes the currently selected desired image style;

[0007] In response to obtaining the image to be processed, the image to be processed is processed based on the target lookup table to obtain a style-transformed image of the target image style;

[0008] The target lookup table includes a pre-stored lookup table corresponding to the target image style. The target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. The initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image to the initial image style.

[0009] In some embodiments, processing the image to be processed based on the target lookup table includes:

[0010] The image to be processed is processed by a preset hardware processing unit based on a target lookup table, which is stored in the preset hardware processing unit.

[0011] In some embodiments, the preset hardware processing unit includes at least one of the following:

[0012] Image processor (ISP), display, image processor (GPU).

[0013] In some embodiments, the method further includes:

[0014] In response to receiving a preset instruction, the system enters a specified shooting mode and displays at least one alternative image style. The preset instruction includes an instruction triggered in the shooting preview interface that indicates entry into the specified shooting mode.

[0015] In response to detecting an operation that selects any image style from the at least one alternative image style, the any image style is determined as the target image style.

[0016] In some embodiments, the method further includes:

[0017] The style-transformed image is output to the display screen for display.

[0018] The style-transformed image is output to either the built-in or external storage unit for storage.

[0019] The style-transferred image is transmitted to a central processing unit (CPU) or a graphics processing unit (GPU) for processing.

[0020] In some embodiments, processing the image to be processed based on a target lookup table to obtain a style-transformed image of the target image style includes:

[0021] The color value of each pixel in the image to be processed is used as the input color value to look up the target lookup table, so as to obtain the output color value corresponding to each pixel.

[0022] The output color value corresponding to each pixel is determined as the color value of each pixel in the style-transferred image, thus obtaining the style-transferred image.

[0023] According to a second aspect of the present disclosure, a lookup table storage method is provided, the method being used to store a target lookup table in the image processing method described in any one aspect of the first aspect;

[0024] The method includes:

[0025] In response to detecting a specified target image style, a first sample image with an initial image style and a second sample image with the target image style are acquired;

[0026] An offset lookup table is determined based on the first sample image and the second sample image, and the offset lookup table is used to reflect the difference between the initial image style and the target image style;

[0027] In a specified color space, the difference is applied to a base lookup table based on the offset lookup table to obtain the target lookup table, wherein the base lookup table is used to convert the style of the image to be processed into the style of the initial image.

[0028] The target lookup table is stored.

[0029] In some embodiments, applying the difference to a base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table includes:

[0030] Based on the base lookup table and the offset lookup table, color style transfer is performed on the original color sample set to obtain the target color sample set. The original color sample set includes the color sample set obtained by sampling in the specified color space.

[0031] The target lookup table is determined based on the mapping relationship between the original color sample set and the target color sample set.

[0032] In some embodiments, the step of performing color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set includes:

[0033] The original color sample set is subjected to interpolation operation through the basic lookup table to obtain a first color sample set that conforms to the initial image style;

[0034] The first color sample set is interpolated using the offset lookup table to obtain the target color sample set that conforms to the style of the target image.

[0035] In some embodiments, the subject in the first sample image is the same as that in the second sample image;

[0036] The step of determining the offset lookup table based on the first sample image and the second sample image includes:

[0037] The first sample image and the second sample image are respectively converted to the specified color space to obtain the first converted image and the second converted image;

[0038] The offset lookup table is determined based on the pixel mapping relationship from the first converted image to the second converted image.

[0039] In some embodiments, storing the target lookup table includes:

[0040] Construct the correspondence between the target lookup table and the target image style;

[0041] Based on the correspondence, the target lookup table is pushed to the device to which the preset hardware processing unit belongs, so that the device can store the target lookup table.

[0042] In some embodiments, the method of acquiring the second sample image includes at least one of the following:

[0043] The second sample image is obtained by photographing the subject using a reference camera.

[0044] The first sample image is edited using preset image editing software to obtain the second sample image.

[0045] According to a third aspect of the present disclosure, an image processing apparatus is provided, the apparatus comprising:

[0046] An image acquisition module is used to acquire an image to be processed in response to determining a target image style, wherein the target image style includes the currently selected desired image style;

[0047] The image processing module is used to process the image to be processed based on a target lookup table in response to acquiring the image to be processed, so as to obtain a style transfer image of the target image style;

[0048] The target lookup table includes a pre-stored lookup table corresponding to the target image style. The target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. The initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image to the initial image style.

[0049] In some embodiments, the image processing module is further configured to process the image to be processed based on a target lookup table by a preset hardware processing unit, wherein the target lookup table is stored in the preset hardware processing unit.

[0050] In some embodiments, the preset hardware processing unit includes at least one of the following:

[0051] Image processor (ISP), display, image processor (GPU).

[0052] In some embodiments, the apparatus further includes a style determination module;

[0053] The style determination module includes:

[0054] A style display unit is used to respond to a received preset instruction, enter a specified shooting mode, and display at least one alternative image style. The preset instruction includes an instruction triggered in the shooting preview interface to indicate entering the specified shooting mode.

[0055] A style determination unit is configured to determine the target image style in response to detecting an operation of selecting any image style from the at least one alternative image styles.

[0056] In some embodiments, the apparatus further includes an image output module, the image output module being used for:

[0057] The style-transformed image is output to the display screen for display.

[0058] The style-transformed image is output to either the built-in or external storage unit for storage.

[0059] The style-transferred image is transmitted to a central processing unit (CPU) or a graphics processing unit (GPU) for processing.

[0060] In some embodiments, the image processing module includes:

[0061] A color value lookup unit is used to look up the color value of each pixel in the image to be processed as an input color value in the target lookup table to obtain the output color value corresponding to each pixel.

[0062] An image processing unit is used to determine the output color value corresponding to each pixel as the color value of each pixel in the style-transferred image, thereby obtaining the style-transferred image.

[0063] According to a fourth aspect of the present disclosure, a lookup table storage device is provided, the device being used to store a target lookup table in the image processing apparatus described in any third aspect;

[0064] The device includes:

[0065] The sample image acquisition module is used to acquire a first sample image with an initial image style and a second sample image with the target image style in response to detecting a specified target image style;

[0066] An offset table determination module is used to determine an offset lookup table based on the first sample image and the second sample image, wherein the offset lookup table is used to reflect the difference between the initial image style and the target image style;

[0067] The target table determination module is used to apply the difference to the base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table. The base lookup table is used to convert the style of the image to be processed into the style of the initial image.

[0068] The target table storage module is used to store the target lookup table.

[0069] In some embodiments, the target table determination module includes:

[0070] The target set acquisition unit is used to perform color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set. The original color sample set includes the color sample set obtained by sampling in the specified color space.

[0071] The target table determination unit is used to determine the target lookup table based on the mapping relationship between the original color sample set and the target color sample set.

[0072] In some embodiments, the target set acquisition unit is further configured to:

[0073] The original color sample set is subjected to interpolation operation through the basic lookup table to obtain a first color sample set that conforms to the initial image style;

[0074] The first color sample set is interpolated using the offset lookup table to obtain the target color sample set that conforms to the style of the target image.

[0075] In some embodiments, the subject in the first sample image is the same as that in the second sample image;

[0076] The offset table determination module includes:

[0077] A sample image conversion unit is used to convert the first sample image and the second sample image to the specified color space respectively, to obtain a first converted image and a second converted image;

[0078] The offset table determination unit is used to determine the offset lookup table based on the pixel mapping relationship from the first converted image to the second converted image.

[0079] In some embodiments, the target table storage module includes:

[0080] The correspondence construction unit is used to construct the correspondence between the target lookup table and the target image style;

[0081] The target table storage unit is used to push the target lookup table to the device to which the preset hardware processing unit belongs based on the correspondence, so that the device can store the target lookup table.

[0082] In some embodiments, the second sample image is acquired in at least one of the following ways:

[0083] The second sample image is obtained by photographing the subject using a reference camera.

[0084] The first sample image is edited using preset image editing software to obtain the second sample image.

[0085] According to a fifth aspect of the present disclosure, an electronic device is provided, the device comprising:

[0086] Processor and memory used to store computer programs;

[0087] The processor is configured to implement the image processing method described in any one of the first aspects when executing the computer program; or, the processor is configured to implement the lookup table storage method described in any one of the second aspects when executing the computer program.

[0088] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, it implements the image processing method described in any one of the first aspects; or, when the program is executed by a processor, it implements the lookup table storage method described in any one of the second aspects.

[0089] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the image processing method described in any one of the first aspects; or, when executed by a processor, the computer program implements the lookup table storage method described in any one of the second aspects.

[0090] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0091] In this disclosure, since the target lookup table is generated based on the difference between the target image style and the initial image style, as well as the base lookup table, the color adjustment parameters of the base lookup table can be reused as much as possible during the generation of the target lookup table. This improves the generation efficiency of the target lookup table and reduces R&D costs. Moreover, the image to be processed only needs to be transformed once by the target lookup table to obtain the style-transformed image of the target image style, without having to first transform the image to be processed to the initial image style and then to the target image style. This improves the efficiency of image style transformation while ensuring the image style transformation effect, saves system computing resources, and thus enhances the user experience.

[0092] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0093] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0094] Figure 1A This is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure;

[0095] Figure 1B This is a flowchart illustrating the processing of the image to be processed based on a target lookup table according to an exemplary embodiment of the present disclosure;

[0096] Figure 2A This is a flowchart illustrating an image processing method according to yet another exemplary embodiment of the present disclosure;

[0097] Figure 2B This is a schematic diagram of a photographing process according to an exemplary embodiment of the present disclosure;

[0098] Figure 3A This is a flowchart illustrating a lookup table storage method according to an exemplary embodiment of the present disclosure;

[0099] Figure 3B This is a flowchart illustrating the acquisition of a sample image according to an exemplary embodiment of the present disclosure;

[0100] Figure 3C This is a schematic diagram illustrating a lookup table storage method framework according to an exemplary embodiment of the present disclosure;

[0101] Figure 3D This is a schematic diagram illustrating an application scenario of a lookup table storage method according to an exemplary embodiment of the present disclosure;

[0102] Figure 4This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, how to apply the difference to a base lookup table based on the offset lookup table in a specified color space;

[0103] Figure 5 This is a flowchart illustrating, according to an exemplary embodiment of the present disclosure, how to perform color style transfer on an original color sample set based on a base lookup table and the offset lookup table;

[0104] Figure 6 This is a flowchart illustrating how to determine an offset lookup table based on the first sample image and the second sample image, according to an exemplary embodiment of the present disclosure;

[0105] Figure 7 This is a schematic diagram illustrating the stylized processing effect of a color chart image under a D65 light source, according to an exemplary embodiment of the present disclosure.

[0106] Figure 8 This is a schematic diagram illustrating the stylized processing effect of a color chart image under a TL84 light source, according to an exemplary embodiment of this disclosure.

[0107] Figure 9 This is a schematic diagram illustrating the stylized processing effect of a color chart image under light source A, according to an exemplary embodiment of this disclosure.

[0108] Figure 10 This is a schematic diagram illustrating the stylized processing effect of a blue sky scene image according to an exemplary embodiment of the present disclosure;

[0109] Figure 11 This is a schematic diagram illustrating the stylized processing effect of a green plant scene image according to an exemplary embodiment of the present disclosure;

[0110] Figure 12 This is a schematic diagram illustrating the stylized processing effect of an architectural scene image according to an exemplary embodiment of the present disclosure;

[0111] Figure 13 This is a schematic diagram illustrating the stylized processing effect of a ground scene image according to an exemplary embodiment of the present disclosure;

[0112] Figure 14 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure;

[0113] Figure 15 This is a block diagram illustrating yet another image processing apparatus according to an exemplary embodiment of the present disclosure;

[0114] Figure 16 This is a block diagram illustrating a lookup table storage device according to an exemplary embodiment of the present disclosure;

[0115] Figure 17 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0116] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0117] As consumers' aesthetic appreciation of images improves, terminal devices such as smartphones that support camera shooting functions are gradually shifting from pursuing hard indicators such as hardware specifications and DxOMark scores to the aesthetic level of image stylization. Image style has become the main image label of current terminal devices.

[0118] In related technologies, image stylization is typically implemented at the software level, such as by integrating pre-defined post-processing algorithms after the processing flow of an ISP (Image Signal Processor) platform. This approach, which integrates post-processing algorithms, is a software-level solution, which increases system integration costs and computing resources, and is not conducive to flexible adjustments later on.

[0119] For example, the overall architecture of a camera in a mobile phone or other terminal device includes both software and hardware layers. Software-layer solutions require adding software pathways to existing hardware platforms, increasing system power consumption and computing resources. This can lead to varying degrees of overheating and response latency issues, which can severely impact user experience. Furthermore, the flexibility of software-layer solutions for later adjustments is easily affected by various stages of software design and development. Poor code quality can result in difficulties in later maintenance. Moreover, software-layer solutions require significantly more manpower for regular maintenance compared to hardware-layer solutions, further increasing system integration costs.

[0120] Moreover, the above solution requires the image to undergo style conversion once on the ISP platform and then again in the post-processing algorithm, resulting in low efficiency of image style conversion, consuming more system computing resources, and thus affecting user experience.

[0121] In view of the above, this disclosure provides the following image processing methods, apparatus, devices, storage media, and program products to solve the aforementioned drawbacks in the related art.

[0122] Figure 1AThis is a flowchart illustrating an image processing method according to an exemplary embodiment. The method of this embodiment can be executed by an image processing device, which can be configured in an electronic device, such as a mobile terminal (e.g., mobile phone, tablet computer, etc.), wearable device (e.g., glasses, watch, etc.), vehicle terminal, camera, video camera, etc. Figure 1A As shown, the method includes the following steps S101-S102:

[0123] In step S101, in response to determining the target image style, the image to be processed is acquired.

[0124] In this embodiment, the electronic device can acquire an image to be processed in response to a user determining a target image style, wherein the target image style includes the desired image style currently selected by the user. For example, the method for selecting the target image style can be described below. Figure 2A The embodiments shown will not be described in detail here.

[0125] For example, the image to be processed includes the original image captured by the image sensor of an electronic device, or an image obtained by preprocessing the original image.

[0126] In step S102, in response to obtaining the image to be processed, the image to be processed is processed based on the target lookup table to obtain a style-transformed image of the target image style.

[0127] The target lookup table includes a pre-stored lookup table corresponding to the target image style. This target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. Therefore, it allows for the reuse of color adjustment parameters from the base lookup table (e.g., camera effect parameters, i.e., tuning parameters for camera developers to debug and verify on a local PC or server) as much as possible during the generation of the target lookup table, improving the efficiency of target lookup table generation and reducing development costs. For example, the initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image (e.g., an image captured by the camera's image sensor) into the initial image style.

[0128] As you can understand, a lookup table (LUT) is a data structure used in image processing and computer graphics that simplifies the color conversion process by mapping input color values ​​to output color values. An LUT is essentially an array or table where each entry defines a mapping from one input color value to one output color value. This mapping can be one-to-one or many-to-one, depending on the LUT's design and application requirements.

[0129] The aforementioned target lookup table can be a three-dimensional lookup table (3D LUT), capable of simultaneously affecting the R, G, and B color channels to achieve more complex and refined color transformations. For example, assuming the color value of a pixel in the image to be processed is (R1, G1, B1), then by using this color value as the input color value and looking it up in the target lookup table, the output color value (R2, G2, B2) can be obtained.

[0130] In some embodiments, a migration lookup table can be determined using an image with an initial image style and an image with a target image style. This migration lookup table reflects the difference between the two image styles (i.e., the offset image style), and a target lookup table can then be determined based on the base lookup table and the offset lookup table. This allows for the reuse of color adjustment parameters from the base lookup table as much as possible during the determination of the target lookup table, thereby improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0131] For example, suppose the process of generating a base lookup table based on the original image requires determining 300 color adjustment parameters (e.g., the effect of the base lookup table is to enhance the green areas in the image, such as increasing saturation), and the process of directly generating a target lookup table based on the original image requires determining 500 parameters (e.g., the effect of the target lookup table is to enhance both the green areas and the sky areas in the image). In this embodiment, by reusing the parameters of the base lookup table during the generation of the target lookup table (i.e., enhancing the sky areas in the image on top of the already enhanced green areas), only 200 color adjustment parameters may need to be determined. This obviously greatly reduces the number of parameters that need to be adjusted, thereby improving the generation efficiency of the target lookup table, saving system computing resources, reducing the investment of human and material resources in R&D, and thus reducing R&D costs.

[0132] For example, the method for determining the target lookup table described above can be found in the following... Figure 3A The embodiments shown will not be described in detail here.

[0133] Therefore, if the effect of the base lookup table is to increase the saturation of the greenery in the image, and the effect of the offset lookup table is to increase the saturation of the blue sky in the image, then the effect of the target lookup table in this embodiment can be to increase the saturation of both the greenery and the blue sky in the image. That is, the image to be processed only needs to be transformed once by the target lookup table to obtain the style-transformed image of the target image style, without having to first transform the image to be processed to the initial image style and then to the target image style. This can improve the efficiency of image style transformation while ensuring the image style transformation effect, save system computing resources, and thus improve the user experience.

[0134] For example, Figure 1B This is a flowchart illustrating the processing of the image to be processed based on a target lookup table according to an exemplary embodiment of this disclosure; such as Figure 1B As shown, the preset image processing flow includes the following steps 102a to 102b:

[0135] 102a. The color value of each pixel in the image to be processed is used as the input color value to search in the target lookup table to obtain the output color value corresponding to each pixel.

[0136] In this embodiment, after the target image style is determined, or after the image to be processed is obtained, the target lookup table can be retrieved, and then the color value of each pixel of the image to be processed can be used as the input color value to look up the corresponding output color value in the target lookup table to obtain the output color value corresponding to each pixel.

[0137] 102b. The output color value corresponding to each pixel is determined as the color value of each pixel in the style transfer image to obtain the style transfer image.

[0138] In this embodiment, after obtaining the output color value corresponding to each pixel, the output color value corresponding to each pixel can be determined as the color value of each pixel in the style transfer image, thereby obtaining the style transfer image.

[0139] Based on this, the electronic device can also perform at least one of the following image output methods (1) to (3):

[0140] (1) Output the style-transformed image to the display screen for display.

[0141] (2) The style-transformed image is output to the built-in storage unit or the external storage unit (e.g., an external SD card) for storage.

[0142] (3) The style-transferred image is transmitted to the central processing unit (CPU) or the image processing unit (GPU) for processing, such as further rendering or optimization by the CPU or GPU.

[0143] As described above, the method of this embodiment, because the target lookup table is generated based on the difference between the target image style and the initial image style, as well as the base lookup table, can reuse the color adjustment parameters of the base lookup table as much as possible during the generation of the target lookup table. This can improve the generation efficiency of the target lookup table and reduce development costs. Moreover, the image to be processed only needs to be transformed once by the target lookup table to obtain the style-transformed image of the target image style, without having to transform the image to be processed to the initial image style first and then to the target image style. This can improve the efficiency of image style transformation while ensuring the image style transformation effect, save system computing resources, and thus improve the user experience.

[0144] As mentioned earlier, the related technologies that integrate preset post-processing algorithms after the ISP (Image Signal Processor) platform processing flow are software-layer solutions. These solutions suffer from drawbacks such as low efficiency, high power consumption, and high maintenance costs. The main reasons are as follows:

[0145] ① The overall camera architecture of mobile phones and other terminal devices includes a software layer and a hardware layer. The software layer solution requires adding software paths to the existing hardware platform, which will increase power consumption, occupy system resources, and may even cause problems such as overheating and response delay to varying degrees, which will affect the user experience.

[0146] ② Software algorithms require regular manpower for maintenance. When project deadlines are tight, the software department may be short-staffed, thus focusing its efforts only on the development and maintenance of urgent issues. If the aforementioned software algorithms encounter problems, the lack of timely maintenance may affect normal operation and user experience.

[0147] Therefore, in this embodiment, after the electronic device acquires the image to be processed, it can process the image to be processed based on a target lookup table through a preset hardware processing unit.

[0148] In other words, after the electronic device acquires the image to be processed, it can execute the above steps 102a to 102b through the built-in preset hardware processing unit to realize the image color conversion process at the hardware layer.

[0149] For example, the aforementioned preset hardware processing unit may include at least one of the following (a) to (c):

[0150] (a) Image processor (ISP), (b) Display, (c) Image processor (GPU).

[0151] In some embodiments, the target lookup table described above may be stored in the preset hardware processing unit. For example, in the digital circuitry of the preset hardware processing unit, the target lookup table may be used as a memory to store the output color value corresponding to the input color value.

[0152] Understandably, compared to software solutions in related technologies that integrate post-processing algorithms after ISP processing, this solution pre-writes a target lookup table corresponding to the target image style into a preset hardware processing unit. This allows the preset hardware processing unit to process the image based on the target lookup table after the image to be processed is acquired, achieving hardware-level image color conversion. Compared to software-based color conversion solutions in related technologies, this solution does not require adding software pathways to existing hardware platforms, thus reducing system power consumption, minimizing system resource consumption, and improving computational efficiency (hardware lookup and other computational methods are generally faster than software computation). Furthermore, hardware solutions are easier to locate when problems occur, resulting in lower maintenance costs.

[0153] Figure 2A This is a flowchart illustrating an image processing method according to another exemplary embodiment of the present disclosure; the method of this embodiment can be executed by an image processing device, which can be configured in an electronic device, such as a mobile terminal (e.g., mobile phone, tablet computer, etc.), wearable device (e.g., glasses, watch, etc.), vehicle terminal, camera, video camera, etc. Figure 2A As shown, the method includes the following steps S201-S204:

[0154] In step S201, in response to receiving a preset instruction, the system enters a specified shooting mode and displays at least one alternative image style.

[0155] In this embodiment, the electronic device can respond to receiving a preset instruction, enter a specified shooting mode (e.g., stylized mode), and display at least one alternative image style.

[0156] The aforementioned preset instructions may include instructions triggered in the shooting preview interface to indicate entering the specified shooting mode.

[0157] In the above embodiments, the electronic device entering the specified shooting mode may include selecting a suitable lens according to the shooting scene, and / or adjusting at least one parameter such as ISO sensitivity, shutter speed, exposure compensation, white balance, etc. This embodiment does not limit this.

[0158] In step S202, in response to detecting the operation of selecting any image style from the at least one alternative image style, the any image style is determined as the target image style.

[0159] In this embodiment, after the electronic device displays at least one alternative image style, the user can trigger an operation to select any image style from the at least one alternative image style based on their own needs. Then, the electronic device can detect the operation and determine the any image style as the target image style.

[0160] For example, Figure 2B This is a schematic diagram of a photographing process according to an exemplary embodiment of the present disclosure; as shown below. Figure 2B As shown, when a user opens the camera on an electronic device, the device can display a shooting preview interface. The user can then choose whether to enter stylized mode on this preview interface. If the user chooses to enter stylized mode, they can send a preset command by triggering a preset function button, etc. The electronic device will then respond to the received preset command, enter the specified shooting mode, and display at least one alternative image style for the user to choose from. After the user selects the target image style, the electronic device will subsequently shoot and output a stylized image that meets the user's requirements.

[0161] However, if the user chooses not to enter stylized mode, the electronic device will take a picture and output a regular image according to the normal process.

[0162] In step S203, in response to determining the target image style, the image to be processed is acquired;

[0163] In step S204, in response to obtaining the image to be processed, the image to be processed is processed based on the target lookup table to obtain a style-transformed image of the target image style.

[0164] The target lookup table includes a pre-stored lookup table corresponding to the target image style. This target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. Therefore, it allows for the reuse of color adjustment parameters from the base lookup table (e.g., camera effect parameters, i.e., tuning parameters for camera developers to debug and verify on a local PC or server) as much as possible during the generation of the target lookup table, improving the efficiency of target lookup table generation and reducing development costs. For example, the initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image (e.g., an image captured by the camera's image sensor) into the initial image style.

[0165] The explanations and descriptions of steps S203-S204 above can be found in the above... Figure 1A Steps S101-S102 in the illustrated embodiment will not be described in detail here.

[0166] As described above, this embodiment, in response to receiving a preset instruction, enters a specified shooting mode and displays at least one alternative image style. Then, in response to detecting the operation of selecting any image style from the at least one alternative image style, the arbitrary image style is determined as the target image style. This can realize the determination of the target image style that meets the user's needs based on the user's selection, which is beneficial for the subsequent accurate and rapid acquisition of the style-converted image.

[0167] On the other hand, this disclosure also provides the following methods for storing the target lookup tables in the above embodiments.

[0168] Figure 3A This is a flowchart illustrating a lookup table storage method according to an exemplary embodiment of the present disclosure. The method of this embodiment can be executed by a lookup table storage device, which can be configured in an electronic device, such as a server, personal computer (PC), workstation, mobile terminal (e.g., mobile phone, tablet computer), wearable device (e.g., glasses, watch), vehicle terminal, camera, video camera, etc. The following description uses a server as an example. Figure 3A As shown, the method includes the following steps S301-S304:

[0169] In step S301, in response to detecting a specified target image style, a first sample image with an initial image style and a second sample image with the target image style are acquired.

[0170] In this embodiment, the server can, in response to detecting a user-specified target image style, obtain a first sample image with an initial image style and a second sample image with the target image style.

[0171] The initial image style includes known image styles used when the camera under development captures images; while the target image style includes the desired image style obtained using a reference camera or image editing software.

[0172] The first and second sample images mentioned above can both be color RGB images, RAW images, or YUV images, meaning the first and second sample images are of the same type. It is understood that if one sample image is an RGB image and the other is a RAW image, the RAW image can be preprocessed and transformed into an RGB image.

[0173] For example, Figure 3B This is a flowchart illustrating the acquisition of a sample image according to an exemplary embodiment of the present disclosure; as follows: Figure 3BAs shown, when a user selects an image style (e.g., target image style), the server can search in the Goldmark Image Library and then output an image pair consisting of a first sample image and a second sample image.

[0174] It's worth noting that the Goldmark image library can be a pre-built sample image library, which may include multiple sample image pairs. Each sample image pair contains one sample image in the initial image style and one sample image in the target image style. Understandably, different sample image pairs may contain the same or different target image styles. In other words, the Goldmark image library may include image pairs composed of sample images of various target image styles and sample images of the initial image style.

[0175] For example, the sample images of the aforementioned initial image style can be captured by the camera of a device such as a mobile phone under development (hereinafter referred to as "user device" for ease of distinction). The sample images of the aforementioned target image style can be obtained based on at least one of the following methods A and B:

[0176] A. The subject is photographed using a reference camera to obtain the second sample image.

[0177] The aforementioned reference camera can be selected based on business needs to achieve the required shooting effect. This embodiment does not limit the specific camera brand.

[0178] B. Edit the first sample image using preset image editing software to obtain the second sample image.

[0179] The preset image editing software can be set according to needs, such as Photoshop, 3DLUT Creator, etc., and this embodiment does not limit it.

[0180] In step S302, an offset lookup table is determined based on the first sample image and the second sample image.

[0181] In this embodiment, the server can determine an offset lookup table based on the first sample image and the second sample image in response to obtaining the first sample image and the second sample image.

[0182] The offset lookup table described above can be used to reflect the difference between the initial image style and the target image style.

[0183] In other embodiments, the method of determining the offset lookup table based on the first sample image and the second sample image described above can also be found in the following: Figure 6 The embodiments shown will not be described in detail here.

[0184] In step S303, in a specified color space, the difference is applied to the base lookup table based on the offset lookup table to obtain the target lookup table.

[0185] In this embodiment, after determining the offset lookup table based on the first sample image and the second sample image, the server can apply the difference to the base lookup table in a specified color space based on the offset lookup table to obtain the target lookup table.

[0186] The aforementioned base lookup table may include a pre-stored lookup table for converting the style of the image to be processed into the style of the initial image.

[0187] In other embodiments, the method of applying the difference to a base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table can also be found in the following: Figure 4 The embodiments shown will not be described in detail here.

[0188] In step S304, the target lookup table is stored.

[0189] In this embodiment, after the difference is applied to the base lookup table based on the offset lookup table to obtain the target lookup table, the server can store the target lookup table in the storage space of a mobile phone or other device so that the device can process the image to be processed based on the target lookup table to obtain the style-transformed image of the target image style.

[0190] In some embodiments, the server may store the target lookup table in the storage space of a preset hardware processing unit of a device such as a mobile phone.

[0191] It is worth noting that the storage space of the aforementioned preset hardware processing unit may vary depending on the type of the preset hardware processing unit, and this embodiment does not limit it.

[0192] For example, if the preset hardware processing unit is an image processor (ISP), its storage space can be the space in the color adjustment module inside the ISP used to store lookup tables (LUTs). This embodiment does not limit this.

[0193] It is understandable that the aforementioned preset hardware processing unit can also be a display or a GPU, and its storage space can be the video memory of the graphics card of the electronic device.

[0194] In some embodiments, a correspondence between the target lookup table and the target image style can be established, and then the target lookup table can be pushed to the device to which the preset hardware processing unit belongs, so that the device can store the target lookup table. Based on this, when a user selects a target image style, the preset hardware processing unit can find and use the target lookup table based on the aforementioned correspondence.

[0195] Understandably, the server can push the target lookup table to the user device before or after it leaves the factory, and the user device can then store the target lookup table in the storage space of the preset hardware processing unit. For example, before the user device leaves the factory, the server can compile the target lookup table into a bin file and push it to the user device, so that the user device has a target lookup table corresponding to the target image style at the time of manufacture. As new usage scenarios or user feedback issues may arise during the use of the user device, adjustments or optimizations to the image processing effect may be needed. At this time, the server can generate a new target lookup table again and push it to the user device through OTA (Over-The-Air) updates to achieve continuous optimization of the image processing effect.

[0196] Therefore, the target lookup table in this embodiment can be generated on the server and then pushed into the electronic device. This means the target lookup table generation process can be performed independently of the user device, eliminating the need to develop additional software pathways on the user device and improving development efficiency. Furthermore, the calculation process by the user device's preset hardware processing unit calling the compiled bin file does not consume additional computing resources from the user device system.

[0197] It is worth noting that, in addition to pushing the target lookup table to the user device, the target lookup table can also be transmitted to the user device via other wired or wireless means based on the scenario requirements. This embodiment does not limit this.

[0198] For example, Figure 3C This is a schematic diagram illustrating a lookup table storage method framework according to an exemplary embodiment of the present disclosure, such as... Figure 3C As shown, the framework mainly includes Figure 3A The illustrated embodiment includes a lookup table storage device and an ISP platform. It is understood that the ISP platform can be used to process image streams from user devices such as mobile phones during the imaging process, and may internally include one or more modules such as a distortion correction module, a brightness adjustment module, a color adjustment module, a focus module, and a noise reduction module (except for the color adjustment module, the other modules are not shown).

[0199] The color adjustment module, as a submodule of the ISP, performs calculations on the input image stream to obtain the output image stream, thereby completing the image color adjustment. By storing the color parameters to be adjusted in a built-in lookup table, system computing resources can be saved. In this embodiment, the ISP platform can provide the basic lookup table stored in the color adjustment module to the lookup table storage device to determine the target lookup table. Subsequently, it receives the target lookup table output by the lookup table storage device and rewrites it into the color adjustment module, so that the color adjustment module can use the target lookup table to process the image stream and achieve image color stylization.

[0200] Figure 3D This is a schematic diagram illustrating an application scenario of a lookup table storage method according to an exemplary embodiment of this disclosure. For example... Figure 3D As shown, the ISP platform can first provide the basic lookup table LUT1 stored in the color adjustment module to the lookup table storage device; after the lookup table storage device reads the basic lookup table LUT1, it can combine it with the first sample image I of the initial image style. src The second sample image I of the target image style tar Determine the target lookup table (LUT) out The data is then pushed to the ISP platform, which can then use the target lookup table (LUT). out This is written into the color adjustment module of the ISP platform. Based on this, the ISP platform can use this Target Lookup Table (LUT). out For the input image stream I in The process is performed to obtain the output image stream I. out This enables image color stylization.

[0201] Therefore, the lookup table storage device in this embodiment can be an offline computing device independent of the ISP platform. By acquiring the specified target style image and initial style image, and reading the basic lookup table in the color adjustment module of the ISP platform, and after offset lookup table calculation and target lookup table calculation, the target lookup table can be obtained and rewritten into the color adjustment module of the ISP platform for its use.

[0202] As described above, the method of this embodiment, in response to obtaining a first sample image of an initial image style and a second sample image of a target image style, determines an offset lookup table based on the first and second sample images, performs color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain a target color sample set, and then determines the target lookup table based on the mapping relationship from the original color sample set to the target color sample set. The target lookup table can then be stored. This method enables the determination of the target lookup table based on the base lookup table and the offset lookup table, thus maximizing the reuse of color adjustment parameters from the base lookup table during the determination process. This achieves the goals of improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0203] Furthermore, by storing the target lookup table corresponding to the target image style in a preset hardware processing unit for retrieval and use, convenient and efficient stylization processing of the image to be processed can be performed at the hardware level. Compared with the software-based solutions in related technologies, this can save system computing resources, reduce system power consumption, improve computing efficiency, and facilitate flexible adjustment of the contents of the target lookup table later.

[0204] Figure 4 This is a flowchart illustrating how to apply the difference to a base lookup table based on the offset lookup table in a specified color space, according to an exemplary embodiment of this disclosure. This embodiment is an illustrative example of how to apply the difference to a base lookup table based on the offset lookup table in a specified color space, based on the above embodiment.

[0205] like Figure 4 As shown, step S303 above, which describes applying the difference to the base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table, may include the following steps S401-S402:

[0206] In step S401, color style transfer is performed on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set.

[0207] In this embodiment, after determining the above offset lookup table, the server can perform color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set.

[0208] The aforementioned original color sample set may include a color sample set obtained by sampling in a specified color space.

[0209] For example, the server can sample in a specified color space based on a preset sampling range and a preset sampling interval to obtain the aforementioned original color sample set. The preset sampling range and preset sampling interval can be obtained through experimental verification, and this embodiment does not limit their determination.

[0210] Taking the HSL color space as an example, the preset sampling range can be "H:0~360; S:0~128; L:5~250" (that is, the sampling ranges corresponding to the H component, S component, and L component are 0~360, 0~128, and 5~250, respectively), and the preset sampling interval can be "H:10S:8L:16" (that is, the sampling intervals corresponding to the H component, S component, and L component are 10, 8, and 16, respectively). The set of three-dimensional coordinates corresponding to this original color sample set in the HSL space is (H,S,L).

[0211] It is worth noting that the HSL color space specified above is only for illustrative purposes. In applications, other color spaces can be specified based on scenario requirements, and this embodiment does not limit this.

[0212] The aforementioned basic lookup table may include a lookup table stored in the preset hardware processing unit for converting any image into the style of the initial image.

[0213] In other embodiments, the method of performing color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set can also be found in the following. Figure 5 The embodiments shown will not be described in detail here.

[0214] In step S402, the target lookup table is determined based on the mapping relationship between the original color sample set and the target color sample set.

[0215] In this embodiment, after obtaining the target color sample set, the server can determine the target lookup table based on the mapping relationship between the original color sample set and the target color sample set.

[0216] It is worth noting that the method of determining the target lookup table based on the mapping relationship from the original color sample set to the target color sample set can be selected from relevant technologies based on business needs, such as regression fitting method or calibration method, etc. This embodiment does not limit this.

[0217] As described above, this embodiment performs color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set, and determines the target lookup table based on the mapping relationship between the original color sample set and the target color sample set. This allows for the reuse of color adjustment parameters from the base lookup table as much as possible during the determination of the target lookup table, thereby improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0218] Figure 5 This is a flowchart illustrating how to perform color style transfer on an original color sample set based on a base lookup table and the offset lookup table, according to an exemplary embodiment of the present disclosure. This embodiment is based on the above embodiment and takes how to perform color style transfer on an original color sample set as an example.

[0219] like Figure 5 As shown, the color style transfer of the original color sample set based on the base lookup table and the offset lookup table described in step S401 above may include the following steps S501-S502:

[0220] In step S501, the original color sample set is subjected to interpolation operation of the basic lookup table to obtain a first color sample set that conforms to the initial image style.

[0221] In this embodiment, when performing color style transfer on the original color sample set based on the base lookup table and the offset lookup table, the original color sample set can be interpolated using the base lookup table to obtain a first color sample set that conforms to the initial image style.

[0222] Since the base lookup table stores the color mapping rules corresponding to the initial image style, the original color sample set can be interpolated using the base lookup table to obtain a color sample set that conforms to the initial image style (i.e., the first color sample set).

[0223] In step S502, the first color sample set is subjected to interpolation operation of the offset lookup table to obtain the target color sample set that conforms to the target image style.

[0224] In this embodiment, after obtaining the first color sample set that conforms to the initial image style, the first color sample set can be interpolated using the offset lookup table to obtain the target color sample set that conforms to the target image style.

[0225] Since the offset lookup table can reflect the difference between the initial image style and the target image style, the original color sample set can be interpolated using the basic lookup table to obtain a color sample set that conforms to the target image style (i.e., the target color sample set).

[0226] As described above, this embodiment obtains a first color sample set conforming to the initial image style by interpolating the original color sample set through the basic lookup table, and then obtains a target color sample set conforming to the target image style by interpolating the first color sample set through the offset lookup table. This allows for color style transfer of the original color sample set based on the basic lookup table and the offset lookup table, resulting in an accurate target color sample set. Subsequently, the target lookup table can be determined based on the mapping relationship between the original color sample set and the target color sample set. This allows for the reuse of color adjustment parameters from the basic lookup table as much as possible during the determination of the target lookup table, thereby improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0227] Figure 6 This is a flowchart illustrating how to determine an offset lookup table based on the first sample image and the second sample image according to an exemplary embodiment of the present disclosure; this embodiment is an exemplary description based on the above embodiment, taking how to determine the offset lookup table based on the first sample image and the second sample image as an example.

[0228] In this embodiment, the subject in the first sample image and the second sample image is the same, and thus... Figure 6 As shown, the step S302 above, which involves determining the offset lookup table based on the first sample image and the second sample image, may include the following steps S601-S602:

[0229] In step S601, the first sample image and the second sample image are converted to the specified color space to obtain the first converted image and the second converted image.

[0230] In this embodiment, after obtaining the first sample image of the initial image style and the second sample image of the target image style, the first sample image and the second sample image can be converted to the specified color space to obtain the first converted image and the second converted image.

[0231] In some embodiments, the specified color space can be selected as a three-dimensional Cartesian coordinate space, such as RGB space, HSL space or HSV space, that is, each color in the above sample image can be represented by a set of three-dimensional coordinates in this space.

[0232] In step S602, the offset lookup table is determined based on the pixel mapping relationship from the first converted image to the second converted image.

[0233] In this embodiment, after converting the first sample image and the second sample image to the specified color space to obtain the first converted image and the second converted image, the offset lookup table can be determined according to the pixel mapping relationship from the first converted image to the second converted image.

[0234] In some embodiments, the method of determining the offset lookup table based on the pixel mapping relationship from the first converted image to the second converted image can be selected from relevant technologies based on business requirements, such as regression fitting or calibration methods, etc. This embodiment does not limit this.

[0235] It is worth noting that in this embodiment, the first sample image and the second sample image are first converted to the specified color space to obtain the first converted image to the second converted image. Then, the offset lookup table is determined according to the pixel mapping relationship between the first converted image and the second converted image. This is because converting to the specified color space is more conducive to evaluating color from the perspective of human eye perception. Taking the HSL color space as an example, where H represents hue, S represents saturation, and L represents brightness, if a certain value is adjusted, the human eye can immediately perceive the corresponding change, thereby facilitating the problem of conveniently locating the lookup table according to the final effect in the subsequent development process.

[0236] As described above, this embodiment converts the first sample image and the second sample image to the specified color space to obtain a first converted image and a second converted image, and determines the offset lookup table based on the pixel mapping relationship between the first converted image and the second converted image. This allows for accurate determination of the offset lookup table based on the first sample image and the second sample image, and enables subsequent storage of the target lookup table based on the base lookup table and the offset lookup table. This allows for the reuse of color adjustment parameters from the base lookup table as much as possible during the determination of the target lookup table, thereby improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0237] The following explanation uses the ISP platform as the default hardware processing unit as an example.

[0238] Figure 7 This is a schematic diagram illustrating the stylized processing effect of a color chart image under a D65 light source, according to an exemplary embodiment of the present disclosure. Figure 8 This is a schematic diagram illustrating the stylized processing effect of a color chart image under a TL84 light source, according to an exemplary embodiment of this disclosure. Figure 9This is a schematic diagram illustrating the stylized processing effect of a color chart image under light source A, according to an exemplary embodiment of this disclosure.

[0239] like Figures 7 to 9 As shown, the sub-images numbered (a), (b), (c), and (d) in the figure are the target style image I. tar Initial style image I src Input image I of ISP platform in Output image I of ISP platform out , where the output image I out For ISP platforms based on target lookup table (LUT) out For input image I in The image obtained after stylization processing.

[0240] By comparing (a), (b), (c), and (d), it can be seen that under the three standard light sources of D65, TL84, and A, the output image (d) obtained by the target lookup table processing of this embodiment is closer to the target style image (a) in terms of the red, orange, green, and blue color blocks of its color card compared to the initial style image (b). This shows that this embodiment can complete effective stylization processing after specifying the target image style, and the processed image is closer to the specified target image style than the original image.

[0241] Figure 10 This is a schematic diagram illustrating the stylized processing effect of a blue sky scene image according to an exemplary embodiment of the present disclosure; Figure 11 This is a schematic diagram illustrating the stylized processing effect of a green plant scene image according to an exemplary embodiment of the present disclosure; Figure 12 This is a schematic diagram illustrating the stylized processing effect of an architectural scene image according to an exemplary embodiment of the present disclosure; Figure 13 This is a schematic diagram illustrating the stylized processing effect of a ground scene image according to an exemplary embodiment of the present disclosure.

[0242] like Figures 10 to 13 As shown, the sub-images numbered (a), (b), (c), and (d) in the figure are the target style image I. tar Initial style image I src Input image I of ISP platform in Output image I of ISP platform out , where the output image I out For ISP platforms based on target lookup table (LUT) out For input image I in The image obtained after stylization processing.

[0243] By comparison Figures 10 to 13As can be seen from (a), (b), (c), and (d) in the above, under the blue sky scene, green plant scene, building scene, and ground scene, the output image (d) obtained by processing the target lookup table of this embodiment is closer to the target style image (a) in color style than the initial style image (b), and is more vivid and lively. This shows that this embodiment can complete effective stylization processing after specifying the target image style, and the processed image is closer to the specified target image style than the original image.

[0244] In summary, this embodiment proposes an offline target lookup table generation method. By writing the generated target lookup table into the color adjustment module of hardware processing units such as ISP, convenient and efficient color stylization can be achieved at the hardware level.

[0245] Compared to pure software solutions in related technologies, this approach can reduce computational load, save system computing resources, and facilitate flexible adjustments to the target lookup table content later.

[0246] In the process of determining the target lookup table, by sampling in a specified color space to obtain the original color sample set and performing mapping calculations, it is possible to reuse more parameters of the initial image style, thereby reducing the number of color adjustment parameters that need to be debugged.

[0247] The idea of ​​using a lookup table (LUT) to achieve color style fusion in this embodiment is adapted to the color space of mainstream chips (such as ISP platforms), enabling flexible setting of color parameter adjustment frameworks and improving R&D efficiency.

[0248] Using user devices such as mobile phones in this embodiment, a filter-like photo effect can be achieved based on the target image style selected by the user when taking a photo, outputting a stylized photo that meets the user's requirements, thereby improving the user experience.

[0249] Figure 14 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure; the apparatus of this embodiment can be configured in an electronic device, such as a mobile terminal (e.g., a mobile phone, tablet computer, etc.), a wearable device (e.g., glasses, watches, etc.), an in-vehicle terminal, a camera, a video camera, etc. Figure 14 As shown, the device may include: an image acquisition module 110 and an image processing module 120, wherein:

[0250] Image acquisition module 110 is used to acquire an image to be processed in response to determining a target image style, wherein the target image style includes the currently selected desired image style;

[0251] Image processing module 120 is configured to process the image to be processed based on a target lookup table in response to acquiring the image to be processed, thereby obtaining a style transfer image of the target image style;

[0252] The target lookup table includes a pre-stored lookup table corresponding to the target image style. The target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. The initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image to the initial image style.

[0253] As described above, the device in this embodiment generates the target lookup table based on the difference between the target image style and the initial image style, as well as the base lookup table. Therefore, it can reuse the color adjustment parameters of the base lookup table as much as possible during the generation of the target lookup table, which can improve the generation efficiency of the target lookup table and reduce the development cost. Moreover, the image to be processed only needs to be transformed once by the target lookup table to obtain the style-transformed image of the target image style, without having to transform the image to be processed to the initial image style first and then to the target image style. This can improve the efficiency of image style transformation while ensuring the image style transformation effect, save system computing resources, and thus improve the user experience.

[0254] Figure 15 This is a block diagram illustrating another image processing apparatus according to an exemplary embodiment of the present disclosure; the apparatus of this embodiment can be configured in an electronic device, such as a mobile terminal (e.g., mobile phone, tablet computer, etc.), wearable device (e.g., glasses, watch, etc.), vehicle terminal, camera, video camera, etc. The image acquisition module 210 and image processing module 220 are as described above. Figure 14 The image acquisition module 110 and the image processing module 120 in the illustrated embodiment have the same function, which will not be described in detail here.

[0255] In some embodiments, the image processing module 220 can also be used to process the image to be processed based on a target lookup table through a preset hardware processing unit, wherein the target lookup table is stored in the preset hardware processing unit.

[0256] For example, the preset hardware processing unit includes, but is not limited to, at least one of the following:

[0257] Image processor (ISP), display, image processor (GPU).

[0258] like Figure 15 As shown, the device in this embodiment may further include a style determination module 230;

[0259] The style determination module 230 may include:

[0260] The style display unit 231 is used to respond to receiving a preset instruction, enter a specified shooting mode, and display at least one alternative image style. The preset instruction includes an instruction triggered in the shooting preview interface to indicate entering the specified shooting mode.

[0261] Style determination unit 232 is configured to determine the target image style in response to detecting an operation of selecting any image style from the at least one alternative image styles.

[0262] In some embodiments, the above-described apparatus may further include an image output module 240, which may be used for:

[0263] The style-transformed image is output to the display screen for display.

[0264] The style-transformed image is output to either the built-in or external storage unit for storage.

[0265] The style-transferred image is transmitted to a central processing unit (CPU) or a graphics processing unit (GPU) for processing.

[0266] In some embodiments, the image processing module 220 may include:

[0267] The color value lookup unit 221 is used to look up the color value of each pixel of the image to be processed in the target lookup table as the input color value to obtain the output color value corresponding to each pixel.

[0268] Image processing unit 222 is used to determine the output color value corresponding to each pixel as the color value of each pixel in the style transfer image, so as to obtain the style transfer image.

[0269] Figure 16 This is a block diagram illustrating a lookup table storage device according to an exemplary embodiment of the present disclosure; the lookup table storage device of this embodiment can be configured in an electronic device, such as a server, personal computer (PC), workstation, mobile terminal (such as mobile phone, tablet computer, etc.), wearable device (such as glasses, watch, etc.), vehicle terminal, camera, video camera, etc.

[0270] The lookup table storage device in this embodiment can be used to store the above-mentioned... Figure 14 or Figure 15 The target lookup table in the image processing apparatus of the illustrated embodiment.

[0271] like Figure 16As shown, the device may include: a sample image acquisition module 310, an offset table determination module 320, a target table determination module 330, and a target table storage module 340, wherein:

[0272] The sample image acquisition module 310 is configured to acquire a first sample image having an initial image style and a second sample image having the target image style in response to detecting a specified target image style.

[0273] The offset table determination module 320 is used to determine an offset lookup table based on the first sample image and the second sample image, wherein the offset lookup table is used to reflect the difference between the initial image style and the target image style;

[0274] The target table determination module 330 is used to apply the difference to the base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table. The base lookup table is used to convert the style of the image to be processed into the style of the initial image.

[0275] The target table storage module 340 is used to store the target lookup table.

[0276] As described above, the device in this embodiment, in response to detecting a specified target image style, acquires a first sample image with an initial image style and a second sample image with the target image style, determines an offset lookup table based on the first and second sample images, and then applies the difference to a base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table. The target lookup table is then stored. This allows for the determination of the target lookup table based on the base lookup table and the offset lookup table. This enables the reuse of color adjustment parameters from the base lookup table as much as possible during the determination process, thereby improving the efficiency of target lookup table generation, saving system computing resources, and reducing development costs.

[0277] Furthermore, by storing the target lookup table corresponding to the target image style in a preset hardware processing unit for retrieval and use, convenient and efficient stylization processing of the image to be processed can be performed at the hardware level. Compared with the software-based solutions in related technologies, this can save system computing resources, reduce system power consumption, improve computing efficiency, and facilitate flexible adjustment of the contents of the target lookup table later.

[0278] In some embodiments, the target table determination module 330 may include:

[0279] The target set acquisition unit 331 is used to perform color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set. The original color sample set includes the color sample set obtained by sampling in the specified color space.

[0280] The target table determination unit 332 is used to determine the target lookup table based on the mapping relationship from the original color sample set to the target color sample set.

[0281] In some embodiments, the target set acquisition unit 331 described above can also be used for:

[0282] The original color sample set is subjected to interpolation operation through the basic lookup table to obtain a first color sample set that conforms to the initial image style;

[0283] The first color sample set is interpolated using the offset lookup table to obtain the target color sample set that conforms to the style of the target image.

[0284] In some embodiments, the subject in the first sample image is the same as that in the second sample image;

[0285] Furthermore, the offset table determination module 320 may include:

[0286] The sample image conversion unit 321 is used to convert the first sample image and the second sample image to the specified color space respectively to obtain a first converted image and a second converted image;

[0287] The offset table determination unit 322 is used to determine the offset lookup table based on the pixel mapping relationship from the first converted image to the second converted image.

[0288] In some embodiments, the target table storage module 340 described above may include:

[0289] The correspondence construction unit 341 is used to construct the correspondence between the target lookup table and the target image style;

[0290] The target table storage unit 342 is used to push the target lookup table to the device to which the preset hardware processing unit belongs based on the correspondence, so that the device can store the target lookup table.

[0291] In some embodiments, the method of obtaining the second sample image may include at least one of the following:

[0292] The second sample image is obtained by photographing the subject using a reference camera.

[0293] The first sample image is edited using preset image editing software to obtain the second sample image.

[0294] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0295] Figure 17 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, device 900 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0296] Reference Figure 17 The device 900 may include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0297] Processing component 902 typically controls the overall operation of device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the image processing method described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0298] Memory 904 is configured to store various types of data to support the operation of device 900. Examples of this data include instructions for any application or method operating on device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0299] Power supply component 906 provides power to various components of device 900. Power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 900.

[0300] Multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display panel and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0301] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0302] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0303] Sensor assembly 914 includes one or more sensors for providing status assessments of various aspects of device 900. For example, sensor assembly 914 can detect the on / off state of device 900, the relative positioning of components such as the display panel and keypad of device 900, changes in the position of device 900 or a component of device 900, the presence or absence of user contact with device 900, the orientation or acceleration / deceleration of device 900, and temperature changes of device 900. Sensor assembly 914 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0304] Communication component 916 is configured to facilitate wired or wireless communication between device 900 and other devices. Device 900 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G or 5G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0305] In an exemplary embodiment, device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing method described above.

[0306] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of the device 900 to complete the image processing method described above. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0307] In an exemplary embodiment, a computer program product including instructions is also provided, which can be executed by the processor 920 of the device 900 to perform the image processing method described above.

[0308] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the foregoing claims.

[0309] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: In response to determining the target image style, an image to be processed is obtained, wherein the target image style includes the currently selected desired image style; In response to obtaining the image to be processed, the image to be processed is processed based on the target lookup table to obtain a style-transformed image of the target image style; The target lookup table includes a pre-stored lookup table corresponding to the target image style. The target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. The initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image to the initial image style.

2. The method according to claim 1, characterized in that, The processing of the image to be processed based on the target lookup table includes: The image to be processed is processed by a preset hardware processing unit based on a target lookup table, which is stored in the preset hardware processing unit.

3. The method according to claim 2, characterized in that, The preset hardware processing unit includes at least one of the following: Image processor (ISP), display, image processor (GPU).

4. The method according to claim 1, characterized in that, The method further includes: In response to receiving a preset instruction, the system enters a specified shooting mode and displays at least one alternative image style. The preset instruction includes an instruction triggered in the shooting preview interface that indicates entry into the specified shooting mode. In response to detecting an operation that selects any image style from the at least one alternative image style, the any image style is determined as the target image style.

5. The method according to claim 1, characterized in that, The method further includes: The style-transformed image is output to the display screen for display. The style-transformed image is output to either the built-in or external storage unit for storage. The style-transferred image is transmitted to a central processing unit (CPU) or a graphics processing unit (GPU) for processing.

6. The method according to claim 1, characterized in that, The process of processing the image to be processed based on the target lookup table to obtain the style-transformed image of the target image style includes: The color value of each pixel in the image to be processed is used as the input color value to look up the target lookup table, so as to obtain the output color value corresponding to each pixel. The output color value corresponding to each pixel is determined as the color value of each pixel in the style-transferred image, thus obtaining the style-transferred image.

7. A lookup table storage method, characterized in that, The method is used to store the target lookup table in the image processing method according to any one of claims 1 to 6; The method includes: In response to detecting a specified target image style, a first sample image with an initial image style and a second sample image with the target image style are acquired; An offset lookup table is determined based on the first sample image and the second sample image, and the offset lookup table is used to reflect the difference between the initial image style and the target image style; In a specified color space, the difference is applied to a base lookup table based on the offset lookup table to obtain the target lookup table, wherein the base lookup table is used to convert the style of the image to be processed into the style of the initial image. The target lookup table is stored.

8. The method according to claim 7, characterized in that, The step of applying the difference to a base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table includes: Based on the base lookup table and the offset lookup table, color style transfer is performed on the original color sample set to obtain the target color sample set. The original color sample set includes the color sample set obtained by sampling in the specified color space. The target lookup table is determined based on the mapping relationship between the original color sample set and the target color sample set.

9. The method according to claim 8, characterized in that, The process of performing color style transfer on the original color sample set based on the base lookup table and the offset lookup table to obtain the target color sample set includes: The original color sample set is subjected to interpolation operation through the basic lookup table to obtain a first color sample set that conforms to the initial image style; The first color sample set is interpolated using the offset lookup table to obtain the target color sample set that conforms to the style of the target image.

10. The method according to claim 7, characterized in that, The subject in the first sample image is the same as that in the second sample image; The step of determining the offset lookup table based on the first sample image and the second sample image includes: The first sample image and the second sample image are respectively converted to the specified color space to obtain the first converted image and the second converted image; The offset lookup table is determined based on the pixel mapping relationship from the first converted image to the second converted image.

11. The method according to claim 7, characterized in that, The storage of the target lookup table includes: Construct the correspondence between the target lookup table and the target image style; Based on the correspondence, the target lookup table is pushed to the device to which the preset hardware processing unit belongs, so that the device can store the target lookup table.

12. The method according to claim 7, characterized in that, The second sample image is obtained in at least one of the following ways: The second sample image is obtained by photographing the subject using a reference camera. The first sample image is edited using preset image editing software to obtain the second sample image.

13. An image processing apparatus, characterized in that, The device includes: An image acquisition module is used to acquire an image to be processed in response to determining a target image style, wherein the target image style includes the currently selected desired image style; The image processing module is used to process the image to be processed based on a target lookup table in response to acquiring the image to be processed, so as to obtain a style transfer image of the target image style; The target lookup table includes a pre-stored lookup table corresponding to the target image style. The target lookup table is generated based on the difference between the target image style and the initial image style, as well as a base lookup table. The initial image style includes the default image style used when the camera captures the image to be processed. The base lookup table is used to convert the style of the original image to the initial image style.

14. A lookup table storage device, characterized in that, The device is used to store the target lookup table in the image processing device of claim 13; The device includes: The sample image acquisition module is used to acquire a first sample image with an initial image style and a second sample image with the target image style in response to detecting a specified target image style; An offset table determination module is used to determine an offset lookup table based on the first sample image and the second sample image, wherein the offset lookup table is used to reflect the difference between the initial image style and the target image style; The target table determination module is used to apply the difference to the base lookup table based on the offset lookup table in a specified color space to obtain the target lookup table. The base lookup table is used to convert the style of the image to be processed into the style of the initial image. The target table storage module is used to store the target lookup table.

15. An electronic device, characterized in that, The device includes: Processor and memory used to store computer programs; The processor is configured to implement the image processing method according to any one of claims 1 to 6 when executing the computer program; or, the processor is configured to implement the lookup table storage method according to any one of claims 7 to 12 when executing the computer program.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image processing method according to any one of claims 1 to 6; or, when the program is executed by the processor, it implements the lookup table storage method according to any one of claims 7 to 12.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing method according to any one of claims 1 to 6; or, when the computer program is executed by the processor, it implements the lookup table storage method according to any one of claims 7 to 12.

Images