Image processing method, image processing device, storage medium, and electronic device

Abstract

This disclosure provides an image processing method, an image processing apparatus, a storage medium, and an electronic device, relating to the field of image and video processing technology. The image processing method includes: performing scene recognition on an image to be processed to determine a target scene corresponding to the image; obtaining color reference information corresponding to the target scene from pre-configured color reference information corresponding to preset scenes; extracting a region of interest (ROI) matching the target scene from the image to be processed, and determining color feature information of the ROI; determining color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the ROI; and performing color transformation processing on the image to be processed using the color transformation parameters. This disclosure can improve the problem of image colors deviating from expected colors, enhancing the visual experience and stability of the image.

Description

Technical Field

[0001] This disclosure relates to the field of image and video processing technology, and in particular to an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device. Background Technology

[0002] When capturing images, the lighting conditions in the shooting environment or the hardware limitations of the shooting device itself may affect the colors of the resulting images, causing them to deviate from the expected colors. Therefore, it is necessary to perform color-related processing on the images. Summary of the Invention

[0003] This disclosure provides an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device, thereby improving, at least to some extent, the problem of image colors deviating from the expected colors.

[0004] According to a first aspect of this disclosure, an image processing method is provided, comprising: acquiring an image to be processed; determining a target scene corresponding to the image to be processed in a preset scene; extracting a region of interest (ROI) matching the target scene from the image to be processed; determining color feature information of the ROI; acquiring color reference information corresponding to the target scene from pre-configured color reference information corresponding to the preset scene; determining color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the ROI; and performing color transformation processing on the image to be processed using the color transformation parameters.

[0005] According to a second aspect of this disclosure, an image processing apparatus is provided, comprising: a target scene determination module configured to acquire an image to be processed and determine a target scene corresponding to the image to be processed in a preset scene; a color feature information determination module configured to extract a region of interest matching the target scene from the image to be processed and determine color feature information of the region of interest; a color reference information acquisition module configured to acquire color reference information corresponding to the target scene from pre-configured color reference information corresponding to the preset scene; a color transformation parameter determination module configured to determine color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest; and a color transformation processing module configured to perform color transformation processing on the image to be processed using the color transformation parameters.

[0006] According to a third aspect of this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image processing method of the first aspect described above and its possible implementations.

[0007] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor. The processor is configured to perform the image processing method of the first aspect and its possible implementations thereof by executing the executable instructions.

[0008] The technical solution disclosed herein has the following beneficial effects:

[0009] On the one hand, this paper proposes a color transformation scheme for images, which can improve the problem of image colors deviating from the expected colors. On the other hand, this scheme obtains dynamic color transformation parameters based on the color feature information of the region of interest in the image that matches the target scene, as well as the color reference information corresponding to the target scene. This makes the color transformation parameters adaptable to the color features and scene in the image, thus having better targeting and improving the effect of image color transformation processing. Compared with the fixed 3DLUT method used in related technologies, this scheme can solve the image color cast problem caused by differences in shooting device hardware, differences in color-related algorithms in ISP, and differences in shooting environment, thereby improving the visual experience and stability of the image. Attached Figure Description

[0010] Figure 1 A schematic diagram of the system architecture in this exemplary embodiment is shown;

[0011] Figure 2 A flowchart illustrating an image processing method in this exemplary embodiment is shown;

[0012] Figure 3 This diagram illustrates the extraction of the region of interest in this exemplary embodiment.

[0013] Figure 4 This diagram illustrates the color distribution area in this exemplary embodiment.

[0014] Figure 5 This example illustrates a flowchart of configuring color reference information.

[0015] Figure 6 This example illustrates a flowchart for determining color transformation parameters.

[0016] Figure 7 A schematic diagram illustrating the color shifting in this exemplary embodiment is shown;

[0017] Figure 8 A schematic flowchart of the image processing method in this exemplary embodiment is shown;

[0018] Figure 9 An example diagram is shown comparing the image processing method in this exemplary embodiment with related technologies;

[0019] Figure 10 This diagram illustrates an image processing apparatus according to an exemplary embodiment of the present invention.

[0020] Figure 11 A schematic diagram of an electronic device according to this exemplary embodiment is shown; Detailed Implementation

[0021] Exemplary embodiments of this disclosure will be described more fully below with reference to the accompanying drawings.

[0022] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0023] In related technologies, a fixed 3DLUT (3D Look-Up Table) is typically used to perform color transformation on images. However, due to differences in hardware between different shooting devices, differences in color-related algorithms (such as automatic white balance processing and color correction processing) in ISPs (Image Signal Processors), or differences in shooting environments, a fixed 3DLUT cannot be applied to all shooting devices, ISPs, or shooting environments, and therefore cannot effectively solve the problem of image color cast.

[0024] In view of one or more of the above-mentioned problems, exemplary embodiments of this disclosure provide an image processing method. The following is in conjunction with... Figure 1 The system architecture of the operating environment for this exemplary embodiment will be described.

[0025] refer to Figure 1As shown, the system architecture 100 may include a terminal 110 and a server 120. The terminal 110 can provide images to be processed. For example, the terminal 110 may be an electronic device with shooting capabilities, such as a mobile phone, tablet, camera, wearable device, or drone; or it may be an electronic device with image storage or display capabilities, such as a personal computer or television. The terminal 110 can use currently captured images, any stored images (such as images selected by the user in the terminal 110's local album), images downloaded from the internet, or images to be displayed as images to be processed. The server 120 generally refers to the backend system that provides image processing-related services in this exemplary embodiment, and may be a single server or a cluster of multiple servers. The terminal 110 and the server 120 can form a network connection through wired or wireless links for data interaction.

[0026] In one implementation, the image processing method described above can be executed by server 120. Specifically, terminal 110 first determines the image to be processed, then uploads the image to be processed to server 120. Server 120 performs color transformation processing on the image to be processed by executing the image processing method, and can return the processed image to terminal 110.

[0027] In one embodiment, the image processing method described above can be executed by terminal 110. Specifically, terminal 110 first determines the image to be processed, then performs color transformation processing on the image to be processed by executing the image processing method, and outputs the processed image, which can then be displayed.

[0028] As can be seen from the above, the execution subject of the image processing method in this exemplary embodiment can be the aforementioned terminal 110 or server 120.

[0029] The following is combined Figure 2 The image processing method in this exemplary embodiment will be described. (See reference...) Figure 2 As shown, the image processing method may include the following steps S210 to S250:

[0030] Step S210: Obtain the image to be processed and determine the target scene corresponding to the image to be processed in the preset scene;

[0031] Step S220: Extract the region of interest that matches the target scene from the image to be processed, and determine the color feature information of the region of interest;

[0032] Step S230: Obtain the color reference information corresponding to the target scene from the pre-configured color reference information corresponding to the preset scene;

[0033] Step S240: Determine the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest;

[0034] Step S250: Perform color transformation processing on the image to be processed using color transformation parameters.

[0035] Based on the above methods, on the one hand, a scheme for color transformation processing of images is proposed, which can improve the problem of image colors deviating from the expected colors. On the other hand, this scheme obtains dynamic color transformation parameters based on the color feature information of the region of interest in the image to be processed that matches the target scene, as well as the color reference information corresponding to the target scene. This makes the color transformation parameters adaptable to the color features and scene in the image, thus having better targeting and improving the effect of image color transformation processing. Compared with the fixed 3DLUT method used in related technologies, this scheme can solve the image color cast problem caused by differences in shooting device hardware, differences in color-related algorithms in ISP, and differences in shooting environment, thereby improving the visual experience and stability of the image.

[0036] The following is about Figure 2 Each step in the process will be explained in detail.

[0037] refer to Figure 2 In step S210, the image to be processed is acquired, and the target scene corresponding to the image to be processed is determined in the preset scene.

[0038] This disclosure does not limit the source or purpose of the image to be processed, such as an image currently captured, a stored image, an image downloaded from the Internet, an image received from another device, or an image to be displayed.

[0039] Preset scenes refer to a series of typical shooting scenes selected in advance for reference, such as blue skies, grasslands, beaches, and the sea. In one implementation, shooting scenes with high color sensitivity can be selected as preset scenes. Color sensitivity refers to the degree to which human vision is sensitive to colors in a shooting scene. For example, in a blue sky scene, people tend to pay more attention to the color of the sky and less to its outline or shape, indicating that a blue sky scene has high color sensitivity and is suitable as a preset scene.

[0040] In step S210, it can be determined which scene(s) in the preset scenes correspond to the image to be processed, and the preset scene corresponding to the image to be processed is called the target scene. The target scene can reflect the main content, theme, or one of themes of the image to be processed. There can be one or more target scenes.

[0041] This disclosure does not specify how to determine the target scenario; two implementation methods are provided below.

[0042] In one implementation, the target scene can be determined based on information other than the image to be processed. This information may include, but is not limited to: the image's shooting information, such as the shooting mode selected by the user when shooting the image. For example, if a portrait shooting mode was selected, the target scene could be a "face" or "human body"; if a sunset shooting mode was selected, the target scene could be a "sunset" or "sunlight"; if a beach shooting mode was selected, the target scene could be the "sea" or "beach"; and textual information related to the image to be processed. For example, if the image to be processed is downloaded from the internet, the image's title text, related descriptive text, and comments can be obtained simultaneously. If the image to be processed is a frame from a video, the textual information of the video can be obtained as the textual information for the image to be processed, and the target scene corresponding to the image can be parsed based on this textual information. Determining the target scene based on information other than the image to be processed typically involves less processing and is beneficial for improving efficiency.

[0043] In another implementation, a pre-trained scene recognition model can be used to identify (or classify) scenes in the image to be processed to determine the target scene. The scene recognition model can be any type of machine learning model. Taking a convolutional neural network (CNN) as an example, if the image to be processed is typically an RGB image, the number of input channels of the CNN can be set to 3. Assuming the number of preset scenes is 100, the output dimension of the CNN (e.g., the number of neurons in the last fully connected layer) can be set to 100. This CNN can accept 3 channels of input from the image to be processed, and after processing, it can output a 100-dimensional vector. The value of each dimension represents the matching probability between the image to be processed and the preset scene corresponding to that dimension. Preset scenes with higher matching probabilities are selected as the target scenes corresponding to the image to be processed. For example, k preset scenes can be selected in descending order of matching probability, or preset scenes with matching probabilities higher than a certain matching threshold can be selected as target scenes. k and the matching threshold can be set based on experience or specific needs. Determining the target scene using a scene recognition model typically has high accuracy.

[0044] After determining the target scenario, continue to refer to Figure 2 In step S220, the region of interest matching the target scene is extracted from the image to be processed, and the color feature information of the region of interest is determined.

[0045] Among them, the Region of Interest (ROI) that matches the target scene refers to the region in the image that can represent image content related to the target scene. For example, reference Figure 3As shown, if the target scene of the image to be processed includes "blue sky", then the region of interest that matches it can be the blue sky area (the area that is not pure black in the figure); or, if the target scene includes "grassland", then the region of interest that matches it can be the grass area, and so on.

[0046] In one implementation, a corresponding keyword library can be configured for each preset scene, and the keywords in the keyword library are all strongly correlated with the preset scene. In step S220, the semantics of different regions in the image to be processed can be identified. If the semantics of a certain region matches the keyword library of the target scene, that is, the semantics of the region belongs to the keywords of the target scene, then the region is a region of interest that matches the target scene, and it can be extracted to determine the color feature information.

[0047] In one implementation, the image to be processed can be segmented into different regions by performing semantic segmentation, and the semantics of each region can be determined. Then, the semantics of these regions can be searched in the keyword library of the target scene to see if they exist. The regions that are found are taken as regions of interest that match the target scene.

[0048] In one implementation, the region of interest (ROI) can be determined and extracted from the image to be processed by color detection. Specifically, for each preset scene, corresponding color conditions can be configured, such as a range of color values. The color conditions are used to filter out colors that frequently appear in the preset scene. In step S220, regions or pixels in the image to be processed that meet the color conditions corresponding to the target scene can be detected and extracted as ROIs matching the target scene.

[0049] The color feature information of the region of interest is used to reflect the color features of the image to be processed that are related to the target scene. For example, if the target scene includes "blue sky" and the region of interest is the blue sky area, then the color feature information of the region of interest can reflect the color features of the blue sky.

[0050] The color feature information of the region of interest may include, but is not limited to, the following two types of information:

[0051] Information on the distribution of various colors within a color space in a region of interest. For example, the distribution region could be a spherical region, and the distribution information could be the coordinates of the center and the radius of that spherical region. Or, as... Figure 4 As shown, the white area represents the distribution of various colors in the region of interest. This area can be an ellipsoidal region, and the information about the distribution area can include the coordinates of the center of the ellipsoid and the axis information.

[0052] Statistical information on various colors within the region of interest. For example, this statistical information could include the mean, mode, and histogram data for each color.

[0053] Correspondingly, color feature information can be obtained by statistically analyzing the pixel values ​​of the region of interest, or by mapping the pixel values ​​of the region of interest to a color space, and then statistically analyzing or fitting the points in the color space.

[0054] In one implementation, the pixel values ​​within the region of interest can be used as the color feature information of the region of interest. For example, the color feature information can be a color set that includes the color value of each pixel within the region of interest.

[0055] This disclosure does not limit the color space; for example, it can be RGB, YUV, HSV, Lab, etc. In one embodiment, a color space corresponding to the color mode of the image to be processed can be used. For example, if the image to be processed is in RGB color mode, the RGB color space is used to determine the color feature information of the region of interest. In another embodiment, one or more preset color spaces can also be used. If the color mode of the image to be processed does not correspond to the color space, the color mode of the image to be processed (or the region of interest) can be converted before determining the color feature information. For example, if the image to be processed is in HSV color mode when using the RGB color space, the image to be processed can be converted to an RGB image before determining the color feature information of the region of interest.

[0056] In one implementation, the color feature information of the region of interest may include: a representative color in the region of interest, and the distance between each color in the region of interest and the representative color.

[0057] The representative color can be the dominant color of the region of interest, such as the color with the highest proportion in the region of interest, the average color of the region of interest, or the dominant or intermediate color of the hue of the color in the region of interest. The representative color can also be the color of the center point of the various colors in the region of interest in the color space. For example, if the various colors in the region of interest are distributed in a spherical shape in the color space, then the color corresponding to the center point of the sphere is the representative color.

[0058] The distance between each color in the region of interest and its representative color can be the distance between each color and its representative color in the color space. For example, let (P...) i r,P i g,P i b) represents the coordinates of any color i in the region of interest in the RGB color space, denoted by (P s r,P s g,P s b) represents the coordinates of color s in the RGB color space, and the distance between them is:

[0059]

[0060] Among them, D RGB (i,s) represents the distance between color i and the representative color s in the RGB color space.

[0061] For example, the distance between different colors in the Lab color space can also be used, as follows:

[0062]

[0063]

[0064] ΔR=P s rP i r

[0065] ΔG=P s gP i g

[0066] ΔB=P s bP i b (2)

[0067] Among them, D Lab (i,s) represents the distance between color i and the representative color s in the Lab color space.

[0068] Using the representative color in the region of interest, and the distance between each color in the region of interest and the representative color, to characterize the color feature information of the region of interest can comprehensively reflect the color distribution in the region of interest, which is beneficial for the subsequent accurate calculation of color transformation parameters. Furthermore, the process of calculating the representative color and color distance is relatively simple, which helps to improve efficiency.

[0069] Continue to refer to Figure 2 In step S230, the color reference information corresponding to the target scene is obtained from the pre-configured color reference information corresponding to the preset scene.

[0070] Each preset scene can correspond to a set of color reference information. The color reference information can include information related to the color characteristics under the corresponding preset scene, providing a reference for subsequently determining the color transformation parameters. The types of color reference information and the aforementioned color feature information can be the same or different.

[0071] In one implementation, reference Figure 5 As shown, color reference information can be configured through the following steps S510 to S530:

[0072] Step S510: Obtain the sample image.

[0073] The sample images can be image datasets used to configure color reference information, and this disclosure does not limit the source of the sample images. For example, the sample images can come from open-source image datasets, or sample images can be specifically captured or searched for different preset scenarios.

[0074] Step S520: Extract the image region that matches the preset scene from the sample image, and obtain the sample region after color adjustment of the image region.

[0075] Extracting image regions that match a preset scene from a sample image can be done by referring to the implementation method described above for extracting regions of interest that match a target scene from an image to be processed. For example, semantic segmentation can be performed on the sample image, and the semantics of each region in the sample image can be used to search in a keyword library of the preset scene to determine whether each region matches the preset scene. It should be noted that if a sample image contains multiple image regions that match different preset scenes, all of them can be extracted. For example, if a sample image includes both a blue sky and grass, the blue sky region that matches the preset scene "blue sky" can be extracted, and the grass region that matches the preset scene "grass" can also be extracted.

[0076] Color correction of image regions can be done manually or automatically using specific software. The purpose of color correction is to make the image regions present ideal colors, which can be colors preferred by the user and related to the preset scene. The color-corrected image regions are called sample regions, and each preset scene can correspond to a set of sample regions, which can include image regions from different sample images that have undergone color correction.

[0077] Step S530: Obtain color reference information corresponding to the preset scene based on the color feature information of the sample area.

[0078] Let preset scene j represent any preset scene. By extracting color feature information from the sample regions in the sample region set corresponding to preset scene j, color reference information corresponding to preset scene j can be obtained. The color reference information can be equivalent to the color feature information, or it can be information obtained by further processing the color feature information.

[0079] The color feature information of the sample region can be the same as or different from the color feature information of the region of interest. For example, the color features of the sample region may include: the distribution area information of various colors in the color space of the sample region; the statistical information of various colors in the sample region; the representative color in the sample region; and the distance between each color in the sample region and the representative color.

[0080] In one implementation, the color feature information of the sample region can be obtained by referring to the method used in step S220 to determine the color feature information of the region of interest. By summarizing the color feature information of the sample region corresponding to the preset scene j, color reference information corresponding to the preset scene j is obtained.

[0081] pass Figure 5 This method utilizes image information related to a preset scene in sample images to obtain color reference information corresponding to the preset scene, making the color reference information consistent with the actual shooting situation and covering different actual shooting environments. Furthermore, the color reference information is obtained based on the color feature information after color adjustment of the image area, which improves the quality of the color reference information.

[0082] In one implementation, the color reference information corresponding to the preset scene may include: the expected color corresponding to the preset scene. The expected color refers to the color a user expects to see in a specific preset scene. It can be one or more iconic colors in the preset scene, or a preferred color, a memory color, etc. A preferred color refers to a color that the user prefers; for example, a user can specify a preferred color for a specific preset scene, which can be user-specific. A memory color refers to the color that a user "remembers" a certain scene, reflecting the user's true and intuitive visual experience of that scene. In other words, the memory color corresponding to a certain preset scene is the color that the user visually perceives when actually seeing the preset scene. The expected color can also be a set of colors, or the color of a region in a color space, etc.

[0083] In one implementation, obtaining the color reference information corresponding to the preset scene based on the color feature information of the sample area may include the following steps:

[0084] The color at the center point of the color in the color space within the sample area is used as the expected color for the preset scene.

[0085] For example, after obtaining the sample region corresponding to the preset scene j, all colors in these sample regions are mapped to a color space. Then, the coordinates of the center point of the distribution area of ​​these colors in the color space are calculated, and the color of the center point represented by the coordinates of the center point is taken as the expected color corresponding to the preset scene j. It can be seen that this expected color is a representative color in the sample region corresponding to the preset scene j.

[0086] In one implementation, the average color value of the sample area can also be calculated as the expected color.

[0087] Using the expected color as color reference information simplifies the process of calculating color reference information and improves efficiency.

[0088] With pre-configured color reference information for various preset scenarios, it is easy to obtain color reference information for the target scenario, such as the expected color for the target scenario.

[0089] It should be noted that this disclosure does not limit the order of steps S220 and S230. For example, step S230 can be executed first, followed by step S220, or both steps can be executed simultaneously.

[0090] Continue to refer to Figure 2 In step S240, the color transformation parameters of the image to be processed are determined based on the color reference information corresponding to the target scene and the color feature information of the region of interest.

[0091] The color reference information of the target scene reflects the color characteristics expected by the user for the target scene, while the color feature information of the region of interest reflects the actual color characteristics of the image to be processed that are related to the target scene. In this exemplary embodiment, the purpose of image processing is to make the image to be processed present the color expected by the user. Therefore, the color transformation parameters of the image to be processed can be determined according to the color reference information corresponding to the target scene and the color feature information of the region of interest, so that the color transformation parameters can realize the transformation between the color reference information and the color feature information.

[0092] In one implementation, the color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene; the color feature information of the region of interest includes: the representative color in the region of interest, and the distance between each color in the region of interest and the representative color. (Reference) Figure 6 As shown, determining the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest may include the following steps S610 to S630:

[0093] Step S610: Determine the offset data between the expected color corresponding to the target scene and the representative color in the region of interest, and determine the basic offset data based on the offset data between the expected color and the representative color.

[0094] For the image to be processed, color transformation can be achieved by shifting its color values. The purpose of the shift is to make the color of the region of interest in the image to be equal to or close to the color of the sample region corresponding to the target scene. Therefore, in this exemplary embodiment, the color transformation parameters can be determined by calculating the shift data.

[0095] The offset data between the expected color and the representative color can include the amount and direction of the offset between the expected color and the representative color in the color space. For example, using (P... s r,Ps g,P s b) represents the coordinates of color s in the RGB color space, where (P d r,P d g,P d b) represents the coordinates of the expected color d in the RGB color space, and the offset data between the two can be an offset vector between them. As shown below:

[0096]

[0097] The offset data between the expected color and the representative color reflects the overall offset between the color reference information corresponding to the target scene and the color feature information of the region of interest. For example, the expected color can be the center point color of the distribution area of ​​various colors in the color space of the sample area corresponding to the target scene, and the representative color can be the center point color of the distribution area of ​​various colors in the color space of the region of interest. The offset data between the expected color and the representative color is the offset between the center points of the two distribution areas. In this exemplary embodiment, basic offset data can be determined based on the offset data between the expected color and the representative color. The basic offset data is used to calculate the offset data of each color in the region of interest.

[0098] In one implementation, the offset data between the expected color and the representative color can be used as the base offset data.

[0099] In one implementation, determining the base offset data based on the offset data between the expected color and the representative color may include the following steps:

[0100] The basic offset data is obtained by multiplying the offset data between the expected color and the representative color with the adjustment coefficient.

[0101] The adjustment coefficient represents the intensity of the color transformation applied to the image being processed, and can be a value ranging from 0.0 to 1.0. The adjustment coefficient can be set by the user or automatically determined based on the shooting mode and environment of the image. The base offset data is obtained by multiplying the offset data between the expected color and the representative color by the adjustment coefficient. A larger adjustment coefficient results in a larger base offset data value, indicating a higher degree of color shift during subsequent color transformation processing. Therefore, the intensity of image color transformation can be flexibly set using the adjustment coefficient to better meet user needs or adapt to different shooting modes and environments.

[0102] Step S620: Determine the offset data for each color based on the basic offset data and the distance between each color and the representative color in the region of interest.

[0103] Specifically, the offset data for each color can be obtained by adjusting the base offset data based on the distance between each color and the representative color within the region of interest. The relationship between the offset data for each color and its distance (i.e., the distance between each color and the representative color) and the base offset data can be pre-defined, and the offset data can be calculated using this relationship.

[0104] In one implementation, determining the offset data for each color based on the base offset data and the distance between each color and the representative color in the region of interest may include the following steps:

[0105] Based on the distance between each color in the region of interest and its representative color, determine the offset coefficient for each color. The offset coefficient is negatively correlated with the distance.

[0106] The offset data for each color is obtained by multiplying the base offset data by the offset coefficient for each color.

[0107] The offset coefficient represents the degree of offset specific to a particular color. For any color i in the region of interest, the farther away it is from the representative color, the smaller the offset coefficient; conversely, the closer it is to the representative color, the larger the offset coefficient. This disclosure does not limit the specific negative correlation between the offset coefficient and the distance; for example, the offset coefficient and the distance can be linearly negatively correlated or non-linearly negatively correlated.

[0108] Multiplying the base offset data by the offset coefficient yields the specific offset data for each color. Clearly, the larger the offset coefficient, the larger the offset data value, indicating a higher degree of offset. Therefore, adjustments in image color transformation processing can be applied more to the representative color of the region of interest or colors close to it, resulting in an adjustment effect that better matches the target scene.

[0109] Step S630: Based on the offset data of each color mentioned above, obtain the color transformation parameters.

[0110] The offset data for each color represents the color mapping relationship before and after the offset. The offset data of all colors in the region of interest can be integrated to obtain the color transformation parameters.

[0111] This disclosure does not limit the specific content or form of the color transformation parameters. In one embodiment, the color transformation parameters can be a set of offset data, such as a set of offset vectors for each color. In one embodiment, the color transformation parameters can be 3DLUT, which includes the RGB values ​​of all colors in the region of interest and the mapped RGB values ​​(i.e., the offset RGB values). The mapped RGB value for each color can be calculated by adding its offset data to the original RGB value of each color. In one embodiment, the color transformation parameters can be CCM (Color Correction Matrix), which can be obtained by solving a 3x3 matrix using the color values ​​before and after the offset.

[0112] The following combines formula (4) with Figure 7 The process of calculating the offset data and determining the color transformation parameters will be further explained. For example, for any color i in the region of interest, its offset data can be calculated using formula (4), as shown below:

[0113]

[0114] in, This represents the offset data for color i; the distribution of various colors within the color space in the region of interest is spherical, where R represents the radius of the spherical region, r. i This represents the distance between color i and the center of the sphere (i.e., the color s); This represents the offset coefficient for color i; ps represents the adjustment coefficient. This represents the base offset data.

[0115] Figure 7 A schematic diagram of offsetting using formula (4) is shown. The grid in the diagram represents the projection of the color space onto a two-dimensional plane, and the circled area is the distribution area of ​​the color in the color space of the region of interest. Comparing the color positions within the circled area before and after offsetting, it can be seen that the center point of the circled area has the highest offset, while the edge points have the lowest offset, which is 0 (i.e., no offset). The offset of other points in the circled area is between that of the center point and the edge points; the closer they are to the center point, the higher the offset.

[0116] Based on the offset data obtained in the above manner, the color values ​​before and after the offset can be determined, thereby configuring 3DLUT or calculating CCM to obtain the color transformation parameters.

[0117] In subsequent color transformation processing of the image, the representative color of the region of interest is shifted to the sample region's color to the maximum extent. Other colors close to the representative color are shifted to the sample region's color to a greater extent, while colors significantly different from the representative color are shifted to the sample region's color to a smaller extent or not at all. This ensures that, on the one hand, the image color transformation focuses more on colors highly matching the target scene (such as the representative color or other colors close to it), highlighting and improving the color transformation effect. On the other hand, the shift data for different colors are smoothly distributed, ensuring color continuity in the transformed image and avoiding abrupt color changes. Furthermore, it avoids drastic color changes to the entire image, preventing image distortion, reducing processing load, and improving efficiency.

[0118] In one implementation, the color reference information corresponding to the preset scene may include: the distribution areas of various colors in the color space of the sample area corresponding to the preset scene; the color feature information of the region of interest may include: the distribution areas of various colors in the color space of the region of interest. A mapping can be formed between the two distribution areas, such as an affine transformation relationship. Furthermore, the colors in the two distribution areas can be mapped. For example, for the distribution areas of various colors in the region of interest, each color can be mapped to the distribution areas of various colors in the sample area, and the mapping relationship is used as a color transformation parameter.

[0119] After obtaining the color transformation parameters, continue to refer to... Figure 2 In step S250, the color transformation parameters are used to perform color transformation processing on the image to be processed.

[0120] For example, if the color transformation parameter is a set of offset data, the corresponding color values ​​in the image to be processed can be offset to complete the color transformation. If the color transformation parameter is 3DLUT, the corresponding color values ​​in the image to be processed can be mapped to new color values ​​by looking up a table to complete the color transformation. If the color transformation parameter is CCM, the corresponding color values ​​in the image to be processed can be multiplied by CCM to obtain new color values ​​to complete the color transformation.

[0121] Color transformation processing can make the colors in the image to be processed (especially the colors in the region of interest) equal to or close to the colors in the sample region, thereby presenting the user's favorite colors, memorable colors, etc., and improving the visual experience of the image.

[0122] Figure 8A schematic flowchart of an image processing method is shown, which may include a process for configuring color reference information and a process for real-time image processing. The process for configuring color reference information can be a pre-process of image processing, such as an offline processing process, including:

[0123] Step S801: Obtain the sample image;

[0124] Step S802: Extract the image region that matches the preset scene from the sample image;

[0125] Step S803: Adjust the color of the image area to obtain the sample area;

[0126] Step S804: Based on the color feature information of the sample area, obtain color reference information, thereby completing the configuration of color reference information corresponding to each preset scene.

[0127] Real-time image processing refers to the process of performing color transformation on an input image in real time. This can be an online process, including:

[0128] Step S805: Obtain the image to be processed;

[0129] Step S806: Perform scene recognition on the image to be processed to determine the target scene;

[0130] Step S807: Extract the region of interest that matches the target scene from the image to be processed;

[0131] Step S808: Determine the color feature information of the region of interest;

[0132] Step S809: Based on the color reference information corresponding to each preset scene configured above, obtain the color reference information corresponding to the target scene;

[0133] Step S810: Calculate the color transformation parameters of the image to be processed based on the color feature information of the region of interest and the color reference information corresponding to the target scene, such as 3DLUT;

[0134] Step S811: Perform color transformation processing on the image to be processed using the above color transformation parameters;

[0135] Step S812: Output the processed image. It should be understood that the color-transformed image can be output or stored as the final image, or further processing can be performed, such as automatic white balance processing, image denoising processing, image enhancement processing, etc.

[0136] Figure 9 An example diagram comparing the image processing method of this exemplary embodiment with related technologies is shown. Figure 9(a) is the original image to be processed; Figure 9 (b) is an image with manually adjusted colors; Figure 9 (c) is an image processed using fixed 3DLUT in a related technique; Figure 9 (d) is the image after processing with dynamic 3DLUT, where dynamic 3DLUT is the color transformation parameters calculated according to this exemplary embodiment. It can be seen that, compared to processing with a fixed 3DLUT, the result after processing with dynamic 3DLUT in this exemplary embodiment is closer to a manually color-corrected image, that is, closer to the effect expected by the user.

[0137] Exemplary embodiments of this disclosure also provide an image processing apparatus. (Reference) Figure 10 As shown, the image processing apparatus 1000 may include:

[0138] The target scene determination module 1010 is configured to acquire the image to be processed and determine the target scene corresponding to the image to be processed in a preset scene.

[0139] The color feature information determination module 1020 is configured to extract the region of interest matching the target scene from the image to be processed and determine the color feature information of the region of interest.

[0140] The color reference information acquisition module 1030 is configured to acquire the color reference information corresponding to the target scene from the color reference information corresponding to the preset scene.

[0141] The color transformation parameter determination module 1040 is configured to determine the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest.

[0142] The color transformation processing module 1050 is configured to perform color transformation processing on the image to be processed using color transformation parameters.

[0143] In one implementation, the color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene; the color feature information of the region of interest includes: the representative color in the region of interest, and the distance between each color in the region of interest and the representative color. The above-mentioned determination of the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest includes:

[0144] Determine the offset data between the expected color corresponding to the target scene and the representative color in the region of interest, and determine the basic offset data based on the offset data between the expected color and the representative color;

[0145] Based on the base offset data and the distance between each color and its representative color in the region of interest, determine the offset data for each color.

[0146] Based on the offset data for each color, the color transformation parameters are obtained.

[0147] In one implementation, determining the offset data for each color based on the base offset data and the distance between each color and the representative color in the region of interest includes:

[0148] Based on the distance between each color in the region of interest and its representative color, determine the offset coefficient for each color. The offset coefficient is negatively correlated with the distance.

[0149] The offset data for each color is obtained by multiplying the base offset data by the offset coefficient for each color.

[0150] In one implementation, determining the base offset data based on the offset data between the expected color and the representative color includes:

[0151] The basic offset data is obtained by multiplying the offset data between the expected color and the representative color with the adjustment coefficient.

[0152] In one implementation, the representative color in the region of interest is the center point color of the various colors in the region of interest in the color space.

[0153] In one implementation, the color reference information acquisition module 1030 is further configured to:

[0154] Acquire sample images;

[0155] Extract image regions that match the preset scene from the sample images, and obtain the sample regions after color adjustment of the image regions;

[0156] Based on the color feature information of the sample area, the color reference information corresponding to the preset scene is obtained.

[0157] In one implementation, the color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene. The above-mentioned obtaining the color reference information corresponding to the preset scene based on the color feature information of the sample area includes:

[0158] The color at the center point of the color in the color space within the sample area is used as the expected color for the preset scene.

[0159] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation, and therefore will not be repeated here.

[0160] Exemplary embodiments of this disclosure also provide a computer-readable storage medium that can be implemented as a program product including program code, which, when run on an electronic device, causes the electronic device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. In an alternative embodiment, the program product can be implemented as a portable compact disc read-only memory (CD-ROM) including program code and can run on an electronic device, such as a personal computer. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0161] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0162] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0163] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0164] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] Exemplary embodiments of this disclosure also provide an electronic device, such as the terminal 110 described above. The electronic device may include a processor and a memory. The memory stores executable instructions for the processor, such as program code. The processor executes these executable instructions to perform the battery data reporting method of this exemplary embodiment.

[0166] The following is based on Figure 11 Taking the mobile terminal 1100 as an example, the construction of this electronic device will be described by way of example. Those skilled in the art will understand that, apart from components specifically designed for mobile purposes, Figure 11 The structure can also be applied to fixed types of equipment.

[0167] like Figure 11 As shown, the mobile terminal 1100 may specifically include: a processor 1101, a memory 1102, a bus 1103, a mobile communication module 1104, an antenna 1, a wireless communication module 1105, an antenna 2, a display screen 1106, a camera module 1107, an audio module 1108, a power module 1109, and a sensor module 1110.

[0168] The processor 1101 may include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit).

[0169] The processor 1101 can be connected to the memory 1102 or other components via the bus 1103.

[0170] The memory 1102 can be used to store computer executable program code, which includes instructions. The processor 1101 executes various functional applications and data processing of the mobile terminal 1100 by running the instructions stored in the memory 1102. The memory 1102 can also store application data, such as images, videos, and other files.

[0171] The communication function of mobile terminal 1100 can be implemented through mobile communication module 1104, antenna 1, wireless communication module 1105, antenna 2, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 1104 can provide 3G, 4G, 5G and other mobile communication solutions for mobile terminal 1100. Wireless communication module 1105 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication for mobile terminal 1100.

[0172] The display screen 1106 is used to implement display functions, such as displaying user interfaces, images, and videos. The camera module 1107 is used to implement shooting functions, such as capturing images to be processed. The audio module 1108 is used to implement audio functions, such as playing audio and capturing voice. The power module 1109 is used to implement power management functions, such as charging the battery, supplying power to the device, and monitoring battery status. The sensor module 1110 may include one or more sensors for implementing corresponding sensing and detection functions.

[0173] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0174] Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be embodied in entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuit,” “module,” or “system.” Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0175] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is defined only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Acquire the image to be processed, and determine the target scene corresponding to the image to be processed in a preset scene; Extract the region of interest (ROI) that matches the target scene from the image to be processed, and determine the color feature information of the ROI; Obtain the color reference information corresponding to the target scene from the pre-configured color reference information corresponding to the preset scene; The color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene; Based on the color reference information corresponding to the target scene and the color feature information of the region of interest, the color transformation parameters of the image to be processed are determined; The color transformation parameters are used to perform color transformation processing on the image to be processed.

2. The method according to claim 1, characterized in that, The color feature information of the region of interest includes: the representative color in the region of interest, and the distance between each color in the region of interest and the representative color; The step of determining the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest includes: Determine the offset data between the expected color corresponding to the target scene and the representative color in the region of interest, and determine the basic offset data based on the offset data between the expected color and the representative color; Based on the basic offset data and the distance between each color in the region of interest and the representative color, the offset data of each color is determined; The color transformation parameters are obtained based on the offset data of each color.

3. The method according to claim 2, characterized in that, The step of determining the offset data for each color based on the basic offset data and the distance between each color in the region of interest and the representative color includes: Based on the distance between each color in the region of interest and the representative color, an offset coefficient is determined for each color, and the offset coefficient is negatively correlated with the distance. The offset data for each color is obtained by multiplying the base offset data by the offset coefficient of each color.

4. The method according to claim 2, characterized in that, The step of determining the basic offset data based on the offset data between the expected color and the representative color includes: The basic offset data is obtained by multiplying the offset data between the expected color and the representative color with the adjustment coefficient.

5. The method according to claim 3, characterized in that, For any color i in the region of interest, its offset data is calculated using the following formula: ； in, This represents the offset data for color i; the distribution area of ​​various colors in the region of interest within the color space is spherical, with the center of the sphere representing the representative color, R representing the radius of the spherical region, and r... i This represents the distance between color i and the representative color; This represents the offset coefficient for color i; ps represents the adjustment coefficient. This represents the base offset data.

6. The method according to claim 1, characterized in that, The method further includes: Acquire sample images; Extract the image region that matches the preset scene from the sample image, and obtain the sample region after color adjustment of the image region; Based on the color feature information of the sample area, color reference information corresponding to the preset scene is obtained.

7. The method according to claim 6, characterized in that, The color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene; obtaining the color reference information corresponding to the preset scene based on the color feature information of the sample area includes: The color at the center point of the color in the color space within the sample area is taken as the expected color corresponding to the preset scene.

8. An image processing apparatus, characterized in that, include: The target scene determination module is configured to acquire the image to be processed and determine the target scene corresponding to the image to be processed in a preset scene; The color feature information determination module is configured to extract the region of interest that matches the target scene from the image to be processed, and determine the color feature information of the region of interest; The color reference information acquisition module is configured to acquire the color reference information corresponding to the target scene from the pre-configured color reference information corresponding to the preset scene; The color reference information corresponding to the preset scene includes: the expected color corresponding to the preset scene; The color transformation parameter determination module is configured to determine the color transformation parameters of the image to be processed based on the color reference information corresponding to the target scene and the color feature information of the region of interest. The color transformation processing module is configured to perform color transformation processing on the image to be processed using the color transformation parameters.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 7 by executing the executable instructions.

Images