Image processing method and apparatus, electronic device, and computer readable medium

Abstract

The present disclosure provides an image processing method, which relates to the technical field of image processing. The method comprises the following steps: performing scene recognition on an image, and determining a scene region according to a scene recognition result; dividing the image into a plurality of image blocks, and determining at least one adjacent image block of a pixel point; obtaining a target image parameter of the pixel point according to an image parameter of the pixel point and a mapping relationship between the adjacent image block and a target scene region corresponding to the image parameter; and adjusting the image parameter of the pixel point in the image according to the corresponding target image parameter. The present disclosure can improve the image parameter correction accuracy of a multi-scene image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically to an image processing method, an image processing apparatus, an electronic device, and a computer-readable medium. Background Technology

[0002] Images or videos captured by devices often deviate from real objects or scenes to some extent. Image parameters can be corrected to improve image fidelity. However, when an image contains different scenes, such as green plants, clouds, people, and buildings, image parameter correction cannot simultaneously account for all scenes, affecting the accuracy of parameter correction. Summary of the Invention

[0003] The purpose of this disclosure is to provide an image processing method, image processing apparatus, electronic device, and computer-readable medium, thereby improving the accuracy of image parameter correction to at least a certain extent.

[0004] According to a first aspect of this disclosure, an image processing method is provided, comprising: performing scene recognition on an image and determining a scene region based on the scene recognition result; dividing the image into multiple image blocks and determining at least one adjacent image block of a pixel; obtaining target image parameters of the pixel based on a mapping relationship between the image parameters of the pixel and the image parameters corresponding to the target scene region to which the adjacent image block belongs; and adjusting the image parameters of the pixels in the image according to the corresponding target image parameters.

[0005] According to a second aspect of this disclosure, an image processing apparatus is provided, comprising: a scene recognition module for performing scene recognition on an image and determining a scene region based on the scene recognition result; an image segmentation module for segmenting the image into multiple image blocks and determining at least one adjacent image block of a pixel; an information processing module for obtaining target image parameters of the pixel based on the image parameters of the pixel and using an image parameter mapping relationship corresponding to the target scene region to which the adjacent image blocks belong; and an image processing module for adjusting the image parameters of the pixels in the image according to the corresponding target image parameters.

[0006] According to a third aspect of this disclosure, an electronic device is provided, characterized in that it includes: a processor; and a memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method described above.

[0007] According to a fourth aspect of this disclosure, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0008] In some embodiments of the present disclosure, the technical solutions provided include, on the one hand, determining scene regions based on scene recognition results of the image, with different scene regions having corresponding image parameter mapping relationships, dividing the image into multiple image blocks, and each image block having a corresponding image parameter mapping relationship based on its scene region. Pixels belonging to the same image block may belong to different scene regions, i.e., have different image parameter mapping relationships. The target image parameters of the pixel are obtained through the image parameters of the pixel and the image mapping parameter relationship corresponding to the target scene region to which at least one adjacent image block of the pixel belongs. The image parameters of the pixels between image blocks can transition smoothly, avoiding abrupt changes in image parameters of image regions, and the image parameters of the pixels between scenes also transition smoothly. On the other hand, by using image parameter mapping relationships of different scene regions, parameter correction for all scenes in the image is taken into account, avoiding parameter compromise between scenes caused by using a unified correction method. The parameter adjustment methods are richer, improving the accuracy of parameter correction for multi-scene images. Furthermore, with the rapid expansion of device application scenarios, improving the accuracy of parameter correction for multi-scene captured images is of practical significance for improving user experience and further increasing device application scenarios.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0011] Figure 1 A schematic diagram illustrating an application scenario where an image processing method and apparatus according to embodiments of the present disclosure can be applied;

[0012] Figure 2 A flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure is shown schematically.

[0013] Figure 3 This schematically illustrates a flowchart of an embodiment of the present disclosure for acquiring a scene recognition model;

[0014] Figure 4 This illustration schematically shows a scene area division diagram in an exemplary embodiment of the present disclosure;

[0015] Figure 5 This schematic diagram illustrates an adjacent image block of a pixel in an exemplary embodiment of the present disclosure;

[0016] Figure 6 The flowchart illustrates an exemplary embodiment of the present disclosure of a process for segmenting an image into multiple image blocks and determining at least one adjacent image block of a pixel.

[0017] Figure 7 This schematically illustrates a flowchart of an exemplary embodiment of the present disclosure for determining at least one adjacent image block of a pixel based on the position of the pixel in its respective image block and the positional relationship between multiple image blocks;

[0018] Figure 8 This schematic diagram illustrates a method for dividing an image region according to an exemplary embodiment of the present disclosure.

[0019] Figure 9 A flowchart illustrating an implementation of interpolation to obtain target image parameters of pixels in an exemplary embodiment of this disclosure is shown.

[0020] Figure 10 This schematic diagram illustrates an interpolation process in an exemplary embodiment of the present disclosure;

[0021] Figure 11 A flowchart illustrating another implementation of interpolation to obtain target image parameters of pixels in an exemplary embodiment of this disclosure;

[0022] Figure 12 This schematic diagram illustrates an interpolation process in an exemplary embodiment of the present disclosure;

[0023] Figure 13 The illustration shows a schematic diagram of an image processing method in an application scenario according to an exemplary embodiment of the present disclosure;

[0024] Figure 14 This schematic diagram illustrates the composition of an image processing apparatus in an exemplary embodiment of the present disclosure.

[0025] Figure 15 A schematic diagram of an electronic device to which embodiments of the present disclosure may be applied is shown. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] Figure 1 The diagram illustrates an application scenario where an image processing method and apparatus according to embodiments of the present disclosure can be applied.

[0029] like Figure 1 As shown, device 101 can be a smart device with image processing capabilities, such as a smartphone, computer, tablet, smartwatch, smart speaker, in-vehicle device, wearable device, monitoring device, etc. Device 101 may or may not have a camera, as long as it can process images. The image to be processed can be any type of image from various scenes, captured by device 101, or obtained from the network or other devices. The image to be processed can be captured in a single scene or in multiple scenes; there are no special limitations on this.

[0030] In this embodiment, device 101 retrieves image 104 to be processed from memory 102 and sends it to processor 103. Processor 103 performs scene recognition on the image, determines the scene region based on the scene recognition result, segments the image into multiple image blocks, and determines at least one adjacent image block for each pixel. Based on the mapping relationship between the image parameters of the pixel and the image parameters corresponding to the target scene region to which the adjacent image blocks belong, target image parameters for the pixel are obtained. The image parameters of the pixels in the image are adjusted according to the corresponding target image parameters to obtain the parameter-corrected target image 105. Before processing the image, processor 103 can first obtain a trained scene recognition model, which can be trained based on training image samples and scene category labels.

[0031] It should be noted that the image processing method provided in this embodiment can be executed by device 101. Accordingly, the image processing method can be set in device 101 through a program or other means. The image processing method provided in this embodiment can also be executed by a server. The server can be a backend system providing image processing-related services in this embodiment, and may include a single electronic device with computing capabilities, such as a portable computer, desktop computer, or smartphone, or a cluster of multiple electronic devices. In this embodiment, the image processing method being executed by a device is used as an example for explanation.

[0032] In related technologies, images captured or received by a device often fail to maintain color consistency with human vision, leading to image distortion. For example, the raw images captured by the device may lack overall vibrancy, requiring color correction. Furthermore, it is desirable to apply specific color hues and saturation adjustments based on the scene—such as greenery, grass, flowers, blue sky, white clouds, beach, buildings, or animals. For instance, when shooting a scene of greenery, it is desirable to make the green more intense, while keeping the colors of other scenes unchanged.

[0033] However, when there are multiple scenes in the field of view, for images with multiple scenes, the corresponding image regions of different scenes always need to compromise with each other to complete parameter correction. For example, using the same color map to take into account all scenes in the image reduces the overall color correction accuracy of the image.

[0034] To address one or more of the aforementioned problems, this example embodiment provides an image processing method for correcting image parameters. (Reference) Figure 2 As shown, the image processing method may include the following steps S210 to S240:

[0035] In step S210, scene recognition is performed on the image, and the scene region is determined based on the scene recognition results.

[0036] In exemplary embodiments of this disclosure, the purpose of scene recognition is to determine different types of scenes in an image, such as green plants, lawns, buildings, people, beaches, and blue skies. Embodiments of this disclosure may pre-train scene recognition models.

[0037] For example, a scene recognition model can be obtained through the following steps S310 to S320:

[0038] Step S310: Obtain the training images and the scene category labels of the training images.

[0039] Training images are used to train the scene recognition model. These images can be captured by a device, obtained from the internet, or directly input locally. Scene category labels are the scene categories to which the training images belong. They are used as a reference for calculating the loss value during model training. For example, scene A might be labeled as "buildings," scene B as "people," and so on.

[0040] Step S320: Input the training image into the scene recognition model to obtain the scene classification result, and combine the scene classification result and the scene category label to obtain the predicted loss value. Then, adjust the parameters of the scene recognition model according to the predicted loss value.

[0041] The process involves inputting training images into a scene recognition model, which then extracts scene features from these images and obtains classification results based on these features, such as people or buildings. The parameters of the scene recognition model are then adjusted by combining the scene classification results and scene category labels. The predicted loss value can be obtained using a loss function; for example, the cross-entropy loss function can be used to calculate the predicted loss value.

[0042] This disclosure embodiment trains a scene recognition model, uses the trained model to recognize scenes in an image, determines scene regions based on the scene recognition results, and divides the image into different scene regions. For example... Figure 4 This schematic diagram illustrates a scene region division in an exemplary embodiment of the present disclosure, such as... Figure 4 As shown, after scene recognition, the image is divided into multiple scene areas, including people, buildings, trees, and clouds.

[0043] It should be noted that the above is merely an exemplary description of the training process of the scene recognition model in this disclosure embodiment. The scene recognition model can also be obtained in other ways, and this disclosure embodiment does not make any special limitations on this.

[0044] In step S220, the image is divided into multiple image blocks, and at least one adjacent image block of a pixel is determined.

[0045] In an exemplary embodiment of this disclosure, an image is segmented into multiple image blocks, with no overlapping areas between the image blocks.

[0046] In some possible implementations, the number of image block segments can be determined based on device hardware information. Specifically, if the field of view of the image parameter acquisition module (e.g., color acquisition) is perfectly matched with that of the device's image sensing module, then the number of image block segments is the same as the array structure of the image parameter acquisition module. The image sensing module can be the image sensor corresponding to a main camera, wide-angle camera, telephoto camera, etc., and the array of the matched image parameter acquisition module is M×N; therefore, the image block segmentation array is M×N.

[0047] Based on this, each image block has a corresponding image parameter acquisition module, which can obtain the parameter information of each image block. The measured parameter information can be directly used in subsequent image parameter correction, resulting in high accuracy.

[0048] In some possible implementations, the number of image segments can be determined based on one or more factors, such as scene distribution, target image resolution, image prediction performance, and prediction computational cost. Scene distribution includes the distance between different scenes and the number of scenes. The number of image blocks can be determined by combining these factors; for example, the image can be segmented into 4×3 or 8×6 image blocks.

[0049] Furthermore, after segmenting the image into multiple image blocks, at least one neighboring image block of a pixel is determined. Specifically, when the number of neighboring image blocks of a pixel is one, the neighboring image blocks of the pixel include the image block to which the pixel belongs; or, when the number of neighboring image blocks of a pixel is greater than one, the neighboring image blocks of the pixel include the image block to which the pixel belongs and at least one neighboring image block of the image block to which the image block belongs. At least one neighboring image block of a pixel can be determined based on the image block to which the pixel belongs and the positional relationships between the image blocks.

[0050] like Figure 5 The illustration schematically shows a diagram of adjacent image blocks of a pixel in an exemplary embodiment of the present disclosure.

[0051] like Figure 5 As shown, the image is divided into 4×5 image blocks. Identifier 1 indicates that the adjacent image blocks of a pixel in the region are the image blocks to which the pixel belongs. That is, the adjacent image blocks of a pixel located in the top corner region of the image are the image blocks to which the pixel belongs. Identifier 2 indicates that the adjacent image blocks of a pixel in the region are the image blocks to which the pixel belongs and one horizontally adjacent image block. That is, the adjacent image blocks of a pixel located in the edge region of the image are the image blocks to which the pixel belongs and one horizontally adjacent image block. Identifier 3 indicates that the adjacent image blocks of a pixel in the region are the image blocks to which the pixel belongs and three adjacent image blocks. In other words, the adjacent image blocks of a pixel located in the inner region of the image are the image blocks to which the pixel belongs and three adjacent image blocks. That is, the adjacent image blocks of a pixel located in the inner region of the image include the four image blocks surrounding the pixel.

[0052] It is worth noting that, Figure 5 The division of the image region shown is based on the center of symmetry of the image block. In this embodiment, the division criteria of the image region can also be adjusted according to the actual correction requirements, without any special restrictions.

[0053] In step S230, the target image parameters of the pixel are obtained according to the mapping relationship between the image parameters of the pixel and the image parameters of the target scene region to which the adjacent image blocks belong.

[0054] In an exemplary embodiment of this disclosure, image parameter mapping relationships corresponding to different scene regions are pre-defined. After scene recognition and image block segmentation of the image, the image parameter mapping relationship of each image block can be determined according to the scene region to which the image block belongs.

[0055] Specifically, the target image parameters of a pixel can be obtained by interpolation based on the image parameters of the target scene region to which the adjacent image blocks belong. That is, the embodiment of this disclosure does not directly map the image parameters of a pixel based on the image parameter mapping relationship of the image block to which the pixel belongs, but rather maps and interpolates the image parameters based on the image parameter mapping relationship of the target scene region to which the adjacent image blocks belong. The image parameters between image blocks can transition smoothly, avoiding abrupt areas in the entire image.

[0056] For example, based on the image parameters of pixels, the image parameters are mapped to mapped image parameters by using the image parameter mapping relationship of the target scene region to which the adjacent image blocks belong. Then, the mapped image parameters are used for interpolation to obtain the target image parameters of pixels, so that the target image parameters of pixels between each image block and the target image parameters between scenes are also smoothly transitioned.

[0057] It is worth noting that the same image patch can include portions of different scene regions; see further. Figure 4 As shown, the same image block can contain Region 1 and Region 2, which belong to different scene areas. Region 1 can belong to the building scene area, and Region 2 can belong to the character scene area. Therefore, Region 1 and Region 2 correspond to different image parameter mapping relationships. Based on the image parameter mapping relationship between the pixel and the target scene area to which the adjacent image block belongs, the target image parameter of the pixel is obtained by interpolation. The target image parameter of the pixel located between Region 1 and Region 2 can achieve a smooth transition between scenes and avoid abrupt changes in the target image parameter between different scenes.

[0058] The image parameter mapping relationship can be a 3D Look-Up Table (3DLUT), such as a color mapping table. The input pixel has three RGB color components, and the corresponding RGB color mapping components are output through the color mapping table. Alternatively, the image parameter mapping relationship can be a non-color mapping table; for example, the input image's infrared information is output as pseudo-color information through a non-color mapping table. This disclosure does not specifically limit the specific type of image parameter mapping relationship; the appropriate image parameter mapping relationship can be selected according to the image parameter correction requirements.

[0059] In step S240, the image parameters of the pixels in the image are adjusted according to the corresponding target image parameters.

[0060] In an exemplary embodiment of this disclosure, after obtaining the target image parameters of the pixels in the image, the pixels are adjusted according to the corresponding target image parameters, that is, each pixel is output according to the target image parameters to obtain the image after correction parameters.

[0061] The image processing method in the exemplary embodiments of this disclosure determines scene regions based on scene recognition results of the image. Different scene regions have corresponding image parameter mapping relationships. The image is divided into multiple image blocks, and each image block has a corresponding image parameter mapping relationship based on its scene region. Pixels belonging to the same image block may belong to different scene regions, that is, they have different image parameter mapping relationships. The target image parameters of the pixel are obtained by interpolation through the image parameters of the pixel and the image mapping parameter relationship of the target scene region to which at least one of the pixel's adjacent image blocks belongs. Not only can the image parameters of pixels between image blocks be smoothly transitioned, avoiding abrupt changes in image parameters of image regions, but the image parameters of pixels between scenes are also smoothly transitioned. By using image parameter mapping relationships of different scene regions, the parameter correction of all scenes in the image is taken into account, avoiding parameter compromises between scenes caused by using a uniform correction method. The parameter adjustment methods are richer, improving the accuracy of parameter correction for multi-scene images.

[0062] According to an exemplary embodiment of this disclosure, an implementation method for determining at least one adjacent image block of a pixel is provided. Dividing an image into multiple image blocks and determining at least one adjacent image block of a pixel may include steps S610 and S620:

[0063] Step S610: Divide the image into multiple image blocks.

[0064] There is no overlap between the multiple image patches. The image can be divided into multiple image patches according to the number of targets. The number of targets is determined at least based on the scene region distribution and the target image resolution. For example, image patch division can group parts belonging to the same scene region into the same image patch as much as possible.

[0065] In practice, in addition to scene area distribution and target image resolution, the number of targets can also be determined by combining image prediction results and prediction computation.

[0066] For example, decision factors for determining the target number of image patch divisions can include the direction of the light source. For instance, if the light source diffuses simultaneously in all directions, dividing the image patch into squares is better than using matrices. The shape of the image patch division can be determined based on the radiation direction of the light source.

[0067] For example, the decision factor for determining the target number of image blocks can include the target image resolution. For instance, if the resolution of the main camera, wide-angle lens, and telephoto lens is 4096×3072, the corresponding image aspect ratio is 4:3. The image can be divided into squares evenly, so the image blocks can be divided into squares, such as 4×3, 8×6, and 12×9.

[0068] For example, decision factors for determining the number of targets in an image patch can include algorithm performance and algorithm effectiveness. A larger number of targets in an image patch increases computational complexity, so the number of targets can be appropriately reduced. However, larger image patches contain richer image content; therefore, to increase the richness of the image patch content, the number of targets should be appropriately increased. Examples include image patches of 4×3 or 8×6.

[0069] It should be noted that the decision factors used in the embodiments of this disclosure are not absolute, but rather determined by combining one or more of the decision factors mentioned above based on actual correction needs, with the goal of improving prediction accuracy while ensuring stable computational performance.

[0070] Step S620: Determine at least one adjacent image block of the pixel based on the position of the pixel in its respective image block and the positional relationship between multiple image blocks.

[0071] Since the number of adjacent image blocks varies for each pixel, in order to achieve a smooth transition between image blocks, the actual number and position of adjacent image blocks for each pixel are first determined. Then, based on the image parameter mapping relationship corresponding to the target scene region to which the pixel belongs, the target image parameters of the pixel are obtained by interpolation.

[0072] Wherein, the actual adjacent image blocks of a pixel include at least the image block to which the pixel belongs, and may also include at least one adjacent image block of the image block to which the pixel belongs. Step S620 may include the following steps:

[0073] Step S710: Based on the position of the pixel in its respective image block and the positional relationship between the multiple image blocks, the image is divided into multiple image regions, wherein the pixels in different image regions correspond to different numbers of adjacent image blocks.

[0074] Step S720: For each pixel in each image region, determine at least one adjacent image block of the pixel.

[0075] The image can be divided into multiple image regions based on the position of a pixel in its respective image block and the positional relationship between multiple image blocks. Pixels in different image regions correspond to different numbers of adjacent image blocks.

[0076] Figure 8 The illustration shows a schematic diagram of dividing an image region in an exemplary embodiment of the present disclosure. After dividing the image into 4×5 image blocks, the image is divided into three regions: the top corner region 1, the edge region 2, and the interior region 3. Each pixel in each region corresponds to an image block with a different number of adjacent image blocks.

[0077] The number of adjacent image blocks for a pixel varies within different image regions. For example, a pixel in the top corner region 1 has 1 adjacent image block (image block A), a pixel in the edge region 2 has 2 adjacent image blocks (image blocks A and B), and a pixel in the inner region 3 has 4 adjacent image blocks (such as image blocks A, B, C, and D). In other words, an adjacent image block for a pixel refers to an image block that completely surrounds the pixel.

[0078] In an exemplary embodiment of this disclosure, an implementation method for interpolating to obtain target image parameters of pixels is also provided. Based on the mapping relationship between the image parameters of a pixel and the image parameters corresponding to the target scene region to which adjacent image blocks belong, interpolating to obtain the target image parameters of a pixel may include the following steps S910 to S920:

[0079] Step S910: Based on the image parameter mapping relationship, perform parameter mapping on the image parameters to obtain the mapped image parameters of the pixels.

[0080] For a given pixel, at least one image parameter mapping relationship can be obtained based on the adjacent image blocks of that pixel. The image parameters of that pixel can be mapped using the image parameter mapping relationship to obtain at least one mapped image parameter.

[0081] See also Figure 8As shown, for a pixel in image region 1, the adjacent image block is the image block to which the pixel belongs (image block A). The image parameters of the pixel are mapped according to the image parameter mapping relationship corresponding to the image block, resulting in one mapped image parameter. For a pixel in image region 2, the adjacent image blocks are the image blocks to which the pixel belongs (image block A and image block B). The image parameters of the pixel are mapped according to the image parameter mapping relationship corresponding to the image block, resulting in two mapped image parameters. Correspondingly, the pixels in image region 3 obtain four mapped image parameters.

[0082] Step S920: Based on the mapped image parameters, interpolate to obtain the target image parameters of the pixels.

[0083] Different image regions correspond to different interpolation methods. The target image parameters can be obtained by interpolating the mapped image parameters according to the target interpolation method corresponding to the image region to which a pixel belongs.

[0084] See also Figure 8 As shown, the pixels in the top corner region 1 of the image are obtained by nearest neighbor interpolation. The target image parameters of each pixel are obtained by interpolation in the edge region 2 of the image. The target image parameters of each pixel are obtained by interpolation in the edge region 2 of the image. The target image parameters of each pixel are obtained by interpolation in the inner region 3 of the image.

[0085] By applying corresponding target interpolation processing methods to each image region, the mapping image parameters of pixels corresponding to different adjacent image blocks are interpolated to obtain the target image parameters of pixels. This avoids abrupt changes in the target image parameters of pixels between image blocks, instead ensuring a smooth transition. This also avoids abrupt regions in the correction of image parameters of pixels throughout the entire image. Furthermore, since the mapping image parameters are determined based on the scene region to which each image block belongs, the target image parameters between pixels smoothly transition within the scene, improving the global correction accuracy of image processing.

[0086] Figure 10 The schematic diagram illustrates an interpolation process in an exemplary embodiment of the present disclosure, which will be described below. Figure 8 Taking the pixels in region 3 of the image as an example, combined with Figure 10 The parameters of the target image obtained by interpolation are explained.

[0087] First, determine the neighboring image blocks of pixel Q, including image block D to which pixel Q belongs, image block B, neighboring image block C, and neighboring image block A of image block D;

[0088] Next, obtain the image parameter mapping relationship of the image block D, the adjacent image block B, the adjacent image block C and the adjacent image block A (as shown in the three-dimensional mapping table), and use each image parameter mapping relationship to map the image parameters of pixel Q to obtain mapped image parameter 4, mapped image parameter 5, mapped image parameter 6 and mapped image parameter 7 respectively.

[0089] Finally, the obtained mapped image parameters 4, 5, 6 and 7 are interpolated using bilinear interpolation to obtain the target image parameters of the pixels.

[0090] By using the image parameter mapping relationship of at least one adjacent image to the scene region to which the image parameters of each pixel belong, the mapped image parameters are obtained respectively. Then, the obtained mapped image parameters are interpolated according to the target interpolation method. The whole process only requires corresponding table lookup and interpolation after obtaining the adjacent image blocks of the pixel to obtain the target image parameters corresponding to each pixel. This not only ensures a smooth transition of target image parameters between image blocks and between scene regions, but also simplifies the logical processing, increases computational performance, and reduces power consumption of the device.

[0091] According to an exemplary example of this disclosure, another method for interpolating to obtain the target image parameters of a pixel is also provided. Based on the mapping relationship between the image parameters of the pixel and the image parameters corresponding to the target scene region to which adjacent image blocks belong, interpolating to obtain the target image parameters of the pixel may include the following steps S1110 to S1120:

[0092] Step S1110: According to the target interpolation method corresponding to the image region to which the pixel belongs, perform interpolation processing on the image parameter mapping relationship to obtain the target image parameter mapping relationship corresponding to the pixel.

[0093] Step S1120: Use the target image parameter mapping relationship to perform parameter mapping on the image parameters to obtain the target image parameters.

[0094] Different image regions correspond to different target interpolation methods. In this embodiment, the image parameter mapping relationship can also be interpolated to obtain the target image parameter mapping relationship corresponding to the pixel point. Then, the target image parameter mapping relationship can be used to perform parameter mapping on the image parameters to obtain the target image parameters.

[0095] Figure 12 The schematic diagram illustrates an interpolation process in an exemplary embodiment of the present disclosure, which will be described below. Figure 8 Taking the pixels in region 3 of the image as an example, combined with Figure 12 The parameters of the target image obtained by interpolation are explained.

[0096] First, determine the neighboring image blocks of pixel Q, including image block D to which pixel Q belongs, image block B, neighboring image block C, and neighboring image block A of image block D;

[0097] Secondly, the image parameter mapping relationship of the belonging image block D, the adjacent image block B, the adjacent image block C and the adjacent image block A is obtained, and the image parameter mapping relationship is interpolated according to the bilinear interpolation method to obtain the target image parameter mapping relationship;

[0098] Finally, based on the target image parameter mapping relationship, the image parameters of pixel Q are mapped to obtain the target image parameters of the pixel.

[0099] By mapping the image parameters of at least one adjacent image block of a pixel, interpolation is performed according to the target interpolation method corresponding to the image region to which the pixel belongs. This eliminates the need for separate parameter mapping using each image parameter mapping, reducing the number of mapping operations. Although interpolating the image parameter mapping increases the computational load to some extent, the target image parameter mapping obtained by interpolating the image parameter mappings corresponding to each scene region integrates the positional relationships between image blocks and the relationships between scene regions, which helps improve the prediction accuracy of the target image parameter mapping.

[0100] It should be noted that the image parameter mapping relationship in the embodiments of this disclosure can be a color mapping relationship or a non-color mapping relationship. For example, pseudo-color can be obtained by using the infrared information of the image through a non-color mapping relationship, and so on. The embodiments of this disclosure can select the type of image parameter mapping relationship according to the specific application scenario, which has universality for image parameter correction, especially for scenarios where the image contains multiple scene regions, avoiding the sacrifice of the correction effect of each scene, completing the parameter correction of the entire image, and ensuring the global effect.

[0101] If the image parameters of the pixels are in RGB format, the pre-set RGB format image parameter mapping relationship can be directly called. If the image parameter format is in YUV format, all image parameter mapping relationships can be converted to YUV format, so that the image parameter mapping relationship has the same format as the image parameters. This disclosure does not impose any special limitations on this.

[0102] In addition, the embodiments of this disclosure can also directly identify the scene region by performing scene recognition on the image without dividing the image into image blocks, and obtain the target image parameters of each pixel in each scene region by looking up a table according to the image parameter mapping relationship corresponding to the scene, which can also realize parameter correction of multi-scene images.

[0103] Figure 13This illustration schematically depicts an image processing method in an application scenario according to an exemplary embodiment of this disclosure. The following example uses image segmentation into image blocks as a case study. Figure 13 The image processing method according to the embodiments of this disclosure will be described in detail.

[0104] First, the image is subjected to scene recognition to determine the scene area, which includes people, buildings, trees, and clouds.

[0105] Secondly, the image is divided into multiple image blocks, with no overlapping areas between the image blocks. Based on the position of the pixel in its respective image block and the positional relationship between the multiple image blocks, the image is divided into multiple image regions, including image region 1301 (image apex region), image region 1302 (image edge region), and image region 1303 (image interior region), so that for a pixel in the image region, at least one adjacent image block of the pixel is determined.

[0106] Furthermore, for image regions 1301, 1302, and 1303 respectively, based on the image parameters of the pixels, color mapping is performed using the image parameter mapping relationship of the target scene region to which the adjacent image blocks of the pixels belong, to obtain the mapped color parameters of the pixels.

[0107] Next, for image region 1301, the target color parameters of the pixels are obtained by nearest-neighbor interpolation. For image region 1302, the target color parameters of the pixels are obtained by linear interpolation. For image region 1303, the target color parameters of the pixels are obtained by bilinear interpolation.

[0108] Finally, the color parameters of the pixels in the image are adjusted according to the corresponding target color parameters to obtain the image with adjusted colors.

[0109] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0110] Further reference Figure 14 As shown, an exemplary embodiment of this disclosure provides an image processing apparatus 1400, including a scene recognition module 1410, an image segmentation module 1420, an information processing module 1430, and an image processing module 1240. Wherein:

[0111] Scene recognition module 1410 is used to perform scene recognition on an image and determine the scene region based on the scene recognition result;

[0112] The image segmentation module 1420 is used to segment the image into multiple image blocks and determine at least one adjacent image block of a pixel.

[0113] Information processing module 1430 is used to obtain the target image parameters of the pixel based on the image parameters of the pixel and by using the image parameter mapping relationship corresponding to the target scene region to which the adjacent image blocks belong.

[0114] The image processing module 1440 is used to adjust the image parameters of the pixels in the image according to the corresponding target image parameters.

[0115] In one exemplary embodiment, the image segmentation module 1420 may include:

[0116] An image segmentation unit is used to segment the image into multiple image blocks, wherein there is no overlap between the multiple image blocks;

[0117] The positional relationship determination unit is used to determine at least one adjacent image block of the pixel based on the position of the pixel in its respective image block and the positional relationship between the plurality of image blocks.

[0118] In an exemplary embodiment, the positional relationship determination unit is configured to:

[0119] Based on the position of the pixel in its respective image block and the positional relationship between the multiple image blocks, the image is divided into multiple image regions, wherein the pixels in different image regions correspond to different numbers of adjacent image blocks;

[0120] For each pixel in the image region, at least one adjacent image block of the pixel is determined.

[0121] In one exemplary embodiment, the adjacent image blocks of the pixel include:

[0122] The image block to which the pixel belongs; or...

[0123] The image block to which the pixel belongs and the adjacent image blocks of the image block to which the pixel belongs.

[0124] In one exemplary embodiment, the information processing module 1430 may include:

[0125] The first mapping unit is used to perform parameter mapping on the image parameters based on the image parameter mapping relationship to obtain the mapped image parameters of the pixel.

[0126] The first interpolation processing unit is used to interpolate and obtain the target image parameters of the pixel based on the mapped image parameters.

[0127] In one exemplary embodiment, the first interpolation processing unit is configured to:

[0128] The target image parameters are obtained by interpolating the mapped image parameters according to the target interpolation method corresponding to the image region to which the pixel belongs.

[0129] In one exemplary embodiment, the information processing module 1430 may further include

[0130] The second interpolation processing unit is used to perform interpolation processing on the image parameter mapping relationship according to the target interpolation method corresponding to the image region to which the pixel belongs, so as to obtain the target image parameter mapping relationship corresponding to the pixel.

[0131] The second mapping unit is used to perform parameter mapping on the image parameters using the target image parameter mapping relationship to obtain the target image parameters.

[0132] In one exemplary embodiment, the image segmentation module 1420 is configured to:

[0133] The image is divided into multiple image blocks according to the target number, wherein the target number is determined at least based on the scene area distribution and the target image resolution.

[0134] In an exemplary embodiment, the image parameter mapping relationship is a color parameter mapping relationship, and the information processing module 1430 is configured to: use the color information mapping relationship corresponding to the target scene region to which the adjacent image blocks belong to interpolate to obtain the target color parameters of the pixel;

[0135] Image processing module 1440 is configured as follows:

[0136] The color parameters of the pixels in the image are adjusted according to the corresponding target color parameters to obtain the color-adjusted image.

[0137] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation. For any undisclosed details, please refer to the implementation content of the method section, and therefore will not be repeated here.

[0138] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0139] An exemplary embodiment of this disclosure also provides an electronic device for implementing an image processing method, which may be... Figure 1 The device 101 in the diagram can also be a server. Generally, the electronic device includes at least a processor and a memory, the memory being used to store executable instructions for the processor, and the processor being configured to perform an image processing method by executing the executable instructions.

[0140] The following is based on Figure 15 Taking the mobile terminal 1500 as an example, the construction of the electronic device in this embodiment of the present disclosure will be described by way of example. Those skilled in the art will understand that, apart from components specifically designed for mobile purposes, Figure 15 The structure shown can also be applied to fixed-type devices. In other embodiments, the mobile terminal 1500 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. The interface connections between the components are only schematic and do not constitute a limitation on the structure of the mobile terminal 1500. In other embodiments, the mobile terminal 1500 may also adopt a similar design to... Figure 15 Different interface connection methods, or combinations of multiple interface connection methods.

[0141] like Figure 15 As shown, the mobile terminal 1500 may specifically include: a processor 1501, a memory 1502, a bus 1503, a mobile communication module 1504, an antenna 1, a wireless communication module 1505, an antenna 2, a display screen 1506, a camera module 1507, an audio module 1508, a power module 1509, and a sensor module 1510.

[0142] Processor 1501 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, an encoder, a decoder, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). The image processing method in this exemplary embodiment can be executed by an AP, an ISP, or a DSP. When the method involves image parameter correction processing, it can be executed by an ISP. For example, the ISP can perform scene recognition on the image, determine the scene region based on the scene recognition result, divide the image into multiple image blocks, and determine at least one adjacent image block for each pixel; obtain the target image parameters of the pixel based on the mapping relationship between the image parameters of the pixel and the image parameters corresponding to the target scene region to which the adjacent image blocks belong; and adjust the image parameters of the pixels in the image according to the corresponding target image parameters.

[0143] An encoder encodes (compresses) images or videos to reduce data size for easier storage or transmission. A decoder decodes (decompresses) the encoded data to restore the original image or video data. The mobile terminal 1500 can support one or more encoders and decoders, such as image formats like JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), and BMP (Bitmap), and video formats like MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0144] The processor 1501 can be connected to the memory 1502 or other components via the bus 1503.

[0145] The memory 1502 can be used to store executable program code, including instructions. The processor 1501 executes various functional applications and data processing of the mobile terminal 1500 by running the instructions stored in the memory 1502. The memory 1502 can also store application data, such as images, videos, and other files.

[0146] The communication functions of mobile terminal 1500 can be implemented through mobile communication module 1504, antenna 1, wireless communication module 1505, antenna 2, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 1504 can provide 3G, 4G, and 5G mobile communication solutions for mobile terminal 1500. Wireless communication module 1505 can provide wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication for mobile terminal 1500.

[0147] The display screen 1506 is used to implement display functions, such as displaying the user interface, images, and videos. The camera module 1507 is used to implement shooting functions, such as capturing images and videos. The audio module 1508 is used to implement audio functions, such as playing audio and capturing voice. The power module 1509 is used to implement power management functions, such as charging the battery, supplying power to the device, and monitoring battery status. The sensor module 1510 may include one or more sensors to implement corresponding sensing and detection functions. For example, the sensor module 1510 may include an inertial sensor, which is used to detect the motion posture of the mobile terminal 1500 and output inertial sensing data.

[0148] Furthermore, exemplary embodiments of this disclosure also provide a computer-readable storage medium storing a program product capable of implementing the methods described above. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0149] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0150] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0151] Furthermore, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0152] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. An image processing method, characterized in that, include: Scene recognition is performed on the image, and scene regions are determined based on the scene recognition results. Scene recognition is used to determine the different types of scenes in the image, and different scene regions have their own corresponding image parameter mapping relationships. The image is divided into multiple image blocks, and at least one neighboring image block of a pixel is determined, wherein the neighboring image block of a pixel includes the image block to which the pixel belongs; or, the neighboring image block of a pixel includes the image block to which the pixel belongs and at least one neighboring image block of the image block to which the pixel belongs. The target image parameters of the pixel are obtained based on the mapping relationship between the image parameters of the pixel and the image parameters of the target scene region to which the adjacent image blocks belong. The image parameters of the pixels in the image are adjusted according to the corresponding target image parameters.

2. The method according to claim 1, characterized in that, The step of segmenting the image into multiple image blocks and determining at least one adjacent image block for a pixel includes: The image is divided into multiple image blocks, and there is no overlap between the multiple image blocks; Based on the position of the pixel in its respective image block and the positional relationship between the plurality of image blocks, at least one adjacent image block of the pixel is determined.

3. The method according to claim 2, characterized in that, The step of determining at least one adjacent image block of the pixel based on the position of the pixel in its respective image block and the positional relationship between the plurality of image blocks includes: Based on the position of the pixel in its respective image block and the positional relationship between the multiple image blocks, the image is divided into multiple image regions, wherein the pixels in different image regions correspond to different numbers of adjacent image blocks; For each pixel in the image region, at least one adjacent image block of the pixel is determined.

4. The method according to claim 3, characterized in that, The step of obtaining the target image parameters of the pixel based on the mapping relationship between the image parameters of the pixel and the image parameters corresponding to the target scene region to which the adjacent image blocks belong includes: Based on the image parameter mapping relationship, the image parameters are mapped to obtain the mapped image parameters of the pixel. The target image parameters of the pixels are obtained by interpolation based on the mapped image parameters.

5. The method according to claim 4, characterized in that, The step of interpolating to obtain the target image parameters of the pixels based on the mapped image parameters includes: The target image parameters are obtained by interpolating the mapped image parameters according to the target interpolation method corresponding to the image region to which the pixel belongs.

6. The method according to claim 3, characterized in that, The step of obtaining the target image parameters of the pixel based on the image parameters of the pixel and utilizing the image parameter mapping relationship corresponding to the target scene region to which the adjacent image blocks belong includes: According to the target interpolation method corresponding to the image region to which the pixel belongs, the image parameter mapping relationship is interpolated to obtain the target image parameter mapping relationship corresponding to the pixel; The target image parameters are obtained by mapping the image parameters using the target image parameter mapping relationship.

7. The method according to claim 2, characterized in that, The step of segmenting the image into multiple image blocks includes: The image is divided into multiple image blocks according to the target number, wherein the target number is determined at least based on the scene area distribution and the target image resolution.

8. The method according to any one of claims 1 to 7, characterized in that, The image parameter mapping relationship is a color parameter mapping relationship. The target color parameter of the pixel is obtained by using the color information mapping relationship corresponding to the target scene area to which the adjacent image blocks belong. The step of adjusting the image parameters of the pixels in the image according to the corresponding target image parameters includes: The color parameters of the pixels in the image are adjusted according to the corresponding target color parameters to obtain the color-adjusted image.

9. An image processing apparatus, characterized in that, include: The scene recognition module is used to perform scene recognition on the image and determine the scene region based on the scene recognition result. The scene recognition is used to determine the different types of scenes in the image, and different scene regions have their own corresponding image parameter mapping relationships. An image segmentation module is used to segment the image into multiple image blocks and determine at least one neighboring image block of a pixel, wherein the neighboring image block of a pixel includes the image block to which the pixel belongs; or, the neighboring image block of a pixel includes the image block to which the pixel belongs and at least one neighboring image block of the image block to which the image block belongs. The information processing module is used to obtain the target image parameters of the pixel by using the image parameter mapping relationship corresponding to the target scene region to which the adjacent image blocks belong, based on the image parameters of the pixel. The image processing module is used to adjust the image parameters of the pixels in the image according to the corresponding target image parameters.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images