Image processing method and device, electronic equipment, chip and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the field of image processing, and the method comprises the steps: obtaining a target residual error according to a first brightness component in a to-be-processed first image; wherein the target residual error is used for representing a processing error of super-resolution processing of the first image; based on a target gain associated with the first image, fusing a second brightness component in the second image and the target residual error to obtain a target brightness component; wherein the target gain is used for representing the enhancement strength of the target residual error; the second image is obtained by performing super-resolution processing on the first image; and updating the second image based on the target brightness component to obtain a target image. Therefore, dynamic optimization of the super-resolution effect can be realized by introducing a dynamically adjustable residual compensation mechanism, the reconstruction quality and detail fidelity of the image are improved, and the adaptability and practicability of the algorithm in diversified, complex and real scenes are enhanced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, chip, and storage medium. Background Technology

[0002] With the continuous advancement of display technology, high-resolution displays have been widely used in smartphones, televisions, tablets, and professional imaging equipment, and users have placed higher demands on the clarity and detail of image and video content. However, in practical applications, a large number of existing image and video resources are still in low-resolution formats, which are prone to blurring, jagged edges, or pixelation when displayed directly on high-resolution displays, seriously affecting the visual experience. Summary of the Invention

[0003] This application proposes an image processing method, apparatus, electronic device, chip, and storage medium to at least partially solve one of the technical problems in the related art.

[0004] One embodiment of this application proposes an image processing method, including: The target residual is obtained based on the first luminance component in the first image to be processed; wherein the target residual is used to characterize the processing error of super-resolution processing of the first image. Based on the target gain associated with the first image, the second luminance component in the second image and the target residual are fused to obtain the target luminance component; wherein, the target gain is used to characterize the enhancement intensity of the target residual; the second image is obtained by performing the super-resolution processing on the first image; The second image is updated based on the target brightness component to obtain the target image.

[0005] Another embodiment of this application provides an image processing apparatus, including: The acquisition module is used to acquire a target residual based on a first luminance component in a first image to be processed; wherein the target residual is used to characterize the processing error of super-resolution processing of the first image; A fusion module is used to fuse a second luminance component and a target residual in a second image based on a target gain associated with the first image to obtain a target luminance component; wherein the target gain is used to characterize the enhancement intensity of the target residual; and the second image is obtained by performing the super-resolution processing on the first image; An update module is used to update the second image based on the target brightness component to obtain the target image.

[0006] In another aspect of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method as described in the foregoing aspect.

[0007] Another aspect of this application provides a chip including an interface circuit and a processing circuit coupled to each other, the interface circuit being used to input or output signals, and the processing circuit being configured to perform the image processing method as described in the foregoing aspect.

[0008] In another aspect of this application, a non-transitory computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the image processing method as described in the foregoing aspect.

[0009] Another aspect of this application provides a computer program product having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described in the foregoing aspect.

[0010] The image processing method, apparatus, electronic device, chip, and storage medium proposed in this application overcome the limitations of the "fixed weights and fixed outputs" of the AISR model in related technologies by introducing a dynamically adjustable residual compensation mechanism. Specifically, firstly, a target residual is obtained based on the first luminance component in the first image to be processed. This target residual is used to characterize the processing error generated after super-resolution processing of the first image, i.e., the difference in luminance between the reconstructed high-resolution image and the ideal high-resolution image. Subsequently, based on the target gain associated with the first image, the second luminance component in the second image (the reconstructed high-resolution image) is fused with the target residual to generate a target luminance component, and the luminance channel in the second image is updated accordingly, finally obtaining the optimized target image. The target gain is used to adjust the enhancement intensity of the residual information; it is essentially a content-adaptive adjustment coefficient that can be dynamically determined according to the visual characteristics of the first image. For example, a higher gain is used in areas with rich detail or low contrast to enhance detail recovery, while a lower gain is used in smooth or high-noise areas to suppress artifacts and noise amplification. This enables dynamic optimization of super-resolution effects, which not only significantly improves the reconstruction quality and detail fidelity of images, but also enhances the adaptability and practicality of the algorithm in diverse and complex real-world scenarios.

[0011] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0012] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A schematic flowchart of an image processing method provided for an exemplary embodiment of this application; Figure 2 A schematic flowchart of another image processing method provided for an exemplary embodiment of this application; Figure 3 A schematic flowchart of yet another image processing method provided for an exemplary embodiment of this application; Figure 4 A schematic flowchart of another image processing method provided for an exemplary embodiment of this application; Figure 5 A schematic diagram of a super-resolution image processing flow provided for an exemplary embodiment of this application; Figure 6 A schematic diagram of the structure of an image processing apparatus provided for an exemplary embodiment of this application; Figure 7 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application; Figure 8 This is a schematic diagram of the structure of a chip proposed in an exemplary embodiment of this application. Detailed Implementation

[0013] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0014] To improve the rendering quality of low-resolution images on high-resolution displays, image super-resolution (SR) technology has emerged and developed rapidly. Image super-resolution aims to recover high-resolution images from single or multiple frames of low-resolution images using algorithmic methods, enhancing detail while preserving the original structure and texture. Based on different technical approaches, super-resolution methods can be mainly divided into two categories: traditional super-resolution algorithms and AI-based super-resolution (AISR) algorithms.

[0015] Traditional super-resolution algorithms are typically based on interpolation, reconstruction, or learning strategies. They refine image edges by analyzing statistical characteristics between pixels, such as edge strength and directional gradient, in order to improve the visual effect after magnification. However, their performance is limited by the expressive power of the prior model, making it difficult to recover real texture details.

[0016] With the continuous development of deep learning technology, super-resolution algorithms based on convolutional neural networks (CNNs) have become mainstream. They can automatically learn the nonlinear mapping relationship between low-resolution and high-resolution images through end-to-end training, effectively extract and reconstruct high-frequency texture features of images, and significantly outperform traditional super-resolution algorithms in terms of peak signal-to-noise ratio, structural similarity, and subjective visual quality.

[0017] However, the AISR algorithm uses fixed weights after network training. During inference after deployment, the network performance remains constant and cannot be changed, resulting in poor flexibility.

[0018] Therefore, in view of at least one of the problems existing in the above-mentioned related technologies, this application proposes an image processing method, apparatus, electronic device, chip and storage medium.

[0019] The image processing method, apparatus, electronic device, chip, and storage medium of this application are described below with reference to the accompanying drawings. Before specifically describing the embodiments of this application, commonly used technical terms are first introduced for ease of understanding: RGB color space (or color space): Based on the three primary colors of red (R), green (G), and blue (B), different degrees of superposition are used to produce a rich and wide range of colors, commonly known as the three primary color mode.

[0020] YUV color space (or color space): A color model that separates luminance information from chrominance information, commonly used in video compression and transmission. Where Y represents luminance (Luma), i.e., the brightness information of the image, and does not include color information; U and V represent blue difference (Cb) and red difference (Cr), respectively, i.e., chrominance information, describing the specific hue of the color.

[0021] Figure 1 This is a schematic flowchart of an image processing method provided for an exemplary embodiment of this application.

[0022] It should be noted that the image processing method of this application embodiment can be applied to an image processing device. In some possible embodiments, the image processing device can be configured in an electronic device or a chip so that the electronic device or chip can perform image processing functions. In addition, in some possible embodiments, the image processing device can also be software in an electronic device.

[0023] In any embodiment of this application, the chip can be integrated into an electronic device. The chip includes a Central Processing Unit (CPU), an Image Signal Processing (ISP), an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), a System-on-Chip (SOC), a Reduced Instruction Set Computer, etc., which will not be listed here.

[0024] The electronic devices mentioned include, but are not limited to, terminals and personal computers. A terminal is a user-side entity used to receive or transmit signals, such as a mobile phone. Terminals can also be called terminal equipment, user equipment (UE), mobile station (MS), mobile terminal equipment (MT), etc. Terminals can be communication-enabled vehicles, smart cars, mobile phones, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminals, augmented reality (AR) terminals, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of this application do not limit the specific technology or device form used in the terminal.

[0025] like Figure 1 As shown, the image processing method may include the following steps S101 to S103: Step S101: Obtain the target residual based on the first luminance component in the first image to be processed; wherein, the target residual is used to characterize the processing error of super-resolution processing of the first image.

[0026] The method of acquiring the first image to be processed is not limited. As an example, the first image may be an image acquired in response to a triggering operation, wherein the triggering operation includes, but is not limited to, a shooting operation, a display operation, a selection operation, etc.

[0027] For example, taking a shooting operation as an example of a triggering operation, the first image may be an image captured by the user, that is, the first image is an image captured in response to a shooting operation triggered by the user, wherein the first image includes the original image captured by the camera (referred to as the captured image in this application), or a new image obtained by image processing of the captured image.

[0028] Taking an image captured in RGB color space as an example, the captured image can be converted to a different color space (such as YUV color space) to obtain a first image.

[0029] For example, taking the triggering operation including a display operation as an example, the first image may be an image to be displayed to the user, or the first image may be a video frame in the decoded video stream, or the first image may be a new image generated based on the video frame, etc.

[0030] The method of acquiring the image to be displayed is not limited. For example, the image to be displayed can be an image captured by a camera, or an image synthesized by an artificial means, or an image captured online, or an image obtained from a training set or a test set, etc. This application embodiment does not limit this.

[0031] For example, taking a selection operation as an example of a triggering operation, the first image may be an image resource selected from local storage resources in response to a selection operation triggered by a user.

[0032] It should be noted that the above-described method of obtaining the first image is merely an illustrative example, and this application is not limited thereto. For example, the first image may also be an artificially synthesized image; the first image may also be an image collected online by the user, such as by using web crawler technology to collect the first image online; the first image may also be a game screen to be rendered in a game, such as the game screen of the current frame in the game.

[0033] The target residual exists in the form of an image, and each pixel in the target residual corresponds to a residual value used to quantify the brightness difference. That is, the target residual is used to characterize the brightness difference between the high-resolution image (referred to as the second image in this application) after super-resolution processing of the first image and the ideal high-resolution image; wherein, the residual value corresponding to each pixel in the target residual is used to indicate the brightness difference between the brightness components of corresponding pixels in the two high-resolution images.

[0034] Super-resolution processing (or resolution enhancement) includes, but is not limited to: interpolation upsampling (such as bilinear and bicubic interpolation algorithms), improved upsampling based on interpolation (such as bicubic interpolation), reconstruction-based super-resolution processing (using prior knowledge of the image itself (such as smoothness and edge continuity) to build an optimization model and recover the high-resolution image by solving the inverse problem), and learning-based super-resolution processing (such as neighborhood embedding, local linear regression, and sparse coding).

[0035] In this embodiment of the application, residual prediction can be performed based on the first luminance component in the first image to be processed to obtain a high-resolution target residual.

[0036] Step S102: Based on the target gain associated with the first image, the second luminance component and the target residual in the second image are fused to obtain the target luminance component; wherein, the target gain is used to characterize the enhancement intensity of the target residual; the second image is obtained by super-resolution processing of the first image.

[0037] The target gain is associated with the first image, and the gain associated with different images can be different.

[0038] In this embodiment of the application, the first image can be super-resolution processed to obtain a high-resolution second image, and based on the target gain associated with the first image, the second luminance component and the target residual in the second image can be fused to obtain a high-resolution target luminance component.

[0039] For example, the first luminance component is labeled Y, and the second luminance component is labeled Y. The high-resolution target residual is R H The target gain is gain, and the high-resolution target luminance component is Y. H Then Y H The following formula can be used to calculate it: ; It should be noted that applying uniform enhancement parameters (such as fixed gain) to all input images makes it difficult to simultaneously address the visual characteristics of different content types. For example, portrait images need to avoid over-sharpening that could distort skin texture, while text or architectural images require stronger edge enhancement to improve readability and clarity. Related super-resolution processing algorithms lack the ability to understand image content, resulting in inconsistent enhancement effects. Figure 1 "Strategy" carries the risk of losing details or amplifying artifacts.

[0040] To address this issue, in any embodiment of this application, a gain selection mechanism based on semantic recognition can be introduced to achieve content awareness and differentiated control in the super-resolution enhancement process. Specifically, in this application, semantic recognition can be performed on the first image to obtain its semantic information, and a gain associated with this semantic information can be determined from multiple set gains as the target gain associated with the first image. Therefore, in this application, this target gain can be used to fuse the second luminance component and the target residual in the second image to obtain the target luminance component.

[0041] For example, a medium-to-high target gain can be configured for the first image in the "food" or "landscape" category to enhance color contrast and local details; a lower target gain can be used for the first image in the "portrait" category to prevent skin blemishes from being over-magnified; and a higher target gain can be used for the first image in the "text" or "chart" category to enhance edge sharpness and line clarity.

[0042] It is understandable that the target gain, as an adjustment coefficient for residual compensation or brightness reconstruction, directly affects the recovery intensity of high-frequency details. In this application, by binding the gain selection with semantic information, the enhancement intensity is dynamically changed with the image content, avoiding the over-enhancement or under-enhancement problems caused by "one-size-fits-all" approach, and significantly improving the detail restoration capability and visual naturalness of super-resolution processing.

[0043] Step S103: Update the second image based on the target brightness component to obtain the target image.

[0044] In this embodiment of the application, the second luminance component in the second image can be updated based on the high-resolution target luminance component to obtain a high-resolution target image.

[0045] In any embodiment of this application, when the first image is an image acquired in response to a shooting operation, a preview can also be performed based on the target image. For example, taking the first image as being in YUV format, the target image can be converted to RGB format before previewing.

[0046] In any embodiment of this application, when the first image is an image acquired in response to a display operation, image rendering can also be performed based on the target image for display. For example, taking the first image as having a YUV format, the target image can be converted to RGB format before image rendering for display.

[0047] Therefore, by processing the target image in a targeted manner based on the acquisition method of the first image, the personalized needs of different users can be met.

[0048] The image processing method of this application, by introducing a dynamically adjustable residual compensation mechanism, can overcome the limitations of the "fixed weights and fixed outputs" of the AISR model in related technologies. Specifically, firstly, a target residual is obtained based on the first luminance component in the first image to be processed. This target residual is used to characterize the processing error generated after super-resolution processing of the first image, that is, the difference in luminance between the reconstructed high-resolution image and the ideal high-resolution image. Subsequently, based on the target gain associated with the first image, the second luminance component in the second image (the reconstructed high-resolution image) is fused with the target residual to generate a target luminance component, and the luminance channel in the second image is updated accordingly, finally obtaining the optimized target image. Among them, the target gain is used to adjust the enhancement intensity of the residual information. In essence, it is a content-adaptive adjustment coefficient that can be dynamically determined according to the visual characteristics of the first image. For example, in areas with rich details or low contrast, a higher gain is used to enhance detail recovery, while in smooth or high-noise areas, the gain is reduced to suppress artifacts and noise amplification. This enables dynamic optimization of super-resolution effects, which not only significantly improves the reconstruction quality and detail fidelity of images, but also enhances the adaptability and practicality of the algorithm in diverse and complex real-world scenarios.

[0049] As one possible implementation method, Figure 2 A schematic flowchart of another image processing method provided for an exemplary embodiment of this application.

[0050] It should be noted that the image processing method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in related technologies. The embodiments of this application do not limit this.

[0051] like Figure 2 As shown, the image processing method may include the following steps S201 to S205: Step S201: Obtain the target residual based on the first luminance component in the first image to be processed; wherein, the target residual is used to characterize the processing error of super-resolution processing of the first image.

[0052] It should be noted that the explanation of step S201 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0053] Step S202: Determine the target application scenario to which the first image belongs based on the metadata in the semantic information of the first image; wherein, the metadata includes: the type of the medium to which the first image belongs and / or content tags.

[0054] The type of media to which the first image belongs includes, but is not limited to, the type of application (APP) on which the first image is based, such as short videos, live streaming, and educational platforms.

[0055] The content tags include the subject classification of the first image, such as food, sports, indoor, outdoor, etc. The content tags can be obtained by classifying the first image using a visual classification model, or by extracting text keywords from the text information (or character information) in the first image; this application embodiment does not impose any limitations on this.

[0056] It's important to note that different application scenarios have fundamentally different requirements for image quality. For example, short video scenarios prioritize high visual impact, favoring vibrant colors and sharp details; live streaming scenarios emphasize low latency and stability, requiring a balance between image quality and frame rate within limited computing power; online education scenarios prioritize the clarity of text and charts, necessitating enhanced edges in PPT presentations or whiteboard writing; video surveillance scenarios focus on the recognizability of faces and license plates, requiring local enhancement rather than global sharpening; and medical imaging scenarios prioritize fidelity, avoiding misdiagnosis due to over-enhancement. Using a uniform enhancement gain would fail to meet the personalized quality goals of these diverse scenarios.

[0057] To address the aforementioned issues, this application identifies the application scenario to which the first image belongs (referred to as the target application scenario in this application) and determines the corresponding target gain accordingly. This achieves a leap from "general processing" to "scenario adaptation," improving image quality and user satisfaction. Specifically, the target application scenario to which the first image belongs can be determined based on the metadata of the first image; the correspondence between different metadata and application scenarios can be pre-set or configured.

[0058] Step S203: Determine the target gain associated with the target application scenario from multiple set gains; wherein the target gain is used to characterize the enhancement intensity of the target residual.

[0059] The target gain associated with different target application scenarios can be different.

[0060] In any embodiment of this application, different gains (referred to as set gains in this application) can be set in advance for different application scenarios, and a mapping relationship can be established to indicate the association between multiple different set gains and application scenarios. Thus, in this application, the above mapping relationship can be queried according to the target application scenario to determine the target gain associated with the target application scenario.

[0061] Understandably, by pre-establishing a mapping relationship between "application scenario → gain value" (i.e., a gain lookup table), and using the lookup table method at runtime to obtain the corresponding target gain based on the target application scenario to which the first image belongs, fast matching and deterministic output of gain configuration are achieved. The lookup table operation is essentially a memory addressing process with a time complexity of O(1), far lower than the large number of multiplication and addition operations required for real-time gain calculation using neural networks or complex algorithms. This significantly reduces latency, especially in edge devices, mobile terminals, or real-time video processing scenarios, meeting the performance requirements of high frame rates and low power consumption.

[0062] In summary, the table lookup method is used to determine the target gain, transforming the complex adaptive enhancement decision into an efficient, stable, and configurable table lookup process. This retains the flexibility of content awareness while avoiding high-cost real-time computation, significantly improving the system's real-time performance, stability, and deployability while ensuring image enhancement quality.

[0063] Step S204: Based on the target gain, the second luminance component and the target residual in the second image are fused to obtain the target luminance component; wherein, the second image is obtained by super-resolution processing of the first image.

[0064] Step S205: Update the second image based on the target brightness component to obtain the target image.

[0065] It should be noted that the explanation of steps S204 to S205 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0066] The image processing method of this application embodiment determines the target gain associated with the first image based on the target application scenario to which the first image belongs. This enables the super-resolution enhancement process to have context-aware capabilities and can dynamically adjust the enhancement intensity according to actual business needs, thereby improving image quality and user satisfaction.

[0067] As one possible implementation method, Figure 3 This is a schematic flowchart of yet another image processing method provided for an exemplary embodiment of this application.

[0068] It should be noted that the image processing method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in related technologies. The embodiments of this application do not limit this.

[0069] like Figure 3 As shown, the image processing method may include the following steps S301 to S304: Step S301: Obtain the target residual based on the first luminance component in the first image to be processed; wherein, the target residual is used to characterize the processing error of super-resolution processing of the first image.

[0070] It should be noted that the explanation of step S301 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0071] Step S302: Based on the semantic information of the first image, determine the target gain associated with the semantic information from a plurality of set gains; wherein the target gain includes the target gain associated with at least one patch in the first image, and each patch displays an object.

[0072] The semantic information of the first image includes the various objects displayed in the first image, including but not limited to: humans, animals, plants, and scenery.

[0073] In this image, each object is associated with a target gain, and the target gains associated with different objects may be different. For example, the target gain associated with each object may be determined based on the visual features of that object in the first image.

[0074] Visual features include, but are not limited to: local structural features (such as texture), edge contour features (such as gradient), lighting features (such as brightness), and statistical distribution features (such as histogram).

[0075] It should be noted that applying a uniform enhancement intensity to the entire image may result in an imbalance in enhancement effects across different object regions. For example, textured areas (such as leaves or fabric) may exhibit artifacts or amplified noise due to excessive gain, while low-contrast areas (such as facial skin) may appear blurry due to insufficient gain.

[0076] To address the aforementioned issues, this application can identify each object in the first image and, based on the visual features of each object in the first image, determine the target gain associated with each object, or the target gain associated with the corresponding patch of each object.

[0077] In any embodiment of this application, an image segmentation algorithm or a semantic segmentation algorithm can be used to segment the first image to obtain the visual features of objects in at least one patch in the first image, and the target gain associated with each object's corresponding patch can be determined from multiple set gains based on the visual features of each object.

[0078] The target gain associated with different visual features can be different.

[0079] As an example, deep learning techniques can be used to determine the target gain associated with each object's corresponding patch from multiple set gains based on the object's visual features. For instance, for any object displayed in the first image, the probability of the object fitting multiple set gains can be predicted based on the object's visual features, thereby determining the target gain with the highest probability from multiple set gains.

[0080] In summary, by analyzing the visual features of each object and determining the target gain independently for each object, the enhancement intensity can be matched with the object's own properties, achieving fine-grained control in the spatial domain. This not only significantly improves the detail restoration capability and visual clarity of key objects, but also effectively avoids over-processing of non-critical areas, balancing image fidelity, subjective quality, and algorithm robustness.

[0081] In any embodiment of this application, firstly, the first image can be downsampled based on a downsampling mechanism to obtain a third image. Then, the third image can be filled with image edges (i.e., image edge padding) based on an image edge filling mechanism to obtain a fourth image. Subsequently, the fourth image can be segmented based on a semantic segmentation mechanism to obtain a mask image. Finally, based on the mask image, features can be extracted from objects in at least one patch in the first image to obtain visual features of objects in at least one patch. Based on the visual features of each object, the target gain associated with each object's corresponding patch can be determined from multiple set gains.

[0082] Among them, the downsampling mechanism can improve processing efficiency and scale adaptability; the image edge filling mechanism can ensure boundary integrity and convolution operation stability; and the semantic segmentation mechanism can achieve pixel-level object recognition and region division.

[0083] Therefore, based on the downsampling mechanism, image edge filling mechanism, and semantic segmentation mechanism, the first image is processed collaboratively to obtain the visual features of objects in each patch of the first image. This can reduce computational complexity while maintaining the integrity of image boundaries and achieving pixel-level object region division, thereby accurately obtaining the visual features of objects in each patch (such as texture, edge, contrast, etc.). This not only improves the accuracy and efficiency of feature extraction but also enhances the adaptability to multi-scale and complex layout images, providing a reliable foundation for subsequent object-level adaptive image enhancement (such as differential gain adjustment).

[0084] Step S303: Using the target gain associated with at least one patch, the second brightness component in the second image and the corresponding patch in the target residual are fused to obtain the target brightness component; wherein, the second image is obtained by super-resolution processing of the first image.

[0085] The explanations and descriptions of the second image and super-resolution processing in the foregoing embodiments also apply to this embodiment, and will not be repeated here.

[0086] In this embodiment, for any patch in the first image, a target gain associated with that patch can be used to fuse the second luminance component in the second image and the corresponding patch in the target residual to obtain the target luminance component. That is, a luminance sub-component belonging to the same object can be determined from the second luminance component, and a sub-residue belonging to the same object can be determined from the target residual. Therefore, a target gain associated with the same object can be used to fuse the luminance sub-component and the sub-residue to obtain the target luminance component.

[0087] Step S304: Update the second image based on the target brightness component to obtain the target image.

[0088] It should be noted that the explanation of step S304 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0089] The image processing method of this application embodiment determines the target gain independently for each object, so that the enhancement intensity matches the object's own properties, achieves local optimal enhancement, and improves the overall visual quality. It is applicable to complex image scenes with multiple mixed objects and significant differences in visual importance.

[0090] As one possible implementation method, Figure 4 This is a schematic flowchart of another image processing method provided for an exemplary embodiment of this application.

[0091] It should be noted that the image processing method can be executed alone, or it can be executed together with any embodiment of this application or any possible implementation in the embodiment, or it can be executed together with any technical solution in related technologies. The embodiments of this application do not limit this.

[0092] like Figure 4 As shown, the image processing method may include the following steps S401 to S404: Step S401: Perform residual prediction based on the first luminance component in the first image to obtain an initial residual; wherein the resolution of the initial residual matches the resolution of the first image.

[0093] In this embodiment of the application, a low-resolution initial residual can be obtained by performing residual prediction based on the first luminance component in the first image using deep learning technology.

[0094] As an example, a trained deep learning model (including but not limited to convolutional neural networks) can be used to perform residual prediction on the first luminance component to obtain the initial residual.

[0095] The deep learning model can be trained based on low-resolution images and the labeled residuals between the reconstructed high-resolution images (obtained by super-resolution processing of low-resolution images) and the ideal high-resolution images. That is, the brightness component of the low-resolution image can be input into the deep learning model for inference to obtain the prediction residual. Based on the prediction residual and the labeled residual, the deep learning model is trained to obtain the trained deep learning model.

[0096] For example, the labeled residual can be a high-resolution image residual, and the predicted residual can be a low-resolution image residual. In this application, the predicted residual can be upsampled to obtain a high-resolution residual, and the value of the loss function can be determined based on the difference between the high-resolution residual and the labeled residual. In this application, this value is denoted as the loss value. Thus, the deep learning model can be trained based on the loss value to minimize the loss value.

[0097] The loss value is positively correlated with the above differences; that is, the smaller the difference, the smaller the loss value, and vice versa.

[0098] It should be noted that the above example only uses minimizing the loss value as the termination condition for model training, but this application is not limited to this. For example, the termination condition may also include: the training time reaches a set time, the training rounds reach a set number of rounds, etc. The embodiments of this application do not limit this.

[0099] Step S402: Perform subpixel convolution operation on the initial residual to obtain the target residual; wherein, the target residual is used to characterize the processing error of super-resolution processing of the first image.

[0100] In this embodiment, a subpixel convolution operation can be performed on the initial residual to upsample it into a target residual. The resolution of the target residual matches the resolution of the second image, and it is used to characterize the processing error of the super-resolution processing of the first image. Specifically, the target residual characterizes the brightness difference between the high-resolution image (the second image) after super-resolution processing of the first image and the ideal high-resolution image. The residual value corresponding to each pixel in the target residual indicates the brightness difference between the brightness components of corresponding pixels in the two high-resolution images.

[0101] Step S403: Based on the target gain associated with the first image, the second luminance component and the target residual in the second image are fused to obtain the target luminance component; wherein, the target gain is used to characterize the enhancement intensity of the target residual; the second image is obtained by super-resolution processing of the first image.

[0102] Step S404: Update the second image based on the target brightness component to obtain the target image.

[0103] It should be noted that the explanations of steps S402 to S404 can be found in the relevant descriptions in any embodiment of this application, and will not be repeated here.

[0104] The image processing method of this application embodiment obtains an initial residual that matches the resolution of the first image by performing residual prediction based on the first luminance component in the first image, and performs subpixel convolution operation on it to generate a target residual. This enables efficient residual feature extraction in low-resolution space, reducing computational complexity and memory usage. At the same time, the subpixel convolution achieves accurate upsampling of the residual, ensuring that the target residual has fine-grained structural alignment capability in high-resolution space, effectively preserving edge and texture details, improving its matching accuracy with the second image after super-resolution processing, and thus enhancing the quality of subsequent luminance component fusion.

[0105] In any embodiment of this application, to address the problem of fixed network effects in related technologies, different enhancement effects and super-resolution effects can be obtained by setting different intensity gain. To address the problem of lack of scalability in related technologies, this application has scalability by accessing local statistical information and adaptively calculating the enhancement intensity of the residual.

[0106] As an example, the super-resolution processing workflow for low-resolution images can be as follows: Figure 5As shown, it mainly includes the following steps: 1. If the input data is a High Dynamic Range (HDR) video stream, a decoder (HEVC, AVC, etc.) is used to decode the HDR video stream to obtain YUV data; if the input data is an RGB image, an RGB2YUV conversion matrix is ​​used to convert the RGB image to a YUV image.

[0107] HEVC stands for High Efficiency Video Coding, and AVC stands for Advanced Video Coding.

[0108] 2. Input the Y component obtained in step 1 into the neural network, and calculate the initial residual R through network inference. L .

[0109] 3. R L Upsampling is performed through subpixel convolution to obtain the high-resolution image residual R. H .

[0110] 4. Process the Y component obtained in step 1 using a super-resolution algorithm or a scaling algorithm to obtain a high-resolution image with relatively poor quality. .

[0111] 5. The final high-resolution image Y is calculated using the following formula. H : ; The gain value can be manually set by the debugger or calculated by connecting other computing modules (traditional modules or neural network modules). For example, the calculation method is as follows: Method 1: Set different gain based on different application scenarios, such as different apps, different types of video content; Method 2: By extending the network structure, traditional image segmentation algorithms or AI-based semantic segmentation algorithms are incorporated to process the YUV image, resulting in a mask image. This mask image divides objects in the YUV image into different regions or patches. This mask image is then used to filter gain values ​​from a pre-defined gain array to obtain gain values ​​associated with different object regions; different object regions use different gain values.

[0112] It should be noted that the UV components in a YUV image can be processed directly using super-resolution or scaling algorithms.

[0113] To implement the above embodiments, this application also proposes an image processing apparatus.

[0114] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided for an exemplary embodiment of this application.

[0115] like Figure 6 As shown, the image processing device 600 may include: an acquisition module 610, a fusion module 620, and an update module 630.

[0116] The acquisition module 610 is used to acquire the target residual based on the first luminance component in the first image to be processed; wherein the target residual is used to characterize the processing error of super-resolution processing of the first image.

[0117] The fusion module 620 is used to fuse the second luminance component and the target residual in the second image based on the target gain associated with the first image to obtain the target luminance component; wherein, the target gain is used to characterize the enhancement intensity of the target residual; the second image is obtained by super-resolution processing of the first image.

[0118] The update module 630 is used to update the second image based on the target brightness component to obtain the target image.

[0119] In one implementation of this application, the target gain is determined using the following modules: The determination module is used to determine the target gain associated with the semantic information from multiple set gains based on the semantic information of the first image.

[0120] In one implementation of this application, in response to semantic information including metadata of a first image, a determining module is configured to: determine the target application scenario to which the first image belongs based on the metadata; wherein the metadata includes: the type of media to which the first image belongs and / or content tags; and determine a target gain associated with the target application scenario from a plurality of set gains.

[0121] In one implementation of this application, the determining module is configured to: query the mapping relationship according to the target application scenario to determine the target gain associated with the target application scenario; wherein, the mapping relationship is used to indicate the association between multiple set gains and the application scenario.

[0122] In one implementation of this application, the target gain includes a target gain associated with at least one patch in the first image, each patch displaying an object; the fusion module 620 is used to: fuse the second brightness component and the corresponding patch in the target residual using the target gain associated with at least one patch to obtain the target brightness component.

[0123] In one implementation of this application, in response to semantic information including various objects displayed in a first image, a determining module is configured to: perform image segmentation on the first image to obtain visual features of objects in at least one patch in the first image; and determine a target gain associated with each object's corresponding patch from a plurality of set gains based on the visual features of each object.

[0124] In one implementation of this application, the determining module is configured to: downsample the first image to obtain a third image; fill the image edges of the third image to obtain a fourth image; segment the fourth image to obtain a mask image; and extract features from objects in at least one patch of the first image based on the mask image to obtain the visual features of the objects in at least one patch.

[0125] In one implementation of this application, the acquisition module 610 is configured to: perform residual prediction based on the first luminance component in the first image to obtain an initial residual; wherein the resolution of the initial residual matches the resolution of the first image; perform a sub-pixel convolution operation on the initial residual to obtain a target residual; wherein the resolution of the target residual matches the resolution of the second image.

[0126] In one implementation of this application, the first image is an image acquired in response to a trigger operation; wherein, the trigger operation includes a shooting operation, the first image is a captured image, or the first image is generated based on the captured image; The image processing device 600 also includes: The preview module is used to preview the target image.

[0127] In one implementation of this application, the response to the trigger operation includes a display operation, wherein the first image is an image to be displayed, or the first image is a video frame in the decoded video stream, or the first image is generated based on the video frame; The image processing device 600 also includes: The display module is used to render and display images based on the target image.

[0128] It should be noted that the foregoing explanation of any image processing method embodiment also applies to the image processing apparatus of this embodiment, and will not be repeated here.

[0129] In the image processing apparatus of this application embodiment, by introducing a dynamically adjustable residual compensation mechanism, the limitations of the AISR model in related technologies, which features "fixed weights and solidified output," can be overcome. Specifically, firstly, a target residual is obtained based on the first luminance component in the first image to be processed. This target residual is used to characterize the processing error generated after super-resolution processing of the first image, i.e., the difference in luminance between the reconstructed high-resolution image and the ideal high-resolution image. Subsequently, based on the target gain associated with the first image, the second luminance component in the second image (the reconstructed high-resolution image) is fused with the target residual to generate a target luminance component. The luminance channel in the second image is then updated accordingly, ultimately obtaining the optimized target image. The target gain is used to adjust the enhancement intensity of the residual information; it is essentially a content-adaptive adjustment coefficient that can be dynamically determined according to the visual characteristics of the first image. For example, in areas with rich detail or low contrast, a higher gain is used to enhance detail recovery; in smooth or high-noise areas, the gain is reduced to suppress artifacts and noise amplification. This enables dynamic optimization of super-resolution effects, which not only significantly improves the reconstruction quality and detail fidelity of images, but also enhances the adaptability and practicality of the algorithm in diverse and complex real-world scenarios.

[0130] To implement the above embodiments, this application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method as described in any of the foregoing embodiments.

[0131] Figure 7 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. For example, the electronic device 700 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0132] Reference Figure 7 The electronic device 700 may include one or more of the following components: processing component 702, memory 704, power component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716.

[0133] Processing component 702 typically controls the overall operation of electronic device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.

[0134] Memory 704 is configured to store various types of data to support the operation of electronic device 700. Examples of this data include instructions for any application or method operating on electronic device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0135] Power component 706 provides power to various components of electronic device 700. Power component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 700.

[0136] Multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0137] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when electronic device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.

[0138] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0139] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of electronic device 700. For example, sensor assembly 714 can detect the on / off state of electronic device 700, the relative positioning of components such as the display and keypad of electronic device 700, changes in position of electronic device 700 or a component of electronic device 700, the presence or absence of user contact with electronic device 700, orientation or acceleration / deceleration of electronic device 700, and temperature changes of electronic device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0140] Communication component 716 is configured to facilitate wired or wireless communication between electronic device 700 and other devices. Electronic device 700 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0141] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0142] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of an electronic device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0143] To implement the above embodiments, this application also proposes a chip, wherein the chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to perform the image processing method provided in any of the foregoing embodiments.

[0144] Figure 8 This is a schematic diagram of the structure of a chip proposed in an exemplary embodiment of this application. See also... Figure 8 The diagram shown is a schematic representation of the structure of chip 800, but is not limited to this.

[0145] Chip 800 includes processing circuit 801, which is configured to perform any of the above image processing methods.

[0146] In some embodiments, the chip 800 further includes one or more interface circuits 802. Optionally, the interface circuit 802 is connected to the memory 803, and the interface circuit 802 can be used to receive signals from the memory 803 or other devices, and the interface circuit 802 can be used to send signals to the memory 803 or other devices. For example, the interface circuit 802 can read instructions stored in the memory 803 and send the instructions to the processing circuit 801.

[0147] In some embodiments, the interface circuit 802 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 801 performs other steps.

[0148] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0149] In some embodiments, chip 800 further includes one or more memories 803 for storing instructions. Optionally, all or part of the memories 803 may be located outside of chip 800.

[0150] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image processing method as described in any of the foregoing method embodiments.

[0151] To implement the above embodiments, this application also proposes a computer program product having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image processing method as described in any of the foregoing method embodiments.

[0152] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0153] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0154] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0155] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and compact disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0156] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0157] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0159] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An image processing method, characterized in that, include: The target residual is obtained based on the first luminance component in the first image to be processed; wherein the target residual is used to characterize the processing error of super-resolution processing of the first image. Based on the target gain associated with the first image, the second luminance component and the target residual in the second image are fused to obtain the target luminance component; wherein, the target gain is used to characterize the enhancement intensity of the target residual; the second image is obtained by performing the super-resolution processing on the first image; The second image is updated based on the target brightness component to obtain the target image.

2. The method according to claim 1, characterized in that, The target gain is determined using the following steps: Based on the semantic information of the first image, a target gain associated with the semantic information is determined from a plurality of set gains.

3. The method according to claim 2, characterized in that, In response to the semantic information including the metadata of the first image, The step of determining a target gain associated with the semantic information from a plurality of preset gains based on the semantic information of the first image includes: Based on the metadata, the target application scenario to which the first image belongs is determined; wherein, the metadata includes: the type of media to which the first image belongs and / or content tags; From the plurality of set gains, determine the target gain associated with the target application scenario.

4. The method according to claim 3, characterized in that, Determining the target gain associated with the target application scenario from the plurality of set gains includes: The target gain associated with the target application scenario is determined by querying the mapping relationship based on the target application scenario. The mapping relationship is used to indicate the association between the multiple set gains and the application scenario.

5. The method according to claim 2, characterized in that, The target gain includes a target gain associated with at least one tile in the first image, each tile displaying an object; The step of fusing the second luminance component in the second image and the target residual based on the target gain associated with the first image to obtain the target luminance component includes: The target brightness component is obtained by fusing the second brightness component and the corresponding block in the target residual using the target gain associated with the at least one patch.

6. The method according to claim 5, characterized in that, In response to the semantic information including the various objects shown in the first image, The step of determining a target gain associated with the semantic information from a plurality of preset gains based on the semantic information of the first image includes: The first image is segmented to obtain the visual features of objects in at least one patch in the first image; Based on the visual characteristics of each object, the target gain associated with the corresponding tile of each object is determined from the plurality of set gains.

7. The method according to claim 6, characterized in that, The step of performing image segmentation on the first image to obtain the visual features of objects in at least one patch of the first image includes: The first image is downsampled to obtain the third image; The third image is then filled with image edges to obtain the fourth image; The fourth image is segmented to obtain a mask image; Based on the mask image, feature extraction is performed on objects in at least one patch of the first image to obtain the visual features of the objects in the at least one patch.

8. The method according to any one of claims 1-7, characterized in that, The step of obtaining the target residual based on the first luminance component in the first image includes: Residual prediction is performed based on the first luminance component in the first image to obtain an initial residual; wherein the resolution of the initial residual matches the resolution of the first image; A subpixel convolution operation is performed on the initial residual to obtain the target residual; wherein the resolution of the target residual matches the resolution of the second image.

9. The method according to any one of claims 1-7, characterized in that, The first image is an image acquired in response to a trigger operation; wherein, In response to the triggering operation, which includes a shooting operation, the first image is either a captured image or the first image is generated based on the captured image. The method further includes: Preview based on the target image.

10. The method according to claim 9, characterized in that, The trigger operation includes a display operation, wherein the first image is an image to be displayed, or the first image is a video frame in a decoded video stream, or the first image is generated based on the video frame; The method further includes: The target image is then rendered and displayed.

11. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire a target residual based on a first luminance component in a first image to be processed; wherein the target residual is used to characterize the processing error of super-resolution processing of the first image; A fusion module is used to fuse a second luminance component and a target residual in a second image based on a target gain associated with the first image to obtain a target luminance component; wherein the target gain is used to characterize the enhancement intensity of the target residual; and the second image is obtained by performing the super-resolution processing on the first image; An update module is used to update the second image based on the target brightness component to obtain the target image.

12. The apparatus according to claim 11, characterized in that, The target gain is determined using the following module: The determining module is used to determine a target gain associated with the semantic information from a plurality of set gains based on the semantic information of the first image.

13. The apparatus according to claim 12, characterized in that, In response to the semantic information including metadata of the first image, the determining module is configured to: Based on the metadata, the target application scenario to which the first image belongs is determined; wherein, the metadata includes: the type of media to which the first image belongs and / or content tags; From the plurality of set gains, determine the target gain associated with the target application scenario.

14. The apparatus according to claim 12, characterized in that, The target gain includes a target gain associated with at least one tile in the first image, each tile displaying an object; the fusion module is configured to: The target brightness component is obtained by fusing the second brightness component and the corresponding block in the target residual using the target gain associated with the at least one patch.

15. The apparatus according to claim 14, characterized in that, In response to the semantic information including the various objects displayed in the first image, the determining module is configured to: The first image is segmented to obtain the visual features of objects in at least one patch in the first image; Based on the visual characteristics of each object, the target gain associated with the corresponding tile of each object is determined from the plurality of set gains.

16. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 10.

17. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method according to any one of claims 1 to 10.

18. A chip, characterized in that, The chip includes an interface circuit and a processing circuit that are coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is used to implement the method of any one of claims 1 to 10.

19. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images