Image Processing Method and Related Device

Abstract

An embodiment of the present application provides an image processing method and related devices, which relate to the field of terminal technologies and are applied to electronic devices. The method includes: receiving an operation for taking a photo; in response to the operation for taking a photo, acquiring an image collected by a camera of the electronic device; processing the image collected by the camera to obtain a first image; and eliminating pseudo-textures in the first image to obtain a photographed image. In this way, in a shooting scenario, the electronic device can obtain high-quality photos with high clarity and no pseudo-textures, thereby improving the user experience.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to an image processing method and related devices. Background Art

[0002] An electronic device can support a photographing function. A user can use the electronic device to take pictures.

[0003] However, in a possible implementation, when the electronic device takes a picture of a photographed object, the picture obtained by the electronic device may have problems. For example, there may be some textures in the picture of the photographed object that do not exist on the photographed object, which affects the quality of the image. Summary of the Invention

[0004] Embodiments of this application provide an image processing method and related devices, which are applied to the technical field of terminals. In a photographing scenario, by eliminating pseudo-textures in an image, the image quality is improved, and thus the user's photographing experience is improved.

[0005] In a first aspect, an embodiment of this application proposes an image processing method, which is applied to an electronic device. The method includes: receiving an operation for taking a picture; in response to the operation for taking a picture, acquiring an image collected by a camera of the electronic device; processing the image collected by the camera to obtain a first image; and eliminating pseudo-textures in the first image to obtain a photographed image. In this way, in a photographing scenario, the electronic device can obtain a high-quality photograph with high clarity and no pseudo-textures, thereby improving the user's experience.

[0006] In a possible implementation manner, eliminating pseudo-textures in the first image to obtain a photographed image includes: detecting a first area in the first image where pseudo-textures exist; processing the image collected by the camera to obtain a second image; where the specification of the second image is the same as that of the first area, the clarity of the second image is less than that of the first image, and there are no pseudo-textures in the second image; and fusing the first area of the first image with the second image to obtain a photographed image. In this way, in a photographing scenario, the electronic device can obtain an image in which there are no pseudo-textures in the first area and the clarity of other areas is relatively high, thereby improving the image quality and the user's photographing experience.

[0007] In a possible implementation, fusing a first region of a first image with a second image includes: fusing a first pixel region in the first region of the first image with a second pixel region in the second image; the second pixel region is the pixel region corresponding to the first pixel region in the second image; the first pixel region includes one or more pixel points. In this way, when the granularity is a pixel point, the fusion effect of Image 1 and Image 3 is better, and a high-quality Image 4 can be obtained; when the granularity is a combination of multiple pixel points, the computing pressure of the electronic device in the image processing process can be reduced, and the computing efficiency can be improved.

[0008] In a possible implementation, the electronic device fuses the first pixel region and the second pixel region based on a first weight and a second weight; wherein, the first pixel region corresponds to the first weight, and the second pixel region corresponds to the second weight; the second weight is related to the probability value that the first pixel region is a pseudo-texture, the texture information of the first pixel region, and the exposure parameter during photographing; the first weight is negatively correlated with the second weight. In this way, the electronic device can comprehensively set the second weight and the first weight through multiple dimensions such as the probability value, texture information, and exposure parameter in Image 1, improve the accuracy of the first weight and the second weight, and thus improve the image fusion effect.

[0009] In a possible implementation, the texture information of the first pixel region includes the gradient information and variance information of the first pixel region, and the exposure parameter includes the sensitivity ISO and the exposure duration; wherein, the second weight is positively correlated with the probability value that the first pixel region is a pseudo-texture; the second weight is positively correlated with the gradient information of the first pixel region; the second weight is positively correlated with the variance information of the first pixel region; the second weight is positively correlated with the ISO; the second weight is negatively correlated with the exposure duration. In this way, the electronic device can comprehensively set the second weight and the first weight through multiple dimensions such as the probability value, texture information, and exposure parameter in Image 1, improve the accuracy of the first weight and the second weight, and thus improve the image fusion effect.

[0010] In a possible implementation, detecting a first region with pseudo-texture in the first image includes: processing the first image to obtain the probability value that the first pixel region is a pseudo-texture; performing edge detection on the first image and dividing the first image into N regions according to the texture density; any one of the N regions is preset with a corresponding probability value threshold, and N is an integer greater than 1; when the first pixel region is located in the i-th region and the probability value that the first pixel region is a pseudo-texture is greater than the probability value threshold corresponding to the i-th region, the electronic device obtains the first region; wherein, the first region includes the first pixel region, and i is a positive integer less than or equal to N. In this way, the electronic device can dynamically adjust the probability value threshold according to the amount of texture in different regions, improve the accuracy and precision of the pseudo-texture region, and further improve the subsequent image fusion effect and image quality.

[0011] In a possible implementation, the probability value threshold corresponding to the i-th region is positively correlated with the texture density of the i-th region. In this way, the electronic device sets a lower probability value threshold in the flat region and a higher probability value threshold in the strong texture region, improving the accuracy and precision of the pseudo-texture region, thereby enhancing the effect of subsequent image fusion and improving the image quality.

[0012] In a possible implementation, the electronic device processes the first image based on the first model, and the first model is trained based on the sample images marked with pseudo-textures. In this way, the electronic device can obtain Image 3 with slightly lower clarity than Image 1 but without pseudo-textures based on the pixel pseudo-texture probability prediction network model.

[0013] In a possible implementation, it further includes: processing the image collected by the camera to obtain a third image; the clarity of the third image is less than that of the second image; performing edge detection on the third image and correcting N regions in the first image according to the texture density of the third image. In this way, although the clarity of a single-frame image is poor, there are no pseudo-textures, and the electronic device can use the single-frame image to calibrate and correct Image 1, thereby improving the accuracy of the region division result and enhancing the effect of subsequent image fusion.

[0014] In a possible implementation, processing the image collected by the camera to obtain a second image includes: obtaining a fourth image from the image collected by the camera according to the position of the first region; the specification of the fourth image is the same as that of the first region; processing the fourth image based on the second model to obtain the second image. In this way, the electronic device can obtain Image 3 without pseudo-textures and with high clarity. During the image fusion process, the pseudo-texture effect in the pseudo-texture region of Image 1 is weakened through Image 3, thereby improving the image quality.

[0015] In a possible implementation, the image collected by the camera is in RAW format, the first image is in RGB format, the second image is in RGB format, the third image is in RGB format, the fourth image is in RAW format, and the captured image is in RGB format.

[0016] Second aspect, embodiments of the present application provide an electronic device, which can also be referred to as a terminal device, a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device can be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and so on.

[0017] The electronic device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method as in the first aspect.

[0018] Third aspect, embodiments of the present application provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method as in the first aspect is implemented.

[0019] Fourth aspect, embodiments of the present application provide a computer program product, which includes a computer program. When the computer program is run, it enables a computer to execute the method as in the first aspect.

[0020] Fifth aspect, embodiments of the present application provide a chip, which includes a processor. The processor is used to call a computer program in a memory to execute the method as described in the first aspect.

[0021] It should be understood that the second aspect to the fifth aspect of the present application correspond to the technical solutions of the first aspect of the present application. The beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar and will not be elaborated herein. Description of the Drawings

[0022] Figure 1 For a pseudo-texture region in an image in a possible implementation;

[0023] Figure 2Schematic diagram of the structure of the electronic device 100 according to an embodiment of the present application;

[0024] Figure 3 Software structure block diagram of the electronic device 100 according to an embodiment of the present application;

[0025] Figure 4 Schematic flow chart of an image processing method provided by an embodiment of the present application;

[0026] Figure 5 Schematic flow chart of detecting a pseudo-texture region provided by an embodiment of the present application;

[0027] Figure 6 Schematic diagram of the prediction result of Image 1 provided by an embodiment of the present application;

[0028] Figure 7 Schematic diagram of the region division result of Image 1 provided by an embodiment of the present application;

[0029] Figure 8 Schematic flow chart of an image fusion provided by an embodiment of the present application;

[0030] Figure 9 Internal interaction flow chart of an image processing method provided by an embodiment of the present application;

[0031] Figure 10 Schematic flow chart of an image processing method provided by an embodiment of the present application;

[0032] Figure 11 Schematic diagram of the structure of an image processing apparatus provided by an embodiment of the present application. Detailed implementation manners

[0033] First, the terms involved in the embodiments of the present application are explained below:

[0034] 1) Pseudo-texture: An electronic device can use, for example, generative adversarial networks (GANs) to process an image to improve the image quality and make the image display clear textures. However, due to the instability of network training, the image after network training will generate rich details and at the same time produce pseudo-textures (artifacts); among them, the pseudo-texture is not the real detail of the object, and the pseudo-texture will affect the user's visual experience.

[0035] 2) Others

[0036] For the convenience of clearly describing the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or more advantageous than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific manner.

[0037] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple.

[0038] It should be noted that "when... " in the embodiments of the present application can be at the instant when a certain situation occurs or within a period of time after a certain situation occurs. The embodiments of the present application do not make specific limitations on this. In addition, the display interface provided in the embodiments of the present application is only an example, and the display interface can also include more or less content.

[0039] The electronic device supports the camera function. The user can use the electronic device to take pictures.

[0040] However, in a possible implementation, when the electronic device takes a picture of the object to be photographed, there may be problems with the photo obtained by the electronic device. For example, there may be some textures in the photo that do not exist on the object to be photographed, which affects the quality of the image.

[0041] For example, take Figure 1 as an example. The user uses the electronic device 100 to take a picture of the Figure 1 scene shown in a of Figure 1 ; among them, the object to be photographed in the scene shown in a of Figure 1 includes a little girl, a tree, and a background wall. The image 101 captured by the electronic device 100 based on the camera application can be as shown in Figure 1 b of Figure 1In the scenario shown in a of [the figure], the texture in Image 101 does not appear on the background wall.

[0042] Among them, the above problem scenario appears in Image 101, possibly because: after the electronic device acquires the original image based on the camera, the electronic device performs super-resolution processing and / or denoising processing on the original image; pseudo-textures may appear in the image processed by the algorithm.

[0043] In view of this, the embodiments of the present application provide an image processing method. In a photographing scenario, the electronic device obtains Image 1 after performing super-resolution and / or denoising processing on the original image. Pseudo-textures may be included in Image 1. The electronic device improves the quality of the photo by eliminating the pseudo-textures in Image 1. The method for eliminating pseudo-textures can be, for example, identifying Image 1 as Region 1 with pseudo-textures and Region 2 without pseudo-textures; the electronic device uses Region 1 without pseudo-textures and Region 1 with pseudo-textures for fusion to reduce the pseudo-textures in Region 1 of Image 1. In this way, the electronic device can obtain an image with a higher clarity in Region 2 and no pseudo-textures in Region 1, improving the image quality and thus enhancing the user's photographing experience.

[0044] In order to better understand the embodiments of the present application, the structure of the electronic device in the embodiments of the present application will be introduced below:

[0045] Figure 2 The structural schematic diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0046] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0047] The electronic device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The display screen 194 is used to display images, videos, etc. In the embodiment of the present application, the electronic device 100 can display the fused image through the display screen.

[0048] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0049] ISP is used to process data fed back by camera 193. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, ISP can be set in camera 193. In the embodiment of the present application, electronic device 100 can obtain shooting parameters such as ISO and exposure time based on ISP.

[0050] The camera 193 is used to capture static images or videos. In the embodiment of the present application, the camera can be used to collect one or more frames of original images.

[0051] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0052] Figure 3 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0053] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system may include: application layer (applications), application framework layer (application framework), hardware abstract layer (HAL) and kernel layer (kernel), where the kernel layer may become a driver layer.

[0054] The application layer may include a series of application packages.

[0055] As Figure 3 shown, the application package may include a camera application.

[0056] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a camera access interface, etc.

[0057] The camera access interface enables the application to manage the camera and access the camera device. For example, manage the camera to take images, etc.

[0058] The hardware abstraction layer may include multiple library modules. The library module may be, for example, an image processing module. The Android system may load the corresponding library module for the device hardware, thereby achieving the purpose of the application framework layer accessing the device hardware. In the embodiments of the present application, the image processing module may include an image model module and an image algorithm module. The image model module includes models for processing images. For example, models for generating high-definition images, models for generating images without pseudo-textures, and models for detecting whether there are pseudo-textures in the image, etc. The image algorithm module includes algorithms for processing images. For example, image gradient algorithms, image variance algorithms, image edge detection algorithms, image fusion algorithms, etc.

[0059] The kernel layer is the layer between the hardware and the software. The kernel layer is used to drive the hardware to make it work. The kernel layer may include a chip driver and a camera driver, etc. The embodiments of the present application do not limit this. For example, in the embodiments of the present application, the camera driver is used to instruct the camera to collect images, and the chip driver is used to send the images to the chip and obtain the images processed by the chip.

[0060] Next, in conjunction with Figure 3 a brief description of the software interaction process of the electronic device in the embodiments of the present application will be given.

[0061] When the camera application of the electronic device receives an operation for taking a picture, it calls the camera access interface. The camera access interface instructs the camera driver to control the camera to collect the original image. After the camera collects the original image, the image is reported to the image processing module through the camera driver, and the original image is transmitted to the chip through the chip driver, so that the chip processes the original image. The image processed by the chip can be a single-frame image, and the chip reports the single-frame image to the image processing module through the chip driver. The image processing module obtains the original image and the single-frame image, and the image processing module uses the image processing method provided in the embodiment of the present application to process the original image and the single-frame image, and obtains an image with high clarity and no pseudo-texture after processing. The image processing module reports the image to the camera application through the camera access interface.

[0062] The following Figure 4 will specifically describe the process of the image processing method provided in the embodiment of the present application. As Figure 4 shown:

[0063] S401. When the electronic device receives a trigger operation for taking a picture, the electronic device collects the original image.

[0064] The trigger operation for taking a picture can be, for example: in a picture-taking scenario, the camera application is running in the foreground of the electronic device, and the electronic device receives a click operation on the shutter button in the camera application interface. In response to the trigger operation for taking a picture, the electronic device obtains the original image collected by the camera; the original image can be an unprocessed original raw image collected by the camera.

[0065] In the embodiment of the present application, the number of original images can be one frame or multiple frames. The multiple frames of original images can be multiple frames of images obtained by the electronic device through exposure when the electronic device receives a single trigger operation for taking a picture; the multiple frames of original images can also include images in the preview stream before and after partial exposure. The electronic device can fuse the multiple frames of original images, and the embodiment of the present application does not limit this.

[0066] S402. The electronic device inputs the original image into a model with high clarity and obtains Image 1.

[0067] The model with high clarity can be used to convert a low-clarity image into a high-clarity image. The model with high clarity can be, for example, a visual geometry group loss (VGG loss) network and a generative adversarial networks (GAN). After the electronic device obtains the original image, it inputs the original image into the model with high clarity and obtains Image 1. Image 1 can be an image in RGB format.

[0068] Among them, Image 1 can be an image obtained by performing multi-frame fusion, denoising processing, and super-resolution reconstruction processing on the original image based on, for example, a VGG loss network and / or a GAN network, etc. Image 1 can be one or more images. In a possible implementation, in step S402, Image 1 may include pseudo-textures, and the pseudo-textures can be, for example, Figure 1 the texture in region 102 in the image shown in b of

[0069] In a possible implementation, the original image can be multi-frame images, and there may be small differences between the multi-frame images. For example, due to unstable lenses, there are slight differences in the positions of objects in consecutive multi-frame original images. The electronic device can register the second frame of the original image to the Nth frame of the original image with the first frame of the original image respectively to obtain multiple original images registered with the first frame of the original image; subsequently, the electronic device inputs the registered original images and the first frame of the original image into a model with high clarity. The electronic device can adjust the noise and clarity of the first frame of the original image with reference to the registered original images to achieve the purpose of reducing noise and improving clarity.

[0070] In the embodiments of the present application, the electronic device can reduce the pseudo-textures in region 102 according to the following steps, as shown in steps S403-S406:

[0071] S403. The electronic device detects the pseudo-texture region in Image 1.

[0072] A pixel pseudo-texture probability prediction network model is set in the electronic device. The pixel pseudo-texture probability prediction network model can be used to obtain the probability value that any pixel point in the image is a pseudo-texture. The electronic device can input Image 1 into the pixel pseudo-texture probability prediction network model to obtain the probability value that any pixel point in Image 1 is a pseudo-texture. When the probability value of the pixel point is greater than the probability threshold, it indicates that the pixel point is a pixel point of a pseudo-texture; the electronic device can obtain multiple pixel points with probability values greater than the probability threshold; the electronic device can determine the region where these pixel points are located as the pseudo-texture region. Correspondingly, when the probability value of the pixel point is less than or equal to the probability threshold, it indicates that the pixel point is not a pixel point of a pseudo-texture; the electronic device can determine that these pixel points are not in the pseudo-texture region.

[0073] It can be understood that the pseudo-texture region can be the region corresponding to the pixel points of the pseudo-texture, and the pseudo-texture region can also be larger than the region corresponding to the pseudo-texture pixel points. For the convenience of operation and subsequent image fusion, the pseudo-texture region can include the pixel points of the pseudo-texture and the pixel points of the normal texture. For example, the pseudo-texture region can be a rectangular region or a region of other shapes. The embodiments of the present application do not limit this.

[0074] It should be noted that the training process and usage process of the pixel pseudo-texture probability prediction network model will be further described in the embodiments of the present application later, and will not be elaborated here.

[0075] S404. The electronic device intercepts Image 2 from the original image according to the position of the pseudo-texture region in Image 1.

[0076] The position of the pseudo-texture region in Image 1 can be, for example Figure 4 In, the pseudo-texture region of Image 1. Image 2 is a part of the original image. The object in Image 2 is the same as the object in the pseudo-texture region of Image 1. The resolution of Image 2 is less than that of Image 1, and Image 2 does not include pseudo-textures. For example, in Figure 4 In, the pseudo-texture region in Image 1 has the same specification as Image 2. It can be understood that Image 1 is obtained by the electronic device after processing the original image. The object in the original image is the same as the object in Image 1, and the original image does not include pseudo-textures. Therefore, the electronic device can obtain Image 2 with low resolution and no pseudo-textures from the original image.

[0077] In the embodiments of the present application, the original image can be one or more. Correspondingly, the number of Image 2 can also be one or more. Any Image 2 can correspond to the original image. The format of Image 2 is the same as that of the original image. For example, Image 2 is an image in raw format.

[0078] In the embodiments of the present application, the electronic device can intercept Image 2 from the original image according to information such as the pixel value, position, and characteristics of the pixel points in the pseudo-texture region. The embodiments of the present application do not limit its specific implementation.

[0079] S405. The electronic device inputs Image 2 into the model without pseudo-textures to obtain Image 3.

[0080] The model without pseudo-textures can be used to convert a low-definition image into a high-definition image, and the converted image does not include pseudo-textures. The model without pseudo-textures can have the ability to convert a low-definition image into a high-definition image. It can be understood that the ability of the model without pseudo-textures to output a high-definition image is weaker than that of a model with high clarity to output a high-definition image. Among them, the clarity of Image 1 is greater than that of Image 3, and the clarity of Image 3 is greater than that of the original image (or Image 2). After obtaining Image 2, the electronic device can input Image 2 into the model without pseudo-textures to obtain Image 3 with improved clarity and no pseudo-textures.

[0081] Image 3 can be an image obtained by processing Image 2 based on a network model, and Image 3 can be one or more images. Among them, Image 3 can be used for fusion with Image 1. Therefore, Image 3 has the same image type as Image 1, and Image 3 can be an RGB format image.

[0082] It can be understood that in the embodiments of the present application, the clarity improved by Image 3 is lower than that improved by Image 1, and the traces after algorithm processing in Image 3 are weaker than those in Image 1. Therefore, the electronic device can achieve that Image 3 does not include pseudo-textures.

[0083] In the embodiments of the present application, Image 2 can be multiple frames of images. Among them, Image 2 can be an image intercepted from multiple original images. The electronic device registers the second frame of Image 2 to the Nth frame of Image 2 with the first frame of Image 2 respectively, and reference can be made to the relevant description in step S402. Image 2 can also be multiple images intercepted from the registered original images and the first frame of the original image, and the embodiments of the present application do not limit this.

[0084] S406. The electronic device performs weighted fusion on the pseudo-texture region in Image 1 and Image 3 to obtain Image 4.

[0085] The electronic device can fuse the pseudo-texture regions of Image 3 and Image 1, and weaken the pseudo-texture effect in Image 1 by adjusting their weights. Image 4 can include the region without pseudo-textures in Image 1 and the region after fusion of the pseudo-texture region in Image 1 and Image 3.

[0086] In some embodiments, the electronic device can fuse the pixel points of Image 3 and the corresponding pixel points of the pseudo-texture region of Image 1; in other embodiments, the electronic device can also fuse the combination of multiple pixel points of Image 3 and the corresponding combination of multiple pixel points of the pseudo-texture region of Image 1. The multiple pixel points can be pixel points of specifications such as 2×2, 3×3, etc.

[0087] In the embodiments of the present application, taking pixel points as an example, the weight of any pixel point in Image 3 is positively correlated with the probability value of the pixel point, the gradient value of the pixel point, and the variance of the pixel point; the weight of any pixel point is also related to the shooting parameters of the electronic device when shooting the original image. For example, the shooting parameters include the international organization for standardization (ISO) and the exposure duration. The weight of any pixel point is positively correlated with the ISO, and the weight of any pixel point is negatively correlated with the exposure duration.

[0088] In the embodiments of the present application, taking the combination of multiple pixel points as an example, the weight of any combination of pixel points in Image 3 can be positively correlated with the probability value of the combination of pixel points, the gradient value of the combination of pixel points, and the variance of the combination of pixel points; the weight of any combination of pixel points is also related to the shooting parameters of the electronic device when shooting the original image. For example, the shooting parameters include the sensitivity and the exposure duration. The weight of any combination of pixel points is positively correlated with the sensitivity, and the weight of any combination of pixel points is negatively correlated with the exposure duration. The embodiments of the present application do not limit the calculation methods of parameters such as the gradient value and the variance value of the combination of multiple pixel points.

[0089] In a possible implementation, the pixel value of the fused pixel point can satisfy the following formula:

[0090] L = α × L 1 + β × L 2

[0091] where L can be the pixel value of the fused pixel point; L 1 can be the pixel value of the pixel point in Image 3; α can be the weight of the pixel point in Image 3; L 2 can be the pixel value of the pixel point in the pseudo-texture area of Image 1; β can be the weight of the pixel point in the pseudo-texture area of Image 1.

[0092] In the embodiments of the present application, α can be related to the probability value, gradient value, variance value, exposure parameters, etc. shown in the above embodiments. β can be related to α, and β can be negatively correlated with α. For example, the sum of β and α is 1, and the sum of β and α is a fixed value. In the embodiments of the present application, the pixel value of the fused pixel point can also satisfy other formulas deformed based on the above formula, and the embodiments of the present application do not limit this.

[0093] It can be understood that the embodiments of the present application illustrate the process of fusing the pseudo-texture area of Image 1 and Image 3 by taking the calculation of the α value through the probability value, gradient value, variance value, exposure parameters, etc. as an example. The embodiments of the present application can also calculate the weight β of the pixel points in the pseudo-texture area of Image 1 through the probability value, gradient value, variance value, exposure parameters, etc., and obtain the α value through the β value. The embodiments of the present application do not limit this.

[0094] In this way, Image 4 includes the area without pseudo-texture in Image 1 and the area after the fusion of the pseudo-texture area in Image 1 and Image 3; the electronic device can retain the high-definition characteristics of the area without pseudo-texture in Image 1, and at the same time weaken or eliminate the pseudo-texture in the pseudo-texture area, so as to obtain a high-quality image with high clarity and no pseudo-texture, improving the user's photo-taking experience.

[0095] The above embodiments briefly introduce the process of the image processing method in the embodiments of the present application. Below, in combination with Figure 5Specific descriptions will be given to the method for detecting pseudo-textures in step 403 and the method for setting weights in image fusion in step S406 in the embodiments of the present application.

[0096] Exemplarily, taking the granularity of pseudo-texture detection as pixel points as an example, the process of pseudo-texture detection can be as Figure 5 shown:

[0097] S501. The electronic device obtains Image 1.

[0098] The process by which the electronic device obtains Image 1 can refer to the relevant descriptions in steps S401 and S402, which will not be elaborated here.

[0099] S502. The electronic device inputs Image 1 into the pixel pseudo-texture probability prediction network model to obtain a prediction result; wherein, the prediction result includes the probability value that any pixel point in Image 1 is a pseudo-texture.

[0100] The prediction result can be, for example, Figure 6 shown. In the Figure 6 shown table, any rectangular area can represent a pixel point, and the value in the rectangular area is the probability value that the pixel point is a pseudo-texture; wherein, the value range of the probability value is between 0 and 1. It can be understood that the larger the probability value, the greater the possibility that the pixel value is in the pseudo-texture area; the smaller the probability value, the smaller the possibility that the pixel value is in the pseudo-texture area. For example, in Figure 6 , the probability values of pixel points in the non-pseudo-texture area can correspond to 0.01, 0.02, 0.03, 0.05, 0.08, etc.; the probability values of pixel points in the pseudo-texture area can correspond to 0.58, 0.67, 0.77, etc.

[0101] S503. The electronic device performs edge detection on Image 1 and divides Image 1 into regions according to the edge detection result.

[0102] The electronic device can divide Image 1 into multiple regions according to the edge detection result. For example, the multiple regions include a flat area, a weak texture area, and a strong texture area. Among them, the texture in the flat area is less than that in the weak texture area; the texture in the weak texture area is less than that in the strong texture area. The embodiments of the present application do not limit the number of region divisions.

[0103] The edge detection result can refer to Figure 7 , in Figure 7In [the figure], Image 1 can be divided into a flat area, a weak texture area, and a strong texture area. Region 701 (e.g., the left diagonal filled area) can be the strong texture area, region 702 (e.g., the horizontal line filled area) can be the weak texture area, and region 703 (e.g., the blank area) can be the flat area. Taking the objects in Image 1 including a little girl, trees, and a background wall as an example, in the region 701 corresponding to the leaves, there are more textures of the leaves. Correspondingly, this region is the strong texture area; in the region 702 corresponding to the tree trunks and the little girl, there are relatively fewer textures. Correspondingly, this region is the weak texture area; in the region 703 corresponding to the background wall, there are fewer textures. Correspondingly, this region is the flat area.

[0104] In a possible implementation, the electronic device can perform edge detection based on the gray value, gradient value, etc. of the pixel points, and set the flat area, weak texture area, and strong texture area according to the change degree of the gray value or the gradient value. For example, when the change of the gray value is small or the gradient value is small, the texture in the area where the pixel point is located is less; when the change of the gray value is large or the gradient value is large, the texture in the area where the pixel point is located is more. The embodiments of the present application do not limit the specific implementation of edge detection, and the embodiments of the present application do not limit the thresholds for dividing the flat area, weak texture area, and strong texture area.

[0105] S504. The electronic device determines the probability value threshold of any pixel point according to the region division result; wherein, the probability value thresholds corresponding to different regions are different.

[0106] The probability value thresholds corresponding to each region are preset in the electronic device. For example, the probability value threshold of the flat area can be less than the probability value threshold of the weak texture area, and the probability value threshold of the weak texture area can be less than the probability value threshold of the strong texture area.

[0107] It can be understood that the probability value threshold of the region division result can affect the position of the pseudo-texture region in Image 1 and the image quality of Image 4. In the flat area, there are relatively fewer textures and the probability value threshold is lower. Subsequently, more pixel points with probability values greater than the probability value threshold are obtained by screening, and the pseudo-texture region and the specification of Image 3 are larger; since the pixel points in Image 3 are located in the flat area and there are fewer textures, the texture information will not be lost too much in the fused Image 4. However, in the strong texture area, there are more textures. If the probability value threshold is low, the pseudo-texture region in Image 1 and the specification of Image 3 will be larger later, and the clarity of Image 3 is less than that of Image 1, which will increase the range of low-clarity regions in the fused Image 4. Therefore, in the embodiments of the present application, the probability value threshold corresponding to the flat area can be set to be relatively low, and the probability value threshold corresponding to the strong texture area can be set to be relatively high. The embodiments of the present application do not limit the specific values of the probability values corresponding to each region.

[0108] S505: The electronic device screens pixel points in a pseudo texture area according to a probability value threshold of the pixel points; wherein, in the pseudo texture area, the probability value of the pixel points is greater than the probability value threshold.

[0109] The electronic device may define an area where pixels having probability values ​​greater than a probability value threshold are located as a pseudo texture area.

[0110] by Figure 6 For example, three pixels with probability values ​​of 0.58, 0.67, and 0.77 are all located in the flat area, and the probability value threshold corresponding to the flat area is, for example, 0.5. The three probability values ​​are all greater than the probability value threshold, and the three pixels are all located in the pseudo texture area. The electronic device can obtain the pseudo texture area including the three pixels in image 1.

[0111] It should be noted that Figure 6 Only some pixels and probability values ​​of pixels are shown as examples. In practical applications, image 1 may correspond to more or fewer pixels, and the pseudo texture area may include more or fewer pixels. This embodiment of the application does not limit this.

[0112] Optionally, in an embodiment of the present application, the electronic device trains a pixel pseudo-texture probability prediction network model in the following manner: Specifically, the electronic device obtains a sample image, wherein the sample image includes a pseudo-texture region, and the electronic device pre-calibrates the pixel points in the pseudo-texture region, and the calibration value can be a probability value of the pixel point being a pseudo-texture. The electronic device uses the sample image to train the model to obtain a pixel pseudo-texture probability prediction network model; the pixel pseudo-texture probability prediction network model has a probability value of a pixel point being a pseudo-texture in the image output according to the image.

[0113] Optionally, the electronic device may use a single frame image to verify and correct steps S502 and S503. Specifically, Figure 5 As shown:

[0114] S701. The electronic device obtains image 1 and a single-frame image.

[0115] The process of the electronic device obtaining the image 1 may refer to the relevant descriptions in step S401 and step S402, which will not be repeated here.

[0116] The single-frame image may be an image obtained by processing the original image, wherein the clarity of the single-frame image is low, for example, the clarity of the single-frame image is lower than that of image 3, and the clarity of image 3 is lower than that of image 1. The single-frame image may be of the same image type as image 1, for example, the single-frame image may be an image in RGB format. In some embodiments, the original image may be a multi-frame image, and the single-frame image may be an image obtained by the electronic device processing the first frame of the original image based on the chip.

[0117] S702. The electronic device inputs Image 1 and a single-frame image into the pixel pseudo-texture probability prediction network model to obtain a prediction result. The prediction result includes the probability value of any pixel point in Image 1 being a pseudo-texture.

[0118] In the embodiment of the present application, during the training process of the pixel pseudo-texture probability prediction network model, the sample single-frame image can be used as reference information to train with the sample image. During the use of the pixel pseudo-texture probability prediction network model, the single-frame image can be used as reference information to assist the electronic device in predicting the pseudo-texture area in Image 1.

[0119] Step S702 can refer to the relevant description in Step S502 and will not be elaborated here.

[0120] S703. The electronic device performs edge detection on Image 1 and divides Image 1 into regions according to the edge detection result of Image 1.

[0121] This step can refer to the relevant description in Step S503 and will not be elaborated here.

[0122] S704. The electronic device performs edge detection on the single-frame image and checks and corrects the region division result of Image 1 according to the edge detection result of the single-frame image.

[0123] In the embodiment of the present application, the object in the single-frame image is the same as the object in Image 1, and the single-frame image does not include pseudo-textures. The electronic device can obtain the region division result of the single-frame image based on the edge detection result of the single-frame image. The electronic device uses the region division result of the single-frame image to check the region division result of Image 1, improving the accuracy of the region division result in Image 1.

[0124] When the region division result of the single-frame image is different from the region division result of Image 1, the electronic device can use the region division result of the single-frame image to correct the region division result of Image 1. For example, the electronic device detects that a certain region in the single-frame image is a strong texture region, and the corresponding region in Image 1 is a weak texture region. The electronic device can correct the corresponding region in Image 1 to a strong texture region, thereby improving the accuracy of the region division result in Image 1.

[0125] S705. The electronic device determines the probability value threshold of any pixel point according to the region division result of Image 1. Different regions correspond to different probability value thresholds.

[0126] S706. The electronic device filters out the pixel points in the pseudo-texture region according to the probability value threshold of the pixel points. In the pseudo-texture region, the probability value of the pixel point is greater than the probability value threshold.

[0127] Steps S705 and S706 can refer to the relevant descriptions in steps S504 and S505, which will not be elaborated here.

[0128] Exemplarily, taking the granularity of image fusion as pixel points as an example, the process of image fusion can be as Figure 8 shown:

[0129] S801. The electronic device obtains the probability value of any pixel point in the pseudo-texture area of Image 1.

[0130] This step can refer to the relevant descriptions in steps S705 and S706, which will not be elaborated here.

[0131] S802. The electronic device calculates the gradient value and variance value of any pixel point in Image 1.

[0132] The electronic device can reflect the texture information in Image 1 through parameters such as the gradient value and variance value. Among them, both the gradient value and variance value can reflect the difference degree of pixel values between adjacent pixel points. For example, the larger the gradient value and variance value, the clearer the edge of the texture; the smaller the gradient value and variance value, the blurrier the edge of the texture.

[0133] The parameters such as the gradient value and variance value in step S802 can also be replaced by other parameters that can reflect texture information. For example, parameters such as the difference in pixel values between pixel points and the contrast between pixel points. In the embodiments of the present application, any one of these parameters or a combination of any multiple of these parameters can be used to measure the weight value of pseudo-texture pixel points. The embodiments of the present application will not list them one by one here.

[0134] S803. The electronic device obtains the shooting parameters.

[0135] The shooting parameters may include ISO and exposure duration. It can be understood that the shooting parameters can affect the brightness of Image 1. When the brightness value of Image 1 is high, the probability of pseudo-texture appearing in Image 1 is low; when the brightness value of Image 1 is low, the probability of pseudo-texture appearing in Image 1 is high. Among them, the smaller the ISO and the longer the exposure duration, the higher the brightness value of Image 1. Therefore, the shooting parameters can also be used as a dimension to measure the weight of pseudo-texture pixel points.

[0136] S804. The electronic device obtains the weight of the pixel point based on the probability value, gradient value, variance value, and shooting parameters of any pixel point.

[0137] Specifically, the weight value of the pixel point in Image 3 is positively correlated with the probability value. It can be understood that when the probability value is larger, the possibility that the pixel point in Image 1 is pseudo-texture is greater. During the image fusion process, the electronic device can increase the weight value of the pixel point in Image 3 so that more information of Image 3 can be retained in Image 4, thereby weakening the pseudo-texture effect of Image 1.

[0138] The weight value of the pixel points in Image 3 is positively correlated with the gradient value and the variance value. It can be understood that when the gradient value and the variance value are larger, the edges of the pseudo-textures in Image 1 are clearer. During the image fusion process, the electronic device can increase the weight value of the pixel points in Image 3 so that more information of Image 3 can be retained in Image 4, thereby blurring the edges of the pseudo-textures in Image 1 and achieving the purpose of weakening the pseudo-texture effect of Image 1.

[0139] The weight value of the pixel points in Image 3 is positively correlated with ISO and negatively correlated with the exposure duration. It can be understood that the weight value of the pixel points in Image 3 is negatively correlated with the brightness value of Image 1; when the brightness value of Image 1 is larger, the probability of pseudo-textures appearing in Image 1 is lower; when the brightness value of Image 1 is smaller, the probability of pseudo-textures appearing in Image 1 is higher; when the probability value is higher, the electronic device can increase the weight value of the pixel points in Image 3. The principle can refer to the relevant description at the probability value, which will not be elaborated here. Further, the brightness value is negatively correlated with ISO and positively correlated with the exposure duration. Therefore, the weight value of the pixel points in Image 3 is positively correlated with ISO and negatively correlated with the exposure duration, which can achieve the purpose of weakening the pseudo-texture effect of Image 1.

[0140] In a possible implementation, for any pixel point in Image 3, the weight value of this pixel point can be obtained by multiplying parameters such as the probability value, the gradient value, the variance value, the reciprocal of the exposure duration, and ISO.

[0141] For example, in Figure 8 , after multi-dimensional weight setting, the weights of the pixel points in Image 3 can be, for example, 0.86, 0.88, 0.92.

[0142] The above embodiments illustrate the influence of five parameters on the weight value of the pixel points in Image 3. In the embodiments of the present application, the weight value can be calculated with more or fewer parameters, and the embodiments of the present application do not limit this.

[0143] Optionally, in step S802, the electronic device can also calculate the gradient value and the variance value of any pixel point in a single-frame image, and verify Image 1 through the single-frame image. The embodiments of the present application do not limit this.

[0144] Next, the internal interaction process of the image processing method in the embodiments of the present application will be described in conjunction with the software architecture and Figure 9 As shown in Figure 9 :

[0145] The electronic device includes an application layer, a hardware abstraction layer, and a driver layer; among them, the application layer includes a camera application; the hardware abstraction layer includes an image processing module, and the image processing module may include an image model module and an image algorithm module. The image model module may include models with high clarity, models without pseudo-textures, and a pixel pseudo-texture probability prediction network model, etc.; the driver layer may include a camera driver and a chip driver.

[0146] The process of the image processing method may include:

[0147] S901. The camera application of the electronic device receives a trigger operation for taking a photo.

[0148] S902. In response to the touch operation, the camera application of the electronic device instructs the camera driver to collect the original image.

[0149] Steps S901 and S902 may refer to the relevant descriptions in step S401, which will not be elaborated here.

[0150] S903. The camera driver of the electronic device obtains the original image, reports the original image to the image model module, and transfers the original image to the chip driver.

[0151] S904. The image model module of the electronic device inputs the original image into the model with high clarity to obtain Image 1.

[0152] Step S904 may refer to the relevant descriptions in step S402, which will not be elaborated here.

[0153] S905. The chip driver of the electronic device obtains a single-frame image and reports the single-frame image to the image model module.

[0154] After the chip driver obtains the original image, it may send the original image to the chip, and the chip calculates the single-frame image; the chip reports the single-frame image to the image model module through the chip driver. Step S905 may refer to the relevant descriptions in step S701, which will not be elaborated here.

[0155] S906. The image model module of the electronic device inputs Image 1 and the single-frame image into the pixel pseudo-texture probability prediction network model to obtain the prediction result of Image 1.

[0156] Step S906 may refer to the relevant descriptions in step S502 and / or S702, which will not be elaborated here.

[0157] S907. The image algorithm module of the electronic device performs edge detection on Image 1 and divides the region of Image 1 according to the edge detection result of Image 1.

[0158] Step S907 may refer to the relevant descriptions in step S503, which will not be elaborated here.

[0159] In S908, the image algorithm module of the electronic device performs edge detection on a single-frame image, and checks and corrects the region division result of Image 1 according to the edge detection result of the single-frame image.

[0160] Step S908 can refer to the relevant description in Step S704, which will not be elaborated here.

[0161] In S909, the image algorithm module of the electronic device determines the probability value threshold of pixel points according to the region division result of Image 1, and obtains the pseudo-texture region in Image 1.

[0162] Step S909 can refer to the relevant description in Step S504, which will not be elaborated here.

[0163] In S910, the image algorithm module of the electronic device intercepts Image 2 from the original image according to the position of the pseudo-texture region in Image 1.

[0164] Step S910 can refer to the relevant description in Step S404, which will not be elaborated here.

[0165] In S911, the image model module of the electronic device inputs Image 2 into the model without pseudo-texture to obtain Image 3.

[0166] Step S911 can refer to the relevant description in Step S405, which will not be elaborated here.

[0167] In S912, the image algorithm module of the electronic device obtains the probability value, gradient value, variance value, ISO, and exposure duration of the pixel points of Image 3, and obtains the weights of the pixel points in Image 3 and the weights of the pixel points in the pseudo-texture region of Image 1.

[0168] Step S912 can refer to the relevant description in Steps S801 - S804, which will not be elaborated here.

[0169] In S913, the image algorithm module of the electronic device performs weight fusion on the pseudo-texture region in Image 1 and Image 3 to obtain Image 4.

[0170] Step S913 can refer to the relevant description in Step S406, which will not be elaborated here.

[0171] In S914, the camera application of the electronic device obtains Image 4.

[0172] The camera application can obtain Image 4, or store Image 4 in the gallery application (also known as the album application). Subsequently, when receiving an operation for viewing Image 4, the electronic device can display Image 4.

[0173] Based on the above embodiments, an embodiment of the present application provides an image processing method. Exemplarily, Figure 10Schematic flowchart of an image processing method provided by an embodiment of the present application.

[0174] As Figure 10 shown, the image processing method may include the following steps:

[0175] S1001. Receive an operation for taking a photo.

[0176] S1002. In response to the operation for taking a photo, obtain an image captured by the camera of the electronic device.

[0177] Steps S1001 - S1002 may refer to the relevant descriptions in step S401 and will not be elaborated here.

[0178] S1003. Process the image captured by the camera to obtain a first image.

[0179] The first image may correspond to Image 1 in the above - mentioned embodiment. Step S1003 may refer to the relevant descriptions in step S402 and will not be elaborated here.

[0180] S1004. Eliminate the pseudo - texture in the first image to obtain a photographed image.

[0181] The photographed image may correspond to Image 4 in the above - mentioned embodiment. Step S1004 may refer to the relevant descriptions in steps S403 - S406 and will not be elaborated here. It can be understood that steps S403 - S406 may be the specific implementation of step S1004, and steps S403 - S406 are used to process the pseudo - texture in the first image to obtain a photographed image.

[0182] In this way, in the shooting scenario, the electronic device can obtain high - quality photos with high clarity and no pseudo - texture, thereby improving the user experience.

[0183] Optionally, eliminating the pseudo - texture in the first image to obtain a photographed image includes: detecting a first area with pseudo - texture in the first image; processing the image captured by the camera to obtain a second image; wherein, the specification of the second image is the same as that of the first area, the clarity of the second image is less than that of the first image, and there is no pseudo - texture in the second image; fusing the first area of the first image with the second image to obtain a photographed image.

[0184] Among them, the first area may correspond to the pseudo - texture area in the above - mentioned embodiment, and the second image may correspond to Image 3 in the above - mentioned embodiment. The size of Image 3 is the same as that of the pseudo - texture area in Image 1, the clarity of Image 3 is less than that of Image 1, and there is no pseudo - texture in Image 3. This step may refer to the relevant descriptions in steps S403 - S406 and will not be elaborated here.

[0185] In this way, in a photo-taking scenario, the electronic device can obtain an image in which there is no false texture in the first area and the clarity of other areas is relatively high, thereby improving the image quality and the user's photo-taking experience.

[0186] Optionally, fusing the first area of the first image with the second image includes: fusing the first pixel area in the first area of the first image with the second pixel area in the second image; the second pixel area is the pixel area corresponding to the first pixel area in the second image; the first pixel area includes one or more pixel points.

[0187] Among them, the image fusion process can refer to the relevant description in step S406. The granularity of image fusion can be a pixel area, and the pixel area can correspond to a single pixel point or a combination of multiple pixel points. When the granularity is a pixel point, the fusion effect of Image 1 and Image 3 is better, and a high-quality Image 4 can be obtained; when the granularity is a combination of multiple pixel points, the computing pressure of the electronic device in the image processing process can be reduced, and the computing efficiency can be improved.

[0188] Optionally, the electronic device fuses the first pixel area and the second pixel area based on the first weight and the second weight; among them, the first pixel area corresponds to the first weight, and the second pixel area corresponds to the second weight; the second weight is related to the probability value that the first pixel area is false texture, the texture information of the first pixel area, and the exposure parameter during photo-taking; the first weight is negatively correlated with the second weight.

[0189] This embodiment can refer to the relevant description in step S406. Among them, the first weight can be the weight value when fusing the pixel points in the false texture area of Image 1. For example, the first weight is β; the second weight can be the weight value when fusing the pixel points of Image 3. For example, the second weight is α. In this way, the electronic device can comprehensively set the second weight and the first weight through multiple dimensions such as the probability value, texture information, and exposure parameter in Image 1, improve the accuracy of the first weight and the second weight, and thus improve the image fusion effect.

[0190] Optionally, the texture information of the first pixel area includes the gradient information and variance information of the first pixel area, and the exposure parameter includes the ISO and exposure time; among them, the second weight is positively correlated with the probability value that the first pixel area is false texture; the second weight is positively correlated with the gradient information of the first pixel area; the second weight is positively correlated with the variance information of the first pixel area; the second weight is positively correlated with the ISO; the second weight is negatively correlated with the exposure time.

[0191] This embodiment can refer to Figure 8The relevant descriptions in the illustrated embodiments are not elaborated herein. It can be understood that only the above-mentioned various factors affecting the weight are shown in the embodiments of the present application. Other texture information, exposure parameters, etc. can also be used in the embodiments of the present application to set the weight. For example, the texture information further includes the pixel value difference, contrast, etc., and the exposure parameters further include the aperture size, etc. The embodiments of the present application do not limit this.

[0192] In this way, the electronic device can comprehensively set the second weight and the first weight through the probability value, texture information, exposure parameters, etc. in Image 1, improve the accuracy of the first weight and the second weight, and thus improve the effect of image fusion.

[0193] Optionally, detecting the first region with pseudo-texture in the first image includes: processing the first image to obtain the probability value that the first pixel region is pseudo-texture; performing edge detection on the first image and dividing the first image into N regions according to the texture density; any one of the N regions is preset with a corresponding probability value threshold, where N is an integer greater than 1; when the first pixel region is located in the i-th region and the probability value that the first pixel region is pseudo-texture is greater than the probability value threshold corresponding to the i-th region, the electronic device obtains the first region; where the first region includes the first pixel region, and i is a positive integer less than or equal to N.

[0194] Among them, the probability value that the first pixel region is pseudo-texture can correspond to the prediction result in the above-mentioned embodiment. For example Figure 6 As shown. In the embodiments of the present application, N is taken as 3, and the N regions include a flat region, a strong texture region, and a weak texture region as an example, but this quantity does not limit the embodiments of the present application. Among them, the texture density of the flat region is less than that of the weak texture region, and the texture density of the weak texture region is less than that of the strong texture region.

[0195] This step can refer to the relevant description in step S503 and will not be elaborated herein.

[0196] In this way, the electronic device can dynamically adjust the probability value threshold according to the amount of texture in different regions, improve the accuracy and precision of the pseudo-texture region, and further improve the effect of subsequent image fusion and the image quality.

[0197] Optionally, the probability value threshold corresponding to the i-th region is positively correlated with the texture density of the i-th region. In this way, the electronic device sets a lower probability value threshold in the flat region and a higher probability value threshold in the strong texture region, improves the accuracy and precision of the pseudo-texture region, and further improves the effect of subsequent image fusion and the image quality.

[0198] Optionally, the electronic device processes the first image based on the first model, and the first model is trained based on the sample images marked with pseudo-texture.

[0199] Among them, the first model may correspond to the pixel pseudo-texture probability prediction network model in the above embodiments.

[0200] In this way, the electronic device can obtain Image 3 with slightly lower sharpness than Image 1 but without pseudo-textures based on the pixel pseudo-texture probability prediction network model.

[0201] Optionally, it further includes: processing the image collected by the camera to obtain a third image; the sharpness of the third image is less than that of the second image; performing edge detection on the third image, and correcting N regions in the first image according to the texture density of the third image.

[0202] Among them, the third image may correspond to the single-frame image in the above embodiments. This step can refer to the relevant descriptions in steps S701 - S706 and will not be elaborated here.

[0203] In this way, although the sharpness of the single-frame image is poor, there are no pseudo-textures. The electronic device can use the single-frame image to calibrate and correct Image 1, thereby improving the accuracy of the region division result and the effect of subsequent image fusion.

[0204] Optionally, processing the image collected by the camera to obtain a second image includes: obtaining a fourth image from the image collected by the camera according to the position of the first region; the specification of the fourth image is the same as that of the first region; processing the fourth image based on the second model to obtain the second image.

[0205] The fourth image may correspond to Image 2 in the above embodiments, and the second model may correspond to the model without pseudo-textures in the above embodiments. This step can refer to the relevant descriptions in steps S404 - S405 and will not be elaborated here.

[0206] In this way, the electronic device can obtain Image 3 without pseudo-textures and with high sharpness. During the image fusion process, the pseudo-texture effect in the pseudo-texture region of Image 1 is weakened through Image 3, thereby improving the image quality.

[0207] Optionally, the image collected by the camera is in RAW format, the first image is in RGB format, the second image is in RGB format, the third image is in RGB format, the fourth image is in RAW format, and the captured image is in RGB format.

[0208] The image processing method of the embodiments of the present application has been described above. Next, the device for executing the above image processing method provided by the embodiments of the present application will be described. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiments of the present application can execute the steps in the above image processing method.

[0209] As Figure 11As shown, the image processing device 1100 can be used in communication devices, circuits, hardware components, or chips. The image processing device includes a display unit 1101 and a processing unit 1102. Among them, the display unit 1101 is used to support the display steps executed by the image processing device 1100; the processing unit 1102 is used to support the information processing steps executed by the image processing device 1100.

[0210] In a possible implementation, the image processing device 1100 may also include a communication unit 1103. Specifically, the communication unit is used to support the steps of data sending and data receiving executed by the image processing device 1100. Among them, the communication unit 1103 may be an input or output interface, a pin, or a circuit, etc.

[0211] In a possible embodiment, the image processing device may further include a storage unit 1104. The processing unit 1102 and the storage unit 1104 are connected by a line. The storage unit 1104 may include one or more memories, and the memory may be a device or a component in one or more devices or circuits for storing programs or data. The storage unit 1104 may exist independently and be connected to the processing unit 1102 of the image processing device through a communication line. The storage unit 1104 may also be integrated with the processing unit 1102.

[0212] The storage unit 1104 may store computer-executable instructions of the method in the terminal device, so that the processing unit 1102 executes the method in the above embodiments. The storage unit 1104 may be a register, a cache, or a RAM, etc. The storage unit 1104 may be integrated with the processing unit 1102. The storage unit 1104 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions. The storage unit 1104 may be independent of the processing unit 1102.

[0213] The image processing method provided by the embodiments of this application can be applied to electronic devices with communication functions. The electronic device includes a terminal device. The specific device form of the terminal device and the like can refer to the above relevant descriptions and will not be elaborated here.

[0214] The embodiments of this application provide an electronic device, which includes a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0215] The embodiments of this application provide a chip. The chip includes a processor, and the processor is used to call a computer program in the memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those of the above relevant embodiments and will not be elaborated here.

[0216] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned method is implemented. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored on a computer-readable medium or transmitted on a computer-readable medium as one or more instructions or codes. The computer-readable medium may include a computer storage medium and a communication medium, and may also include any medium that can transmit a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0217] In a possible implementation, the computer-readable medium may include RAM, ROM, a compact disc read-only memory (CD-ROM), or other optical disc storage, a magnetic disk storage, or other magnetic storage device, or any other medium targeted to carry or store the required program code in the form of instructions or data structures and accessible by a computer. Moreover, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. As used herein, disk and disc include optical disc, laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using a laser. The above combinations should also be included within the scope of the computer-readable medium.

[0218] The embodiments of the present application provide a computer program product. The computer program product includes a computer program. When the computer program is run, the computer is caused to execute the above-mentioned method.

[0219] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable devices to generate a machine, such that the instructions executed by the processing unit of the computer or other programmable data processing device generate a means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0220] In the above specific embodiments, the purpose, technical solution, and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, applied to an electronic device, the method includes: Receiving an operation for taking a photo; In response to the operation for taking a photo, obtaining an image collected by a camera of the electronic device; Inputting the image collected by the camera into a high-definition model to obtain a first image; Detecting a first area with pseudo-texture in the first image; Based on the position of the first area, intercepting a fourth image from the image collected by the camera, and inputting the fourth image into a pseudo-texture-free model to obtain a second image; wherein, the specification of the second image is the same as that of the first area, and there is no pseudo-texture in the second image, and the pseudo-texture-free model is used to convert a low-definition image into a high-definition image; Fusing according to a first weight of a first pixel area in the first area and a second weight of a second pixel area in the second image to obtain a photographed image, the second weight is positively correlated with a probability value that the first pixel area is pseudo-texture, texture information of the first pixel area, and ISO sensitivity, and is negatively correlated with the exposure duration during photographing, the texture information includes gradient information and variance information, and the pseudo-texture probability value is obtained by inputting the first image into a pixel pseudo-texture probability prediction network model.

2. The method according to claim 1, characterized in that, The clarity of the second image is less than that of the first image.

3. The method according to claim 2, characterized in that, The second pixel area is a pixel area corresponding to the first pixel area in the second image; the first pixel area includes one or more pixel points.

4. The method according to any one of claims 1-3, characterized in that, The first weight is negatively correlated with the second weight.

5. The method according to any one of claims 1-3, characterized in that, Detecting a first area with pseudo-texture in the first image includes: Processing the first image to obtain a probability value that the first pixel area is pseudo-texture; Performing edge detection on the first image, and dividing the first image into N areas according to texture density; any one of the N areas is preset with a corresponding probability value threshold, and N is an integer greater than 1; When the first pixel area is in the i-th area and the probability value that the first pixel area is pseudo-texture is greater than the probability value threshold corresponding to the i-th area, the electronic device obtains the first area; wherein, the first area includes the first pixel area, and i is a positive integer less than or equal to N.

6. The method according to claim 5, characterized in that, The probability value threshold corresponding to the i-th area is positively correlated with the texture density of the i-th area.

7. The method according to claim 5, characterized in that, The electronic device processes the first image based on a first model, and the first model is trained based on a sample image marked with pseudo-texture.

8. The method according to claim 6 or 7, characterized in that, Further includes: Process the image collected by the camera to obtain a third image; the clarity of the third image is less than that of the second image; Perform edge detection on the third image, and correct N regions in the first image according to the texture density of the third image.

9. The method according to any one of claims 1-3, 6-7, characterized in that, The image collected by the camera is in RAW format, the first image is in RGB format, the second image is in RGB format, the third image is in RGB format, the fourth image is in RAW format, and the captured image is in RGB format.

10. An electronic device, characterized in that, comprising: a processor and a memory; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1-9.

11. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-9 is implemented.

12. A computer program product, characterized in that, Comprising a computer program, when the computer program is run, the computer is made to execute the method according to any one of claims 1-9.

13. A chip, characterized in that, Comprising a processor, the processor is used to execute a computer program, so that the chip executes the method according to any one of claims 1-9.

Images