Image processing method and device, electronic equipment, storage medium and chip

Abstract

The invention provides an image processing method and device, electronic equipment, a storage medium and a chip. The method comprises the following steps: carrying out filtering processing on an original image for multiple times to obtain a plurality of to-be-processed images with different definitions; obtaining first gain information of each to-be-processed image based on neighborhood information of each pixel point of each to-be-processed image in the plurality of to-be-processed images; acquiring second gain information of each to-be-processed image based on neighborhood information of each pixel point of the original image; based on the first gain information and the second gain information of each to-be-processed image, obtaining edge gain information of each to-be-processed image; based on the edge gain information of each to-be-processed image, performing edge gain processing on each to-be-processed image to obtain a plurality of to-be-processed images after edge enhancement; and fusing the original image and the plurality of to-be-processed images after edge enhancement to obtain a target image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device, storage medium and chip. Background Art

[0002] Image contrast enhancement method is also called contrast stretching method. One of its purposes is to provide maximum detail information at appropriate brightness. Image contrast enhancement method can change the grayscale range occupied by image pixels according to user requirements. The larger the grayscale range, the clearer the image details.

[0003] However, when using the image contrast enhancement method in the related art to process images, problems such as insufficient local contrast and black edges may occur, affecting the image display quality. Summary of the invention

[0004] The present disclosure provides an image processing method, device, electronic device, storage medium and chip to solve the problems in the related art and improve the quality of images.

[0005] A first aspect of the present disclosure provides an image processing method, the method comprising:

[0006] Perform multiple filtering processes on the original image to obtain multiple images to be processed with different clarity;

[0007] Based on the neighborhood information of each pixel point of each image to be processed in the plurality of images to be processed, acquiring first gain information of each image to be processed;

[0008] Based on the neighborhood information of each pixel point of the original image, obtaining the second gain information of each image to be processed;

[0009] Based on the first gain information and the second gain information of each image to be processed, acquiring edge gain information of each image to be processed;

[0010] Based on the edge gain information of each image to be processed, edge gain processing is performed on each image to be processed to obtain a plurality of edge-enhanced images to be processed;

[0011] The original image is fused with the plurality of edge-enhanced images to be processed to obtain a target image.

[0012] In some embodiments of the present disclosure, the first gain information includes contrast gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0013] Based on a preset first neighborhood size, obtaining a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed;

[0014] Acquire contrast information of each pixel point of each image to be processed based on a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed;

[0015] Based on the contrast information of each pixel point of each image to be processed, the contrast gain information of each image to be processed is obtained.

[0016] In some embodiments of the present disclosure, the first gain information includes texture gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0017] Based on a preset second neighborhood size, acquiring brightness gradient information in multiple directions of each pixel point of each image to be processed;

[0018] Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining first brightness gradient information of each pixel point of each image to be processed;

[0019] Based on the first brightness gradient information of each pixel point of each image to be processed, the texture gain information of each image to be processed is acquired.

[0020] In some embodiments of the present disclosure, the acquiring first gradient information of each pixel of each image to be processed based on the brightness gradient information of multiple directions of each pixel of each image to be processed includes:

[0021] Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining the brightness gradient mean value of each pixel point of each image to be processed;

[0022] Normalize the brightness gradient mean of each pixel point of each image to be processed to obtain first brightness gradient information of each pixel point of each image to be processed.

[0023] In some embodiments of the present disclosure, the first gain information includes chrominance gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0024] Based on a preset third neighborhood size, obtaining first chromaticity gradient information of each pixel point of each image to be processed;

[0025] Based on the first chromaticity gradient information of each pixel point of each image to be processed, the chromaticity gain information of each image to be processed is obtained.

[0026] In some embodiments of the present disclosure, the step of acquiring the first chromaticity gradient information of each pixel of each image to be processed based on a preset third neighborhood size includes:

[0027] Based on a preset third neighborhood size and preset chromaticity filter kernels in multiple directions, acquiring chromaticity gradient information in multiple directions of each pixel point of each image to be processed;

[0028] The sum of the chromaticity gradient information in multiple directions of each pixel of each image to be processed is determined as the first chromaticity gradient information of each pixel of each image to be processed.

[0029] In some embodiments of the present disclosure, the acquiring the second gain information of each image to be processed based on the neighborhood information of each pixel point of the original image includes:

[0030] Based on a preset fourth neighborhood size, obtaining third brightness gradient information in multiple directions of each pixel of the original image;

[0031] Determine the strong edge direction of each pixel point of the original image based on third brightness gradient information of each direction in multiple directions of each pixel point of the original image;

[0032] Based on the strong edge direction of each pixel point of the original image, obtaining the strong edge gain information of each image to be processed;

[0033] Based on the strong edge gain information of each image to be processed, second gain information of each image to be processed is acquired.

[0034] In some embodiments of the present disclosure, determining the strong edge direction of each pixel of the original image based on the third brightness gradient information of each direction in multiple directions of each pixel of the original image includes:

[0035] Comparing the third brightness gradient information of each direction in multiple directions of each pixel point of the original image, and determining the edge direction corresponding to the maximum value of the third brightness gradient information;

[0036] An edge direction corresponding to the maximum value of the third brightness gradient information is determined as a strong edge direction of each pixel point of the original image.

[0037] In some embodiments of the present disclosure, the step of acquiring the second gain information of each image to be processed based on the strong edge gain information of each image to be processed includes:

[0038] Determine and compare the strong edge gain information of each image to be processed with the preset maximum strong edge gain information and the preset minimum strong edge gain information;

[0039] If the strong edge gain information of each image to be processed is greater than the preset maximum strong edge gain information, determining the second gain information of each image to be processed to be the preset first strong edge gain information;

[0040] If the strong edge gain information of each to-be-processed image is less than the preset minimum strong edge gain information, determining that the second gain information of each to-be-processed image is the preset second strong edge gain information;

[0041] If the strong edge gain information of each image to be processed is between the preset minimum strong edge gain information and the preset maximum strong edge gain information, the second gain information of each image to be processed is determined based on the strong edge gain information of each image to be processed, the preset minimum strong edge gain information, the preset maximum strong edge gain information and the preset third strong edge gain information.

[0042] In some embodiments of the present disclosure, the image processing method further includes:

[0043] Based on the first scene information of each to-be-processed image, acquiring the first scene gain information of each to-be-processed image;

[0044] The second gain information of each image to be processed is updated by using the product of the first scene gain information of each image to be processed and the second gain information.

[0045] In some embodiments of the present disclosure, the acquiring edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed includes:

[0046] Based on a preset fifth neighborhood size, obtaining second scene gain information of each image to be processed;

[0047] Based on the second scene gain information, the first gain information and the second gain information of each image to be processed, edge gain information of each image to be processed is acquired.

[0048] In some embodiments of the present disclosure, the fusing the original image with the plurality of edge-enhanced images to be processed to obtain a target image includes:

[0049] Based on a plurality of preset rights, weighted processing is performed on the plurality of edge-enhanced images to be processed;

[0050] The original image is fused with the plurality of edge-enhanced images to be processed after weighted processing to obtain a target image.

[0051] A second aspect of the present disclosure provides an image processing device, the device comprising:

[0052] A filtering module is used to perform multiple filtering processes on the original image to obtain multiple images to be processed with different sharpness;

[0053] A first acquisition module, configured to acquire first gain information of each image to be processed based on neighborhood information of each pixel point of each image to be processed among the multiple images to be processed;

[0054] A second acquisition module, used for acquiring second gain information of each to-be-processed image based on neighborhood information of each pixel point of the original image;

[0055] A third acquisition module, configured to acquire edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed;

[0056] An enhancement module, configured to perform edge gain processing on each of the images to be processed based on the edge gain information of each of the images to be processed, and obtain a plurality of edge-enhanced images to be processed;

[0057] The fusion module is used to fuse the original image with the plurality of edge-enhanced images to be processed to obtain a target image.

[0058] A third aspect of the present disclosure provides an electronic device, including:

[0059] at least one processor;

[0060] and a memory communicatively coupled to the at least one processor;

[0061] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the method described in the embodiment of the first aspect of the present disclosure.

[0062] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0063] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect embodiment of the present disclosure.

[0064] In summary, the image processing method, device, electronic device, storage medium and chip proposed in the present disclosure. The method includes: performing multiple filtering processes on the original image to obtain multiple images to be processed with different clarity; obtaining the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed; obtaining the second gain information of each image to be processed based on the neighborhood information of each pixel point of the original image; obtaining the edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed; performing edge gain processing on each image to be processed based on the edge gain information of each image to be processed to obtain multiple edge-enhanced images to be processed; fusing the original image with the multiple edge-enhanced images to be processed to obtain the target image. According to the solution provided by the present disclosure, the multiple images to be processed obtained through the original image can more completely collect the detail information of different frequencies of the original image; at the same time, the first gain information obtained through the image to be processed is superimposed with the second gain information obtained through the original image to obtain the edge gain information of each image to be processed, which can better enhance the strong edges of the image, prevent the occurrence of problems such as black edges caused by over-sharpening, and effectively improve the quality of the target image.

[0065] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0067] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0068] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0069] Figure 3 A flowchart of a method for obtaining contrast gain information provided for an application example of the present disclosure;

[0070] Figure 4 A flowchart of a method for obtaining texture gain information provided for an application example of the present disclosure;

[0071] Figure 5 A schematic diagram of a process for obtaining chromaticity gain information provided for an application example of the present disclosure;

[0072] Figure 6 A flowchart of a method for obtaining second gain information provided for an application example of the present disclosure;

[0073] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0074] Figure 8 A schematic diagram of a target image provided by an embodiment of the present disclosure;

[0075] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0076] Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0077] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0078] Image sharpening method is a method to compensate the contour of the image, enhance the edge of the image and the grayscale jump part, so that the image becomes clear. It is mainly divided into two categories: spatial domain processing and frequency domain processing. Image sharpening processing can highlight the characteristics of the edge and contour of the object in the image, or the characteristics of certain linear target elements, so as to improve the contrast between the edge of the object and the surrounding pixels, thereby improving the clarity of the blurred area in the image. For example, after the image is smoothed, the boundary and contour of the image may become blurred. Through image sharpening technology, the clarity of the boundary and contour of the image can be improved. For another example, for underwater images, image sharpening technology can be used to enhance the edges and contours of the scene in the image. Edges and contours are often located in the place where the grayscale changes suddenly in the image, so it can be intuitively thought of using grayscale differences to extract edges and contours.

[0079] The image contrast enhancement method is also called the image contrast stretching method. One of the purposes of adjusting the image brightness and contrast is to provide the maximum detail information at an appropriate brightness. The deeper the grooves of the detail texture, the clearer the image. However, in the local brightness and contrast enhancement control method of the image in the related art, in the local contrast enhancement part, the brightness information of the image to be processed and the local contrast gain mask are mainly used to differentiate to obtain the local contrast back-added information, and the control gain of the back-added information is obtained by looking up the table based on the brightness information of the image. Among them, the frequency of the above-mentioned back-added information is not controlled, that is, it is impossible to only add high-frequency or medium-frequency information to the image to be processed to perform more refined local contrast enhancement, which will cause the image to be processed to have insufficient local contrast or / and black edges.

[0080] The following two schemes are used to briefly introduce the image contrast enhancement method in the related art.

[0081] Solution A: This solution provides an image sharpening method, including: performing bilateral filtering and Gaussian difference filtering on the original image information to obtain first layer information and second layer information; subtracting the original image information from the first layer information to obtain third layer information; fusing and superimposing the second layer information and the third layer information to obtain fourth layer information; adding the original image information and the fourth layer information to obtain processed image information. This solution combines bilateral filtering and Gaussian difference filtering for image sharpening, which reduces or even eliminates the halo effect at the high-contrast edge of the image and improves the image sharpening effect.

[0082] Solution B: This solution provides an image sharpening method, including: converting the image frame to be sharpened from the RGB color space to the YUV color space, and performing Gaussian low-pass filtering on the Y brightness component to obtain the Y brightness low-pass component, and taking the difference between the Y brightness component and the Y brightness low-pass component to obtain the Y brightness high-frequency component; filtering the Y brightness component according to a preset filtering operator to obtain a Y brightness image background; calculating the image gradient map of the Y brightness component according to a preset gradient operator, and obtaining the image texture map of the Y brightness component according to the image gradient map; calculating the Y brightness minimum perceptible difference image according to the Y brightness image background and the Y brightness image texture map; calculating the gain coefficient based on the Y brightness minimum perceptible difference image and the Y brightness high-frequency component; and calculating the sharpened image according to the gain coefficient. The image sharpening method of the present invention can enhance image details and avoid noise enhancement; the obtained gain coefficient is more in line with the visual characteristics of the human eye.

[0083] The above two solutions have the following defects:

[0084] Solution A: The first layer information and the second layer information in Solution A are obtained by respectively processing the original image information through bilateral filtering and Gaussian difference filtering. Therefore, the first layer information and the second layer information themselves lack detailed information of the image.

[0085] Solution B: Solution B adds back the Y brightness high-frequency component, but does not control the gain of the Y brightness high-frequency component, which may cause a halo effect on the high-contrast edge of the image.

[0086] Furthermore, in both Scheme A and Scheme B, there is no targeted processing in high-contrast strong edge areas and low-contrast weak edge and texture areas. However, the human eye has different requirements for image sharpening and detail enhancement algorithms in high-contrast strong edge areas and low-contrast weak edge and texture areas. In the above two areas, uniform high-frequency signal addition and enhancement will lead to image imbalance. For example, if you want to obtain better texture detail intensity, the strong edges and high-contrast areas will be highly sharpened. If you want to maintain a good sharpening level in strong edges and high-contrast areas, the sharpening level in weak texture areas will be weakened. For example, the edge areas of the tree crowns in the image will appear blurred in the human eye, and the ideal subjective vision of clear and sharp textures cannot be achieved.

[0087] In order to solve at least one of the above-mentioned defects, the present disclosure provides an image processing method, device, electronic device, storage medium and chip.

[0088] Before introducing the detailed solution of the present disclosure, the scenario to which the solution of the present disclosure is applied is described first. Figure 1 FIG. 1 is an application scenario diagram of an image processing method in an embodiment. Figure 1 As shown, the application scenario includes an electronic device 101, in which a camera module and several applications can be installed. The application can initiate an image acquisition instruction to acquire an image, and the camera module acquires an image 102. Among them, the camera module can include a front camera module and / or a rear camera module. Finally, the image 102 is sent to the target application. Among them, the electronic device 101 can be a smart phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0089] In some optional embodiments, the electronic device 101 may also be a vehicle-mounted device or a vehicle networking device, such as a smart car, etc. In the present disclosure, only a smart phone is taken as an example, but it does not limit the scope of the present disclosure.

[0090] A camera may be installed on the electronic device 101, and an image 102 may be acquired through the installed camera. The camera may be divided into laser cameras, visible light cameras, and other types according to the different acquired images 102. The laser camera may acquire the image 102 formed by irradiating the laser onto the object, and the visible light camera may acquire the image formed by irradiating the visible light onto the object. Several cameras may be installed on the electronic device 101, and the installation positions are not limited. For example, a camera may be installed on the front panel of the electronic device 101, and two cameras may be installed on the back panel. The camera may also be installed inside the electronic device 101 in an embedded manner, and then the camera may be opened by rotating or sliding. Specifically, a front camera and a rear camera may be installed on the electronic device 101. The front camera and the rear camera may acquire the image 102 from different perspectives. Generally, the front camera may acquire the image 102 from the front perspective of the electronic device, and the rear camera may acquire the image 102 from the back perspective of the electronic device.

[0091] It should be understood that in the present disclosure, the front camera or the rear camera is only used as an example to distinguish the shooting angles of different cameras, rather than to limit the functions of multiple cameras. The multiple cameras in the present disclosure can be rear cameras or front cameras at the same time, which is not limited in the present disclosure.

[0092] Several applications can be installed in the electronic device 101. An application refers to software written in an electronic device for a certain application purpose. The electronic device 101 can provide services to users through the application. When the application needs to capture the image 102, an image capture instruction will be initiated. The electronic device 101 will call the camera module according to the image capture instruction to capture the image 102. The image capture instruction refers to an instruction used to trigger the image capture operation.

[0093] The electronic device 101 is also provided with a processor, and the image processing module in the processor can reconstruct the image 102 captured by the camera module, for example, execute the image processing method provided in the present disclosure.

[0094] The embodiments of the present disclosure can be applied to scenes that require image processing, for example, image processing in single-frame shooting mode, image processing in multi-frame shooting mode, image processing in preview mode, image processing in video recording mode, image processing in night scene shooting or video mode, etc. For another example, it can also be applied to the ISP module as a key module that affects the processor. These include but are not limited to the following scenes:

[0095] Image processing in single-frame shooting mode, image processing in multi-frame shooting mode, image processing in preview mode, image processing in video recording mode, night scene shooting, image processing in video mode, etc.

[0096] Figure 2 The following is a flow chart of an image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0097] Step 201, performing multiple filtering processes on the original image to obtain multiple images to be processed with different sharpness;

[0098] In one embodiment, the method of performing multiple filtering processes on the original image includes but is not limited to median filtering, low-pass filtering, high-pass filtering, downsampling at different frequencies, etc. For example, the original image can be filtered multiple times using a median filtering method with windows of different sizes to obtain multiple images to be processed with different sharpness. For another example, the original image can be downsampled at sampling frequencies of one-half, one-quarter, etc. to obtain multiple images to be processed with different sharpness. For another example, the original image can be filtered multiple times using median filtering, low-pass filtering, high-pass filtering, and downsampling at different frequencies to obtain multiple images to be processed with different sharpness. The specific method for obtaining multiple images to be processed with different sharpness is not limited in the present disclosure.

[0099] Step 202, based on the neighborhood information of each pixel point of each image to be processed in the plurality of images to be processed, obtaining first gain information of each image to be processed;

[0100] In one embodiment, the first gain information may be one or more of contrast gain information, texture gain information, and chroma gain information, which is not limited in the present disclosure.

[0101] In one embodiment, the method for acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed may be:

[0102] Determine the contrast information of each pixel of each image to be processed in the plurality of images to be processed by using the neighborhood information of each pixel of each image to be processed in the plurality of images to be processed; obtain the first gain information of each image to be processed based on the contrast information of each pixel of each image to be processed in the plurality of images to be processed by using a preset contrast gain lookup table;

[0103] It can also be:

[0104] Determine the brightness gradient information of each pixel of each image to be processed in the plurality of images to be processed by using the neighborhood information of each pixel of each image to be processed in the plurality of images to be processed; obtain the first gain information of each image to be processed based on the brightness gradient information of each pixel of each image to be processed in the plurality of images to be processed by using a preset brightness gradient gain lookup table;

[0105] This disclosure is not limited to this.

[0106] The neighborhood information may be neighborhood information within a neighborhood size of 3*3, 5*5, 7*7 or 9*9 for each pixel, which is not limited in the present disclosure.

[0107] Step 203, obtaining second gain information of each to-be-processed image based on neighborhood information of each pixel point of the original image;

[0108] In one embodiment, obtaining the neighborhood information of each pixel in the original image can obtain more real and comprehensive neighborhood information compared to obtaining the neighborhood information of each pixel in each of the multiple images to be processed. The second gain information obtained based on this can also better reflect the global gain information of the original image.

[0109] In one embodiment, the second gain information refers to gain information related to edge information of the original image, especially gain information related to strong edge information of the original image. The strong edge refers to the edge direction corresponding to the gradient direction with a larger gradient value of the pixel point.

[0110] Step 204, acquiring edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed;

[0111] In one embodiment, the first gain information obtained in step 203 and the second gain information obtained in step 204 are gain information of different aspects. Therefore, the edge gain information obtained from the first gain information and the second gain information can better enhance each image to be processed.

[0112] In one embodiment, the product of the first gain information and the second gain information may be used as the edge gain information, or the weighted sum of the first gain information and the second gain information may be used as the edge gain information, which is not limited in the present disclosure.

[0113] Step 205, performing edge gain processing on each of the images to be processed based on the edge gain information of each of the images to be processed, to obtain a plurality of edge-enhanced images to be processed;

[0114] In one embodiment, the edge gain information of each image to be processed may be added back to each image to be processed to obtain the plurality of edge-enhanced images to be processed.

[0115] In one embodiment, the edge gain information may indicate a gain value of each pixel in the image, and the edge gain information of each image to be processed may be added back to each image to be processed by multiplying the gain value of each pixel in the image to be processed by the pixel value of each pixel, and using the product as the pixel value of the corresponding pixel in the image to be processed after edge enhancement.

[0116] Step 206: Fusing the original image with the plurality of edge-enhanced images to be processed to obtain a target image.

[0117] In one embodiment, image processing software such as OpenCV and MATLAB may be used, or an image fusion algorithm may be used to fuse the original image with the plurality of edge-enhanced images to be processed, which is not limited in the present disclosure.

[0118] In summary, according to the solution provided by the present invention, the multiple images to be processed obtained through the original image can more completely collect the detail information of different frequencies of the original image; at the same time, the first gain information obtained through the image to be processed is superimposed with the second gain information obtained through the original image to obtain the edge gain information of each image to be processed, which can better enhance the strong edges of the image, prevent the occurrence of problems such as black edges caused by excessive sharpening, and effectively improve the quality of the target image.

[0119] Based on the above description, the first gain information may be one or more of contrast gain information, texture gain information, and chroma gain information.

[0120] Based on this, in one embodiment, the first gain information includes at least one of contrast gain information, texture gain information, and chroma gain information. For example, the first gain information is the contrast gain information, or the first gain information includes the contrast gain information, texture gain information, and chroma gain information.

[0121] In one embodiment, if Figure 3 As shown, when the first gain information includes the contrast gain information, acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0122] Step 301, based on a preset first neighborhood size, obtaining a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed;

[0123] In one embodiment, the first neighborhood size may be a 3*3, 5*5, 7*7 or 9*9 neighborhood size of each pixel, which is not limited in the present disclosure.

[0124] Step 302, acquiring contrast information of each pixel point of each image to be processed based on the maximum brightness value and the minimum brightness value in the neighborhood of each pixel point of each image to be processed;

[0125] In one embodiment, the difference between the maximum brightness value and the minimum brightness value in the neighborhood of each pixel point of each image to be processed can be used as the contrast information of each pixel point of each image to be processed; the absolute value of the difference between the maximum brightness value and the minimum brightness value in the neighborhood of each pixel point of each image to be processed can be used as the contrast information of each pixel point of each image to be processed.

[0126] Step 303: Acquire contrast gain information of each image to be processed based on contrast information of each pixel of each image to be processed.

[0127] In one embodiment, the contrast gain information of each image to be processed can be obtained based on the contrast information of each pixel of each image to be processed according to a preset contrast gain lookup table. In the contrast gain lookup table, each contrast information corresponds to a contrast gain, that is, the contrast gain lookup table identifies the mapping relationship between the contrast information and the contrast gain information.

[0128] In one embodiment, the contrast gain information of each image to be processed may also be acquired based on historical experience values ​​and on the contrast information of each pixel of each image to be processed.

[0129] In one embodiment, if Figure 4 As shown, when the first gain information includes the texture gain information, the obtaining of the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0130] Step 401, based on a preset second neighborhood size, obtaining brightness gradient information in multiple directions of each pixel point of each image to be processed;

[0131] In one embodiment, the second neighborhood size may be a 3*3, 5*5, 7*7 or 9*9 neighborhood size of each pixel, which is not limited in the present disclosure.

[0132] In one embodiment, the brightness gradient information of each pixel of each image to be processed in multiple directions is obtained, including obtaining the brightness gradient information of each pixel of each image to be processed in two directions, four directions or eight directions. For example, the brightness gradient information of each pixel of each image to be processed in the upward and downward directions is obtained, or the brightness gradient information of each pixel of each image to be processed in the upward, downward, left and right directions is obtained, or the brightness gradient information of each pixel of each image to be processed in the upward, downward, left, right, upper left, lower left, upper right and lower right directions is obtained.

[0133] In one embodiment, taking the upward brightness gradient information of one pixel of one of the images to be processed as an example, the difference between the brightness values ​​of the pixel and the pixel above it can be used as the upward brightness gradient information of the pixel.

[0134] Step 402, based on the brightness gradient information of each pixel of each image to be processed in multiple directions, obtaining first brightness gradient information of each pixel of each image to be processed;

[0135] In one embodiment, the mean value of the brightness gradient information in multiple directions of each pixel point of each image to be processed can be used as the first brightness gradient information, or the weighted mean value of the brightness gradient information in multiple directions of each pixel point of each image to be processed can be used as the first brightness gradient information, wherein the weight of each direction can be preset.

[0136] Based on this, in one embodiment, the step of acquiring first brightness gradient information of each pixel of each image to be processed based on brightness gradient information of multiple directions of each pixel of each image to be processed includes:

[0137] Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining the brightness gradient mean value of each pixel point of each image to be processed;

[0138] Normalize the brightness gradient mean of each pixel point of each image to be processed to obtain first brightness gradient information of each pixel point of each image to be processed.

[0139] In one embodiment, there are multiple ways to normalize the brightness gradient mean of each pixel point of each image to be processed. The brightness gradient mean of each pixel point of each image to be processed can be normalized relative to the pixel value of the pixel point at the center of its neighborhood, or the brightness gradient mean of each pixel point of each image to be processed can be normalized relative to the sum of the pixel values ​​of all pixels in its neighborhood, or the brightness gradient mean of each pixel point of each image to be processed can be normalized relative to the weighted sum of the pixel values ​​of all pixels in its neighborhood, which is not limited in the present disclosure.

[0140] Step 403: Acquire texture gain information of each image to be processed based on the first brightness gradient information of each pixel point of each image to be processed.

[0141] In one embodiment, the texture gain information of each image to be processed can be obtained based on the first brightness gradient information of each pixel point of each image to be processed according to a preset texture gain lookup table. In the texture gain lookup table, each first brightness gradient information corresponds to a texture gain, that is, the texture gain lookup table identifies the mapping relationship between the first brightness gradient information and the texture gain information.

[0142] In one embodiment, the texture gain information of each image to be processed may also be acquired based on historical experience values ​​and on the first brightness gradient information of each pixel point of each image to be processed.

[0143] In one embodiment, if Figure 5 As shown, when the first gain information includes the chrominance gain information, the obtaining of the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes:

[0144] Step 501, based on a preset third neighborhood size, obtaining first chromaticity gradient information of each pixel point of each image to be processed;

[0145] In one embodiment, the third neighborhood size may be a 3*3, 5*5, 7*7 or 9*9 neighborhood size of each pixel, which is not limited in the present disclosure.

[0146] In one embodiment, obtaining the first chromaticity gradient information of multiple directions of each pixel of each image to be processed includes obtaining the first chromaticity gradient information of two directions or four directions of each pixel of each image to be processed. For example, obtaining the first chromaticity gradient information of the horizontal and vertical directions of each pixel of each image to be processed, or obtaining the first chromaticity gradient information of the horizontal, vertical, main diagonal and sub-diagonal directions of each pixel of each image to be processed.

[0147] In one embodiment, the obtaining, based on a preset third neighborhood size, the first chromaticity gradient information of each pixel of each image to be processed includes:

[0148] Based on a preset third neighborhood size and preset chromaticity filter kernels in multiple directions, acquiring chromaticity gradient information in multiple directions of each pixel point of each image to be processed;

[0149] In one embodiment, the chroma filter kernel in each direction may be convolved with all pixels within the third neighborhood size, and the result of the convolution operation may be used as the chroma gradient information in the corresponding direction, wherein the size of the chroma filter kernel is the same as the size of the third neighborhood.

[0150] The sum of the chromaticity gradient information in multiple directions of each pixel of each image to be processed is determined as the first chromaticity gradient information of each pixel of each image to be processed.

[0151] In one embodiment, the sum of the chromaticity gradient information in multiple directions of each pixel point of each image to be processed refers to the sum of the absolute values ​​of the chromaticity gradient information in multiple directions of each pixel point of each image to be processed.

[0152] Step 502: Acquire chromaticity gain information of each image to be processed based on first chromaticity gradient information of each pixel point of each image to be processed.

[0153] In one embodiment, the chromaticity gain information of each image to be processed can be obtained based on the first chromaticity gradient information of each pixel of each image to be processed according to a preset chromaticity gain lookup table. In the chromaticity gain lookup table, each first chromaticity gradient information corresponds to a chromaticity gain, that is, the chromaticity gain lookup table identifies the mapping relationship between the first chromaticity gradient information and the chromaticity gain information. It should be noted that in the chromaticity gain lookup table, the larger the first chromaticity gradient information, the smaller the chromaticity gain information and all chromaticity gain values ​​are less than 1. That is to say, at the edge position where the chromaticity gradient is larger, the effect of its gain is attenuation, avoiding the impact of image sharpening on the edge color of the image and erroneous enhancement. In the chromaticity gain lookup table, if there is a situation where the first chromaticity gradient information does not have a corresponding chromaticity gain, the chromaticity gain information can be obtained by interpolation.

[0154] In one embodiment, since the chromaticity value of each pixel of the image to be processed exists in the YUV image space, before step 501, it also includes judging whether each image to be processed is a YUV image, and if not, converting each image to be processed into a YUV image.

[0155] In one embodiment, in order to avoid erroneous enhancement of strong edges of the image due to sharpening, second gain information applicable to the strong edges of the image needs to be determined.

[0156] Based on this, in one embodiment, if Figure 6 As shown, the obtaining of the second gain information of each image to be processed based on the neighborhood information of each pixel point of the original image includes:

[0157] Step 601, based on a preset fourth neighborhood size, obtaining third brightness gradient information in multiple directions of each pixel of the original image;

[0158] In one embodiment, the third brightness gradient information of multiple directions of each pixel of the original image may be small-range third brightness gradient information or large-range third brightness gradient information. The small-range third brightness gradient information refers to the third brightness gradient information determined by using the adjacent pixels of the current pixel; the large-range third brightness gradient information refers to the third brightness gradient information determined by using the interval pixels of the current pixel. The interval pixels may be one pixel or multiple pixels, which is not limited in the present disclosure.

[0159] In one embodiment, since the small-range third brightness gradient information is determined based on adjacent pixels, the front edges of some pixels may be missed, while the large-range third brightness gradient information can cover a larger part, and since the large-range third brightness gradient information includes multiple first-order difference information in a uniform gradient direction, it has better noise detection performance.

[0160] In one embodiment, obtaining the third brightness gradient information of multiple directions for each pixel of each image to be processed includes obtaining the third brightness gradient information of two directions or four directions for each pixel of each image to be processed. For example, obtaining the third brightness gradient information of the horizontal and vertical directions for each pixel of each image to be processed, or obtaining the third brightness gradient information of the horizontal, vertical, main diagonal and sub-diagonal directions for each pixel of each image to be processed.

[0161] Step 602, determining a strong edge direction of each pixel point of the original image based on third brightness gradient information of each direction in multiple directions of each pixel point of the original image;

[0162] In one embodiment, determining the strong edge direction of each pixel of the original image based on the third brightness gradient information of each direction in multiple directions of each pixel of the original image may include:

[0163] Comparing the third brightness gradient information of each direction in multiple directions of each pixel point of the original image, and determining the edge direction corresponding to the maximum value of the third brightness gradient information;

[0164] An edge direction corresponding to the maximum value of the third brightness gradient information is determined as a strong edge direction of each pixel point of the original image.

[0165] In one embodiment, determining the strong edge direction of each pixel of the original image based on the third brightness gradient information of each direction in multiple directions of each pixel of the original image may also include:

[0166] The third brightness gradient information of each pixel point of the original image in the first direction is compared with the magnification value of the third brightness gradient information in the second direction, and if the third brightness gradient information in the first direction is greater than the magnification value of the third brightness gradient information in the second direction, the first direction is determined to be the strong edge direction of each pixel point of the original image. The magnification indicated by the magnification value is a preset value, such as 1024.

[0167] Step 603, based on the strong edge direction of each pixel point of the original image, obtaining the strong edge gain information of each image to be processed;

[0168] In one embodiment, the strong edge gain information of each image to be processed may be a preset value, such as 1024. It may also be a ratio of the magnification value of the third brightness gradient information in the first direction to the third brightness gradient information in the second direction. This is not limited in the present disclosure.

[0169] Step 604: Based on the strong edge gain information of each image to be processed, obtain second gain information of each image to be processed.

[0170] In one embodiment, the step of acquiring the second gain information of each image to be processed based on the strong edge gain information of each image to be processed includes:

[0171] Determine and compare the strong edge gain information of each image to be processed with the preset maximum strong edge gain information and the preset minimum strong edge gain information;

[0172] If the strong edge gain information of each image to be processed is greater than the preset maximum strong edge gain information, determining the second gain information of each image to be processed to be the preset first strong edge gain information;

[0173] If the strong edge gain information of each to-be-processed image is less than the preset minimum strong edge gain information, determining that the second gain information of each to-be-processed image is the preset second strong edge gain information;

[0174] If the strong edge gain information of each image to be processed is between the preset minimum strong edge gain information and the preset maximum strong edge gain information, the second gain information of each image to be processed is determined based on the strong edge gain information of each image to be processed, the preset minimum strong edge gain information, the preset maximum strong edge gain information and the preset third strong edge gain information.

[0175] In one embodiment, the preset first strong edge gain information is 768, the preset second strong edge gain information is 1024, the preset third strong edge gain information is 4, the preset maximum strong edge gain information is 5120, and the preset minimum strong edge gain information is 1024. Of course, other values ​​may also be used, which is not limited in the present disclosure.

[0176] In one embodiment, the method for determining the second gain information of each image to be processed based on the strong edge gain information of each image to be processed, the preset minimum strong edge gain information, the preset maximum strong edge gain information and the preset third strong edge gain information is:

[0177] The second gain information of each image to be processed is equal to the sum of the third gain information and the fourth gain information. The third gain information refers to the ratio of the difference between the strong edge gain information of each image to be processed and the preset minimum strong edge gain information to the preset third strong edge gain information. The fourth gain information is equal to the difference between the preset maximum strong edge gain information and the strong edge gain information of each image to be processed and the ratio of the preset third strong edge gain information.

[0178] In one embodiment, the multiple images to be processed obtained from the original image may have different sharpness, resulting in different scenes. For example, for an original image containing a large number of tree crowns, images with different sharpness indicate different scenes.

[0179] Based on this, in one embodiment, the method further includes:

[0180] Step 605, based on the first scene information of each image to be processed, obtaining the first scene gain information of each image to be processed;

[0181] In one embodiment, a first scene gain lookup table may be preset, where the first scene gain lookup table identifies a mapping relationship between the first scene information and the first scene gain.

[0182] Step 606: Update the second gain information of each image to be processed by using the product of the first scene gain information of each image to be processed and the second gain information.

[0183] In one embodiment, the acquiring edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed includes:

[0184] Based on a preset fifth neighborhood size, obtaining second scene gain information of each image to be processed;

[0185] Based on the second scene gain information, the first gain information and the second gain information of each image to be processed, edge gain information of each image to be processed is acquired.

[0186] In one embodiment, the edge gain information of each to-be-processed image is the product of the second scene gain information, the first gain information and the second gain information.

[0187] In one embodiment, the fusing the original image with the plurality of edge-enhanced images to be processed to obtain a target image includes:

[0188] Based on a plurality of preset rights, weighted processing is performed on the plurality of edge-enhanced images to be processed;

[0189] The original image is fused with the plurality of edge-enhanced images to be processed after weighted processing to obtain a target image.

[0190] In one embodiment, image processing software such as OpenCV and MATLAB may be used, or an image fusion algorithm may be used to fuse the original image with the plurality of edge-enhanced images to be processed, which is not limited in the present disclosure.

[0191] like Figure 7 As shown, the embodiment of the present disclosure further provides an image processing device, the image processing device 700 comprising:

[0192] The filtering module 701 is used to perform multiple filtering processes on the original image to obtain multiple images to be processed with different sharpness;

[0193] A first acquisition module 702, configured to acquire first gain information of each image to be processed based on neighborhood information of each pixel point of each image to be processed among the multiple images to be processed;

[0194] A second acquisition module 703, configured to acquire second gain information of each to-be-processed image based on neighborhood information of each pixel point of the original image;

[0195] A third acquisition module 704 is used to acquire edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed;

[0196] An enhancement module 705 is used to perform edge gain processing on each of the images to be processed based on the edge gain information of each of the images to be processed, so as to obtain a plurality of edge-enhanced images to be processed;

[0197] The fusion module 706 is used to fuse the original image with the plurality of edge-enhanced images to be processed to obtain a target image.

[0198] In one embodiment, the first acquisition module 702 is specifically configured to:

[0199] Based on a preset first neighborhood size, obtaining a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed;

[0200] Acquire contrast information of each pixel point of each image to be processed based on a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed;

[0201] Based on the contrast information of each pixel point of each image to be processed, the contrast gain information of each image to be processed is obtained.

[0202] In one embodiment, the first acquisition module 702 is further specifically configured to:

[0203] Based on a preset second neighborhood size, acquiring brightness gradient information in multiple directions of each pixel point of each image to be processed;

[0204] Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining first brightness gradient information of each pixel point of each image to be processed;

[0205] Based on the first brightness gradient information of each pixel point of each image to be processed, the texture gain information of each image to be processed is acquired.

[0206] In one embodiment, the first acquisition module 702 is specifically configured to:

[0207] Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining the brightness gradient mean value of each pixel point of each image to be processed;

[0208] Normalize the brightness gradient mean of each pixel point of each image to be processed to obtain first brightness gradient information of each pixel point of each image to be processed.

[0209] In one embodiment, the first acquisition module 702 is specifically configured to:

[0210] Based on a preset third neighborhood size, obtaining first chromaticity gradient information of each pixel point of each image to be processed;

[0211] Based on the first chromaticity gradient information of each pixel point of each image to be processed, the chromaticity gain information of each image to be processed is obtained.

[0212] In one embodiment, the first acquisition module 702 is specifically configured to:

[0213] Based on a preset third neighborhood size and preset chromaticity filter kernels in multiple directions, acquiring chromaticity gradient information in multiple directions of each pixel point of each image to be processed;

[0214] The sum of the chromaticity gradient information in multiple directions of each pixel of each image to be processed is determined as the first chromaticity gradient information of each pixel of each image to be processed.

[0215] In one embodiment, the second acquisition module 703 is specifically used to:

[0216] Based on a preset fourth neighborhood size, obtaining third brightness gradient information in multiple directions of each pixel of the original image;

[0217] Determine the strong edge direction of each pixel point of the original image based on third brightness gradient information of each direction in multiple directions of each pixel point of the original image;

[0218] Based on the strong edge direction of each pixel point of the original image, obtaining the strong edge gain information of each image to be processed;

[0219] Based on the strong edge gain information of each image to be processed, second gain information of each image to be processed is acquired.

[0220] In one embodiment, the second acquisition module 703 is specifically used to:

[0221] Comparing the third brightness gradient information of each direction in multiple directions of each pixel point of the original image, and determining the edge direction corresponding to the maximum value of the third brightness gradient information;

[0222] An edge direction corresponding to the maximum value of the third brightness gradient information is determined as a strong edge direction of each pixel point of the original image.

[0223] In one embodiment, the second acquisition module 703 is specifically used to:

[0224] Determine and compare the strong edge gain information of each image to be processed with the preset maximum strong edge gain information and the preset minimum strong edge gain information;

[0225] If the strong edge gain information of each image to be processed is greater than the preset maximum strong edge gain information, determining the second gain information of each image to be processed to be the preset first strong edge gain information;

[0226] If the strong edge gain information of each to-be-processed image is less than the preset minimum strong edge gain information, determining that the second gain information of each to-be-processed image is the preset second strong edge gain information;

[0227] If the strong edge gain information of each image to be processed is between the preset minimum strong edge gain information and the preset maximum strong edge gain information, the second gain information of each image to be processed is determined based on the strong edge gain information of each image to be processed, the preset minimum strong edge gain information, the preset maximum strong edge gain information and the preset third strong edge gain information.

[0228] In one embodiment, the second acquisition module 703 is specifically used to:

[0229] Based on the first scene information of each to-be-processed image, acquiring the first scene gain information of each to-be-processed image;

[0230] The second gain information of each image to be processed is updated by using the product of the first scene gain information of each image to be processed and the second gain information.

[0231] In one embodiment, the third acquisition module 704 is specifically configured to:

[0232] Based on a preset fifth neighborhood size, obtaining second scene gain information of each image to be processed;

[0233] Based on the second scene gain information, the first gain information and the second gain information of each image to be processed, edge gain information of each image to be processed is acquired.

[0234] In one embodiment, the fusion module 706 is specifically used to:

[0235] Based on a plurality of preset rights, weighted processing is performed on the plurality of edge-enhanced images to be processed;

[0236] The original image is fused with the plurality of edge-enhanced images to be processed after weighted processing to obtain a target image.

[0237] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0238] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0239] like Figure 8 As shown, Figure 8 The middle right picture is a target image obtained by using the image processing method provided by the embodiment of the present disclosure, and the left picture is a target image obtained by using the image processing method provided by the related art. From the comparison between the left and right pictures, it can be seen that the image processing method provided by the embodiment of the present disclosure has significantly improved the black edge problem at the edge of the image, and the image quality is higher.

[0240] Fig. 91 is a block diagram of an electronic device 900 for implementing the above-mentioned image processing method according to an exemplary embodiment. For example, the electronic device 900 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0241] Reference Fig. 9 , the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0242] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0243] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0244] The power supply component 906 provides power to the various components of the electronic device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 900.

[0245] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0246] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0247] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0248] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900, and the sensor assembly 914 can also detect the position change of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and the temperature change of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0249] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0250] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0251] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the electronic device 900 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0252] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image processing method described in the above embodiment of the present disclosure.

[0253] The embodiment of the present disclosure also provides a chip, which can be seen in Fig.10 Schematic diagram of the chip structure shown. Fig.10 The chip shown includes a processor 1001 and an interface 1002. The number of processors 1001 may be one or more, and the number of interfaces 1002 may be multiple. The interface 1002 is used to receive a signal from a memory of an electronic device and send the signal to the processor 1001, wherein the signal includes a computer instruction stored in the memory.

[0254] Optionally, the chip further includes a memory 1003, and the memory 1003 is used to store necessary computer instructions and data. When the processor 1001 executes the computer instructions, the electronic device executes the image processing method described in the above embodiment of the present disclosure.

[0255] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0256] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0257] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0258] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0259] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0260] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0261] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0262] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Perform multiple filtering processes on the original image to obtain multiple images to be processed with different clarity; Based on the neighborhood information of each pixel point of each image to be processed in the plurality of images to be processed, acquiring first gain information of each image to be processed; Based on the neighborhood information of each pixel point of the original image, obtaining the second gain information of each image to be processed; Based on the first gain information and the second gain information of each image to be processed, acquiring edge gain information of each image to be processed; Based on the edge gain information of each image to be processed, edge gain processing is performed on each image to be processed to obtain a plurality of edge-enhanced images to be processed; The original image is fused with the plurality of edge-enhanced images to be processed to obtain a target image.

2. The method according to claim 1, characterized in that The first gain information includes contrast gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes: Based on a preset first neighborhood size, obtaining a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed; Acquire contrast information of each pixel point of each image to be processed based on a maximum brightness value and a minimum brightness value in a neighborhood of each pixel point of each image to be processed; Based on the contrast information of each pixel point of each image to be processed, the contrast gain information of each image to be processed is obtained.

3. The method according to claim 1, characterized in that The first gain information includes texture gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes: Based on a preset second neighborhood size, acquiring brightness gradient information in multiple directions of each pixel point of each image to be processed; Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining first brightness gradient information of each pixel point of each image to be processed; Based on the first brightness gradient information of each pixel point of each image to be processed, the texture gain information of each image to be processed is obtained.

4. The method according to claim 3, characterized in that The acquiring first gradient information of each pixel point of each image to be processed based on the brightness gradient information of multiple directions of each pixel point of each image to be processed comprises: Based on the brightness gradient information of each pixel point of each image to be processed in multiple directions, obtaining the brightness gradient mean value of each pixel point of each image to be processed; Normalize the brightness gradient mean of each pixel point of each image to be processed to obtain first brightness gradient information of each pixel point of each image to be processed.

5. The method according to claim 1, characterized in that The first gain information includes chroma gain information, and acquiring the first gain information of each image to be processed based on the neighborhood information of each pixel point of each image to be processed in the multiple images to be processed includes: Based on a preset third neighborhood size, obtaining first chromaticity gradient information of each pixel point of each image to be processed; Based on the first chromaticity gradient information of each pixel point of each image to be processed, the chromaticity gain information of each image to be processed is obtained.

6. The method according to claim 5, characterized in that The obtaining, based on a preset third neighborhood size, first chromaticity gradient information of each pixel point of each image to be processed includes: Based on a preset third neighborhood size and preset chromaticity filter kernels in multiple directions, acquiring chromaticity gradient information in multiple directions of each pixel point of each image to be processed; The sum of the chromaticity gradient information in multiple directions of each pixel of each image to be processed is determined as the first chromaticity gradient information of each pixel of each image to be processed.

7. The method according to claim 1, characterized in that The acquiring the second gain information of each to-be-processed image based on the neighborhood information of each pixel point of the original image comprises: Based on a preset fourth neighborhood size, obtaining third brightness gradient information in multiple directions of each pixel of the original image; Determine the strong edge direction of each pixel point of the original image based on third brightness gradient information of each direction in multiple directions of each pixel point of the original image; Based on the strong edge direction of each pixel point of the original image, obtaining the strong edge gain information of each image to be processed; Based on the strong edge gain information of each image to be processed, second gain information of each image to be processed is acquired.

8. The method according to claim 7, characterized in that The determining, based on the third brightness gradient information of each direction of multiple directions of each pixel of the original image, the strong edge direction of each pixel of the original image comprises: Comparing the third brightness gradient information of each direction in multiple directions of each pixel point of the original image, and determining the edge direction corresponding to the maximum value of the third brightness gradient information; An edge direction corresponding to the maximum value of the third brightness gradient information is determined as a strong edge direction of each pixel point of the original image.

9. The method according to claim 7, characterized in that: The step of acquiring the second gain information of each image to be processed based on the strong edge gain information of each image to be processed comprises: Determine and compare the strong edge gain information of each image to be processed with the preset maximum strong edge gain information and the preset minimum strong edge gain information; If the strong edge gain information of each image to be processed is greater than the preset maximum strong edge gain information, determining the second gain information of each image to be processed to be the preset first strong edge gain information; If the strong edge gain information of each to-be-processed image is less than the preset minimum strong edge gain information, determining that the second gain information of each to-be-processed image is the preset second strong edge gain information; If the strong edge gain information of each image to be processed is between the preset minimum strong edge gain information and the preset maximum strong edge gain information, the second gain information of each image to be processed is determined based on the strong edge gain information of each image to be processed, the preset minimum strong edge gain information, the preset maximum strong edge gain information and the preset third strong edge gain information.

10. The method according to claim 9, characterized in that The method further comprises: Based on the first scene information of each to-be-processed image, acquiring the first scene gain information of each to-be-processed image; The second gain information of each image to be processed is updated by using the product of the first scene gain information of each image to be processed and the second gain information.

11. The method according to claim 1, characterized in that: The acquiring edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed includes: Based on a preset fifth neighborhood size, obtaining second scene gain information of each image to be processed; Based on the second scene gain information, the first gain information and the second gain information of each image to be processed, edge gain information of each image to be processed is acquired.

12. The method according to claim 1, characterized in that The fusing the original image with the plurality of edge-enhanced images to be processed to obtain a target image includes: Based on a plurality of preset rights, weighted processing is performed on the plurality of edge-enhanced images to be processed; The original image is fused with the plurality of edge-enhanced images to be processed after weighted processing to obtain a target image.

13. An image processing device, characterized in that: include: A filtering module is used to perform multiple filtering processes on the original image to obtain multiple images to be processed with different sharpness; A first acquisition module, configured to acquire first gain information of each image to be processed based on neighborhood information of each pixel point of each image to be processed among the multiple images to be processed; A second acquisition module, used for acquiring second gain information of each to-be-processed image based on neighborhood information of each pixel point of the original image; A third acquisition module, configured to acquire edge gain information of each image to be processed based on the first gain information and the second gain information of each image to be processed; An enhancement module, configured to perform edge gain processing on each of the images to be processed based on the edge gain information of each of the images to be processed, and obtain a plurality of edge-enhanced images to be processed; The fusion module is used to fuse the original image with the plurality of edge-enhanced images to be processed to obtain a target image.

14. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

16. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal includes a computer instruction stored in the memory, and when the processor executes the computer instruction, the electronic device executes the method described in any one of claims 1 to 12.

Images