Image processing method, device, electronic device and storage medium

Abstract

This application discloses an image processing method, apparatus, electronic device, and storage medium, belonging to the field of image processing technology. The method includes: performing multi-scale downsampling on an image to be processed using a first filter kernel and a second filter kernel to obtain a first image sequence and a second image sequence; determining a first weight map corresponding to each of the preset grayscale slice values ​​based on M preset grayscale slice values ​​and the grayscale values ​​of each pixel in the image to be processed; performing coefficient processing on each of the first weight maps based on image processing coefficients to obtain M second weight maps, wherein the image processing coefficients are noise removal coefficients or local enhancement coefficients, and M is a positive integer; and obtaining a target image based on each of the second weight maps, the first image sequence, and the second image sequence.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of electronic devices, electronic devices have begun to be integrated into people's daily lives, and more and more users have begun to use electronic devices to capture images.

[0003] In the process of shooting with electronic devices, the final image effect may be poor due to the influence of hardware or external environment. The existing technology can optimize the images taken by electronic devices through noise removal or local image enhancement. However, in the existing technology, it is impossible to simultaneously use a set of image processing algorithms to be compatible with these two image optimization solutions, which results in a large algorithm program size and high power consumption.

[0004] Therefore, how to use a set of image processing algorithms to simultaneously be compatible with both noise removal and local image enhancement processing, two image optimization solutions, has become a problem that needs to be solved urgently in the industry. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device and storage medium, which can solve the problem of simultaneously being compatible with two image optimization solutions, noise removal and image local enhancement processing, through a set of image processing algorithms.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0007] The image to be processed is respectively subjected to multi-scale downsampling by using a first filter kernel and a second filter kernel to obtain a first image sequence and a second image sequence;

[0008] Based on M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed, determining a first weight map corresponding to each preset grayscale slice value;

[0009] Based on an image processing coefficient, coefficient processing is performed on each of the first weight maps to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer;

[0010] A target image is obtained based on the second weight maps, the first image sequence and the second image sequence.

[0011] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0012] A downsampling module, used to perform multi-scale downsampling on the image to be processed by using a first filter kernel and a second filter kernel, respectively, to obtain a first image sequence and a second image sequence;

[0013] A first determination module, configured to determine a first weight map corresponding to each of the preset grayscale slice values ​​based on the M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed;

[0014] A first processing module, configured to perform coefficient processing on each of the first weight maps based on an image processing coefficient to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer;

[0015] The second processing module is used to obtain a target image based on each of the second weight maps, the first image sequence and the second image sequence.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0019] In an embodiment of the present application, in a set of image processing methods, two different image processing coefficients are introduced, namely, a noise removal coefficient or a local enhancement coefficient, so as to respectively support image optimization processing schemes for performing local enhancement processing and image noise removal processing on the image. In the embodiment of the present application, on the basis of effectively supporting local enhancement optimization and image noise removal optimization, it is possible to effectively save program volume and effectively reduce the area of ​​a chip carrying the image processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the image processing method provided in the embodiment of the present application;

[0021] Figure 2 An image processing device provided in an embodiment of the present application;

[0022] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0026] In the following, in conjunction with the accompanying drawings, an image processing method, device, electronic device and storage medium provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0027] Figure 1 A schematic diagram of the image processing method provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, including:

[0028] Step 110, performing multi-scale downsampling on the image to be processed by using a first filter kernel and a second filter kernel, respectively, to obtain a first image sequence and a second image sequence;

[0029] Specifically, the image to be processed described in the embodiment of the present application can be obtained by fusing multiple frames of input with different exposure times to obtain the image to be processed or a frame image in the video stream, and after performing global tone mapping on the fused frame image, the image to be processed can be obtained.

[0030] Specifically, in an embodiment of the present application, for a specific frame of image, since different types of image sensors support different types of high dynamic range image (HDR) data acquisition methods, 1 to N frames of data may be received in an embodiment of the present application. For these data, basic brightness alignment is performed according to different exposure amounts, and then fusion is performed, that is, according to the exposure difference between N frames, each pixel is adjusted to the same exposure level. At this moment, the exposure reference frame is the frame with the shortest exposure time. After brightness alignment, multiple frames are fused pixel by pixel to obtain the image to be processed or a frame of image in the video stream.

[0031] In an embodiment of the present application, after obtaining an image after multi-frame fusion, global tone mapping is performed on it. When the image is a high-bitwidth HDR fused image, it is mapped by a Bezier curve and then manually adjusted to obtain an ideal global tone mapping result, that is, the image to be processed.

[0032] When the multi-frame fused image is a regular bit width HDR fused image, a reasonable mapping curve is generated according to the local exposure statistics and the global histogram distribution information, and global tone mapping is performed to obtain the global tone mapping result, that is, the image to be processed.

[0033] The first filter kernel and the second filter kernel described in the embodiment of the present application are different types of filter kernels. Correspondingly, after processing the images to be processed respectively, two different groups of image data groups can be obtained, namely, a first image sequence composed of N first images arranged in sequence according to image size and a second image sequence composed of N second images arranged in sequence according to image size.

[0034] In the embodiment of the present application, the number of first images and the number of second images generated are the same, and the first images and the second images may be arranged in sequence according to the image size, that is, in the shape of an image arrangement pyramid.

[0035] Step 120, based on M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed, determining a first weight map corresponding to each preset grayscale slice value;

[0036] In the embodiment of the present application, the M preset grayscale slice values ​​described in the embodiment of the present application can be pre-set, and the interval step between each grayscale slice value is the same. The number of preset grayscale slice values ​​M=1 / step. For example, when the interval step between each grayscale slice value is 0.1, then correspondingly, M=1 / 0.1=10, and the grayscale slice value=(0.1, 0.2, 0.3…1).

[0037] The image to be processed described in the embodiment of the present application is composed of individual pixel points. Therefore, for each pixel point in the image to be processed, the difference between the grayscale value of the pixel point and the currently selected grayscale slice value is calculated, and then the variance f (image to be processed input-grayscale slice value) is calculated based on the difference, and the weight map JOJO corresponding to the grayscale slice value is obtained based on the variance information.

[0038] In the embodiment of the present application, there are different grayscale slice values. Therefore, according to the above steps, the weight map JOJO corresponding to each grayscale slice value can be determined. For example, if there are 10 grayscale slice values, there are ten different weight maps JOJO.

[0039] A first weight map corresponding to each of the preset grayscale slice values ​​is generated according to the variance corresponding to each pixel point.

[0040] Step 130, performing coefficient processing on each of the first weight maps based on an image processing coefficient to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer;

[0041] Specifically, the image processing coefficients described in the embodiments of the present application are used to achieve different image adjustment effects, the noise removal coefficients are used to achieve noise removal effects on images, and the local enhancement coefficients are used to achieve local enhancement processing on images.

[0042] In the embodiment of the present application, only one type of image processing coefficient can be selected during each optimization process, that is, the image processing method described in the embodiment of the present application can only support noise removal effect or local enhancement processing respectively, and cannot support both algorithms at the same time.

[0043] Since the image processing coefficient described in the embodiment of the present application can be input or adjusted by the user later, when the image processing coefficient magic_number>0, the image processing coefficient is a local enhancement coefficient; when the image processing coefficient magic_number≤0, the image processing coefficient is a noise removal coefficient.

[0044] Therefore, in the embodiment of the present application, two image optimization schemes can be supported in the same set of algorithms by controlling the input of the image adjustment coefficient.

[0045] In the present application, coefficient processing is performed on each of the first weight maps based on the image processing coefficient, which may specifically refer to point-by-point coefficient multiplication processing of pixels in each first weight map, and the coefficient is the image processing coefficient.

[0046] The second weight map described in this application is the first weight map after coefficient multiplication processing.

[0047] In the embodiment of the present application, since there are M preset grayscale slice values, and each preset grayscale slice value corresponds to a first weight map, after coefficient processing is performed on each first weight map, M second weight maps will be obtained.

[0048] Step 140: Obtain a target image based on each of the second weight maps, the first image sequence, and the second image sequence.

[0049] Specifically, in the embodiment of the present application, the image group can be cyclically updated according to each of the second weight maps, the first image sequence and the second image sequence, and then the image can be reconstructed to finally obtain an optimized target image.

[0050] In an embodiment of the present application, in a set of image processing methods, two different image processing coefficients are introduced, namely, a noise removal coefficient or a local enhancement coefficient, so as to respectively support image optimization processing schemes for performing local enhancement processing and image noise removal processing on the image. In the embodiment of the present application, on the basis of effectively supporting local enhancement optimization and image noise removal optimization, it is possible to effectively save program volume and effectively reduce the area of ​​a chip carrying the image processing method.

[0051] Optionally, obtaining a target image based on each of the second weight maps, the first image sequence, and the second image sequence includes:

[0052] Perform multi-scale downsampling on each of the second weight maps by using the second filter kernel to obtain M groups of third weight map sequences, wherein each group of the third weight map sequence includes N third weight maps arranged according to the image size;

[0053] Based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence, a third image sequence is obtained, wherein the first image sequence includes N first images arranged in sequence according to image sizes, and the second image sequence includes N second images arranged in sequence according to image sizes, and N is a positive integer;

[0054] After obtaining the third image sequence, in each iterative updating process of the third image sequence, a new third image sequence is obtained according to any group of the third weight map sequences in the M-1 groups of third weight map sequences, each of the first images and the third image sequence obtained in the previous iteration process, until all the third weight map sequences are traversed to output a fourth image sequence;

[0055] The fourth image sequence is reconstructed to obtain a target image.

[0056] Specifically, corresponding to the number of grayscale slice values ​​being M, the number of second weight maps described in the embodiment of the present application is also M. By performing multi-scale downsampling on each second weight map through the above-mentioned second filter kernel, M groups of third weight map sequences will be obtained.

[0057] In the embodiment of the present application, each group of third weight map sequences includes N third weight maps arranged according to the image size.

[0058] Since the second filter kernel in the above embodiment is used to downsample the second weight map here, the number of third images in each group of third weight map sequences obtained after multi-scale downsampling is the same as the number of first images and second images, that is, the number of third images is also N.

[0059] The Mth group of third weight map sequences described in the embodiments of the present application can specifically be a third weight map sequence generated by the second weight map corresponding to the maximum grayscale slice value, or a third weight map sequence generated by the second weight map corresponding to the minimum grayscale slice value.

[0060] The M-1 groups of third weight map sequences described in the embodiments of the present application are all third weight map sequences in the M groups of third weight map sequences except the Mth group of third weight map sequences.

[0061] When the third image sequence is initially generated in the embodiment of the present application, the third image sequence is mainly based on the N second images and combined with the image adjustment information determined by the third weight map sequence of the Mth group and the N first images to obtain the third image sequence.

[0062] After the third image sequence is obtained for the first time, it is necessary to continuously iterate and update the third image sequence through M-1 groups of third weight map sequences. Each subsequent process of updating the third image sequence is based on the third image sequence obtained in the previous iteration, and is combined with the new third weight map sequence and the image adjustment information determined by the N first images.

[0063] In the embodiment of the present application, after completing traversal of M-1 groups of third weight map sequences, the iterative update of the third image sequence is completed, and the result of the last iterative update is used as the output result to output the fourth image sequence.

[0064] In the embodiment of the present application, after the fourth image sequence is obtained, it is reconstructed into a frame of image to obtain a target image after image processing is completed.

[0065] In an embodiment of the present application, by utilizing M groups of third weight map sequences obtained after downsampling each second weight map and combining each first image and each second image, the third image sequence can be effectively updated, the image processing can be effectively realized, and finally a fourth image sequence is obtained.

[0066] Optionally, obtaining a third image sequence based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence comprises:

[0067] Performing the third image updating process N times to obtain a third image sequence, wherein the value of K is different in different updating processes, and K is a positive integer from 1 to N;

[0068] The third image updating process comprises:

[0069] Determine a target pixel in the Kth first image based on the grayscale value of each pixel in the Kth first image and the first preset grayscale slice value corresponding to the Mth group of the third weight map sequence;

[0070] Determining a Kth image adjustment coefficient based on a Kth third weight map in the third weight map sequence and a target pixel in the Kth first image;

[0071] A Kth third image is obtained based on the Kth image adjustment coefficient and the Kth first image.

[0072] Specifically, the third weight map sequence of the Mth group described in the embodiment of the present application is generated based on the second weight map, and the second weight map is based on the first weight map corresponding to the preset grayscale slice value. Therefore, the third weight map sequence at this time will also have the corresponding first preset grayscale slice value.

[0073] The K first images described in the present application refer to the Kth images among the N first images arranged according to image size.

[0074] The method of determining the target pixel point in the Kth first image described in the embodiment of the present application can specifically include first calculating the absolute value of the difference between the grayscale value of each pixel point in the Kth first image and the first preset grayscale slice value, that is, the first absolute value, and then comparing the relationship between the first absolute value and the first preset grayscale slice value, and selecting the pixel point whose first absolute value is greater than the first preset grayscale slice value as the target pixel point in the Kth first image.

[0075] In the embodiment of the present application, after the target pixel point is determined, the first pixel point set corresponding to the target pixel point in the Kth third weight map is first obtained.

[0076] Then, the difference between the grayscale value of the pixel point and the first preset grayscale slice value in the Kth first image is obtained, and the pixel points that can be divided by the interval step between each grayscale slice value are obtained, that is, the second pixel point set.

[0077] In the embodiment of the present application, the Kth image adjustment coefficient is determined based on the product between the second pixel point set, the first pixel point set and each target pixel point.

[0078] Finally, a Kth third image is obtained according to the sum of the Kth image adjustment coefficient and the Kth first image.

[0079] In the embodiment of the present application, after the Kth third image is obtained, the third image update process will continue to be executed N times, and the value of K is different in different update processes, where K is a positive integer from 1 to N, until N third images are obtained to obtain a complete third image sequence.

[0080] Optionally, obtaining a new third image sequence according to any group of the third weight map sequences in the M-1 groups of the third weight map sequences, each of the first images and the third image sequence acquired in the previous iteration process comprises:

[0081] Performing N new third image updating processes to obtain a new third image sequence, wherein the value of P is different in different updating processes, and P is a positive integer from 1 to N;

[0082] The new third image update process includes:

[0083] Determine a target pixel in the Pth first image based on the grayscale value of each pixel in the Pth first image and the second preset grayscale slice value corresponding to the third weight map sequence;

[0084] Determining a Pth image adjustment coefficient based on a Pth third weight map in the third weight map sequence and a target pixel in the Pth first image;

[0085] Based on the P-th image adjustment coefficient and the third image sequence acquired in the last iteration process, a P-th new third image is obtained.

[0086] Specifically, the methods for determining the target pixel point and the image adjustment coefficient described in the embodiments of the present application are consistent with the methods in the above embodiments.

[0087] In the embodiment of the present application, each new third image is obtained based on the sum of the Pth image adjustment coefficient and the third image sequence obtained in the previous iteration process.

[0088] In the cyclic update process in the embodiment of the present application, each third image in the third image sequence in the previous cycle is continuously used, and only one set of corresponding third image sequences is always saved. N new third image update processes are performed to obtain a new third image sequence. The value of P is different in different update processes, and P is a positive integer from 1 to N.

[0089] In the embodiment of the present application, after constructing the third image sequence, the third image sequence is continuously updated, which can effectively ensure the validity of image processing and the effect of image processing.

[0090] Optionally, reconstructing the fourth image sequence to obtain a target image includes:

[0091] Replacing the Nth fourth image in the fourth image sequence with the Nth second image to obtain a fourth target image sequence;

[0092] Based on the image size arrangement of the fourth target image sequence, the fourth target image sequence is stacked layer by layer to obtain a target image.

[0093] Specifically, the Nth fourth image described in the embodiment of the present application refers to the fourth image with the largest image size in the fourth image sequence, and the Nth second image refers to the second image with the largest image size among all second images.

[0094] In the embodiment of the present application, the Nth fourth image in the fourth image sequence is replaced by the Nth second image in order to ensure that the size of the final output image is consistent with the size of the image to be processed.

[0095] The layer-by-layer stacking process of the fourth target image sequence described in the embodiment of the present application may specifically be started from the image with the largest image size in the fourth target image sequence, and then layer-by-layer stacking is started in sequence to finally output the target image.

[0096] In the embodiment of the present application, by reconstructing the fourth image sequence, a target image having the same size as the image to be processed can be obtained, thereby completing the image processing.

[0097] It should be noted that the image processing method provided in the embodiment of the present application can be executed by an image processing device or a control module in the image processing device for executing the image processing method. In the embodiment of the present application, the image processing device provided in the embodiment of the present application is described by taking the image processing device executing the image processing method as an example.

[0098] Figure 2 The image processing device provided in the embodiment of the present application is as follows: Figure 2As shown, it includes: a downsampling module 210, a determination module 220, a first processing module 230 and a second processing module 240; wherein the downsampling module 210 is used to perform multi-scale downsampling on the image to be processed through the first filter kernel and the second filter kernel, respectively, to obtain a first image sequence and a second image sequence; wherein the first determination module 220 is used to determine the first weight map corresponding to each preset grayscale slice value based on M preset grayscale slice values ​​and the grayscale value of each pixel point in the image to be processed; wherein the first processing module 230 is used to perform coefficient processing on each of the first weight maps based on the image processing coefficient to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer; wherein the second processing module 240 is used to obtain a target image based on each of the second weight maps, the first image sequence and the second image sequence.

[0099] Optionally, the second processing module is specifically configured to:

[0100] Perform multi-scale downsampling on each of the second weight maps by using the second filter kernel to obtain M groups of third weight map sequences, wherein each group of the third weight map sequence includes N third weight maps arranged according to the image size;

[0101] Based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence, a third image sequence is obtained, wherein the first image sequence includes N first images arranged in sequence according to image sizes, and the second image sequence includes N second images arranged in sequence according to image sizes, and N is a positive integer;

[0102] After obtaining the third image sequence, in each iterative updating process of the third image sequence, a new third image sequence is obtained according to any group of the third weight map sequences in the M-1 groups of third weight map sequences, each of the first images and the third image sequence obtained in the previous iteration process, until all the third weight map sequences are traversed to output a fourth image sequence;

[0103] The fourth image sequence is reconstructed to obtain a target image.

[0104] Optionally, the second processing module is further configured to:

[0105] Performing the third image updating process N times to obtain a third image sequence, wherein the value of K is different in different updating processes, and K is a positive integer from 1 to N;

[0106] The third image updating process comprises:

[0107] Determine a target pixel in the Kth first image based on the grayscale value of each pixel in the Kth first image and the first preset grayscale slice value corresponding to the Mth group of the third weight map sequence;

[0108] Determining a Kth image adjustment coefficient based on a Kth third weight map in the third weight map sequence and a target pixel in the Kth first image;

[0109] A Kth third image is obtained based on the Kth image adjustment coefficient and the Kth first image.

[0110] Optionally, the second processing module is further configured to:

[0111] Performing N new third image updating processes to obtain a new third image sequence, wherein the value of P is different in different updating processes, and P is a positive integer from 1 to N;

[0112] The new third image update process includes:

[0113] Determine a target pixel in the Pth first image based on the grayscale value of each pixel in the Pth first image and the second preset grayscale slice value corresponding to the third weight map sequence;

[0114] Determining a Pth image adjustment coefficient based on a Pth third weight map in the third weight map sequence and a target pixel in the Pth first image;

[0115] Based on the P-th image adjustment coefficient and the third image sequence acquired in the last iteration process, a P-th new third image is obtained.

[0116] Optionally, the second processing module is further configured to:

[0117] Replacing the Nth fourth image in the fourth image sequence with the Nth second image to obtain a fourth target image sequence;

[0118] Based on the image size arrangement of the fourth target image sequence, the fourth target image sequence is stacked layer by layer to obtain a target image.

[0119] In an embodiment of the present application, in a set of image processing methods, two different image processing coefficients are introduced, namely, a noise removal coefficient or a local enhancement coefficient, so as to respectively support image optimization processing schemes for performing local enhancement processing and image noise removal processing on the image. In the embodiment of the present application, on the basis of effectively supporting local enhancement optimization and image noise removal optimization, it is possible to effectively save program volume and effectively reduce the area of ​​a chip carrying the image processing method.

[0120] The image processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0121] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0122] The image processing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0123] Optionally, Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 3 As shown, an embodiment of the present application also provides an electronic device 300, including a processor 301, a memory 302, and a program or instruction stored in the memory 302 and executable on the processor 301. When the program or instruction is executed by the processor 301, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0124] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0125] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0126] The electronic device 400 includes but is not limited to components such as a radio frequency unit 401 , a network module 402 , an audio output unit 403 , an input unit 404 , a sensor 405 , a display unit 406 , a user input unit 407 , an interface unit 408 , a memory 409 , and a processor 410 .

[0127] Those skilled in the art will appreciate that the electronic device 400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0128] The processor 410 is used to perform multi-scale downsampling on the image to be processed by using the first filter kernel and the second filter kernel, respectively, to obtain a first image sequence and a second image sequence;

[0129] Based on M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed, determining a first weight map corresponding to each preset grayscale slice value;

[0130] Based on an image processing coefficient, coefficient processing is performed on each of the first weight maps to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer;

[0131] A target image is obtained based on the second weight maps, the first image sequence and the second image sequence.

[0132] The processor 410 is configured to perform multi-scale downsampling on each of the second weight maps by using the second filter kernel to obtain M groups of third weight map sequences, wherein each group of the third weight map sequence includes N third weight maps arranged according to the image size;

[0133] Based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence, a third image sequence is obtained, wherein the first image sequence includes N first images arranged in sequence according to image sizes, and the second image sequence includes N second images arranged in sequence according to image sizes, and N is a positive integer;

[0134] After obtaining the third image sequence, in each iterative updating process of the third image sequence, a new third image sequence is obtained according to any group of the third weight map sequences in the M-1 groups of third weight map sequences, each of the first images and the third image sequence obtained in the previous iteration process, until all the third weight map sequences are traversed to output a fourth image sequence;

[0135] The fourth image sequence is reconstructed to obtain a target image.

[0136] The processor 410 is configured to execute N third image update processes to obtain a third image sequence, wherein the value of K is different in different update processes, and K is a positive integer from 1 to N;

[0137] The third image updating process comprises:

[0138] Determine a target pixel in the Kth first image based on the grayscale value of each pixel in the Kth first image and the first preset grayscale slice value corresponding to the Mth group of the third weight map sequence;

[0139] Determining a Kth image adjustment coefficient based on a Kth third weight map in the third weight map sequence and a target pixel in the Kth first image;

[0140] A Kth third image is obtained based on the Kth image adjustment coefficient and the Kth first image.

[0141] The processor 410 is used to execute N new third image update processes to obtain a new third image sequence, where the value of P is different in different update processes, and P is a positive integer from 1 to N;

[0142] The new third image update process includes:

[0143] Determine a target pixel in the Pth first image based on the grayscale value of each pixel in the Pth first image and the second preset grayscale slice value corresponding to the third weight map sequence;

[0144] Determining a Pth image adjustment coefficient based on a Pth third weight map in the third weight map sequence and a target pixel in the Pth first image;

[0145] Based on the P-th image adjustment coefficient and the third image sequence acquired in the last iteration process, a P-th new third image is obtained.

[0146] The processor 410 is configured to replace the Nth fourth image in the fourth image sequence with the Nth second image to obtain a fourth target image sequence;

[0147] Based on the image size arrangement of the fourth target image sequence, the fourth target image sequence is stacked layer by layer to obtain a target image.

[0148] In an embodiment of the present application, in a set of image processing methods, two different image processing coefficients are introduced, namely, a noise removal coefficient or a local enhancement coefficient, so as to respectively support image optimization processing schemes for performing local enhancement processing and image noise removal processing on the image. In the embodiment of the present application, on the basis of effectively supporting local enhancement optimization and image noise removal optimization, it is possible to effectively save program volume and effectively reduce the area of ​​a chip carrying the image processing method.

[0149] It should be understood that in the embodiment of the present application, the input unit 404 may include a graphics processor (GPU) 4041 and a microphone 4042, and the graphics processor 4041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 409 may be used to store software programs and various data, including but not limited to applications and operating systems. The processor 410 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 410.

[0150] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0151] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0152] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0153] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0154] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0155] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0156] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: The image to be processed is respectively subjected to multi-scale downsampling by using a first filter kernel and a second filter kernel to obtain a first image sequence and a second image sequence; Based on M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed, determining a first weight map corresponding to each preset grayscale slice value; Based on an image processing coefficient, coefficient processing is performed on each of the first weight maps to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer; Obtaining a target image based on each of the second weight maps, the first image sequence, and the second image sequence; The step of obtaining a target image based on each of the second weight maps, the first image sequence, and the second image sequence includes: Perform multi-scale downsampling on each of the second weight maps by using the second filter kernel to obtain M groups of third weight map sequences, wherein each group of the third weight map sequence includes N third weight maps arranged according to the image size; Based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence, a third image sequence is obtained, wherein the first image sequence includes N first images arranged in sequence according to image sizes, and the second image sequence includes N second images arranged in sequence according to image sizes, and N is a positive integer; After obtaining the third image sequence, in each iterative updating process of the third image sequence, a new third image sequence is obtained according to any group of the third weight map sequences in the M-1 groups of third weight map sequences, each of the first images and the third image sequence obtained in the previous iteration process, until all the third weight map sequences are traversed to output a fourth image sequence; The fourth image sequence is reconstructed to obtain a target image.

2. The image processing method according to claim 1, characterized in that: The obtaining a third image sequence based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence comprises: Performing the third image updating process N times to obtain a third image sequence, wherein the value of K is different in different updating processes, and K is a positive integer from 1 to N; The third image updating process comprises: Determine a target pixel in the Kth first image based on the grayscale value of each pixel in the Kth first image and the first preset grayscale slice value corresponding to the Mth group of the third weight map sequence; Determining a Kth image adjustment coefficient based on a Kth third weight map in the third weight map sequence and a target pixel in the Kth first image; A Kth third image is obtained based on the Kth image adjustment coefficient and the Kth second image.

3. The image processing method according to claim 1, characterized in that: The step of obtaining a new third image sequence according to any group of the third weight map sequences in the M-1 groups of the third weight map sequences, each of the first images, and the third image sequence acquired in the last iteration process comprises: Performing N new third image updating processes to obtain a new third image sequence, wherein the value of P is different in different updating processes, and P is a positive integer from 1 to N; The new third image update process includes: Determine a target pixel in the Pth first image based on the grayscale value of each pixel in the Pth first image and the second preset grayscale slice value corresponding to the third weight map sequence; Determining a Pth image adjustment coefficient based on a Pth third weight map in the third weight map sequence and a target pixel in the Pth first image; Based on the P-th image adjustment coefficient and the third image sequence acquired in the last iteration process, a P-th new third image is obtained.

4. The image processing method according to claim 1, characterized in that: The reconstructing the fourth image sequence to obtain a target image includes: Replacing the Nth fourth image in the fourth image sequence with the Nth second image to obtain a fourth target image sequence; Based on the image size arrangement of the fourth target image sequence, the fourth target image sequence is stacked layer by layer to obtain a target image.

5. An image processing device, characterized in that: include: A downsampling module, used to perform multi-scale downsampling on the image to be processed by using a first filter kernel and a second filter kernel, respectively, to obtain a first image sequence and a second image sequence; A first determination module, configured to determine a first weight map corresponding to each of the preset grayscale slice values ​​based on the M preset grayscale slice values ​​and the grayscale value of each pixel in the image to be processed; A first processing module, configured to perform coefficient processing on each of the first weight maps based on an image processing coefficient to obtain M second weight maps, wherein the image processing coefficient is a noise removal coefficient or a local enhancement coefficient, and M is a positive integer; A second processing module, configured to obtain a target image based on each of the second weight maps, the first image sequence and the second image sequence; Wherein, the second processing module is specifically used for: Perform multi-scale downsampling on each of the second weight maps by using the second filter kernel to obtain M groups of third weight map sequences, wherein each group of the third weight map sequence includes N third weight maps arranged according to the image size; Based on the Mth group of the third weight map sequence, the first image sequence and the second image sequence, a third image sequence is obtained, wherein the first image sequence includes N first images arranged in sequence according to image sizes, and the second image sequence includes N second images arranged in sequence according to image sizes, and N is a positive integer; After obtaining the third image sequence, in each iterative updating process of the third image sequence, a new third image sequence is obtained according to any group of the third weight map sequences in the M-1 groups of third weight map sequences, each of the first images and the third image sequence obtained in the previous iteration process, until all the third weight map sequences are traversed to output a fourth image sequence; The fourth image sequence is reconstructed to obtain a target image.

6. The image processing device according to claim 5, characterized in that: The second processing module is further configured to: Performing the third image updating process N times to obtain a third image sequence, wherein the value of K is different in different updating processes, and K is a positive integer from 1 to N; The third image updating process comprises: Determine a target pixel in the Kth first image based on the grayscale value of each pixel in the Kth first image and the first preset grayscale slice value corresponding to the Mth group of the third weight map sequence; Determining a Kth image adjustment coefficient based on a Kth third weight map in the third weight map sequence and a target pixel in the Kth first image; A Kth third image is obtained based on the Kth image adjustment coefficient and the Kth first image.

7. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method according to any one of claims 1 to 4.

8. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 4 are implemented.

Images