Image processing method and apparatus, image processing architecture, and electronic device

Abstract

Disclosed are an image processing method and apparatus, an image processing architecture, and an electronic device. The method comprises: obtaining an image to be processed; performing first processing on said image to obtain a plurality of first images of different frequency bands of said image, each frequency band comprising one first image; and sequentially performing image processing on each frequency band according to a set processing sequence, image processing results of each frequency band being fused in the obtained final target image, wherein the image processing configuration of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

Description

Image processing method and device, image processing architecture and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202411395770.0, filed on September 30, 2024, and entitled "Image processing method and device, image processing architecture and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image processing, and more particularly, to an image processing method and device, an image processing architecture and an electronic device. BACKGROUND

[0003] There will be some noise in any image taken by a device such as a mobile phone or a digital camera, and the noise is usually displayed as random spots in the image. In order to make the image present a better display effect, some processing modules will be used to process the image, such as noise reduction processing, sharpening, tone mapping, etc. The current image denoising technology not only consumes a lot of resources, but also needs to be further optimized in terms of processing effect. SUMMARY

[0004] Therefore, the present application provides the following technical solutions.

[0005] The first aspect of the present application provides an image processing method, which comprises:

[0006] obtaining a to-be-processed image;

[0007] performing first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image;

[0008] performing image processing of each frequency band in a set processing order, and finally obtaining a target image in which the image processing results of each frequency band are fused;

[0009] wherein the configuration of the image processing of each frequency band corresponds to the features of the first image corresponding to the frequency band.

[0010] In one possible implementation, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing object of the other frequency bands is the fusion image of the first image corresponding to the current processing frequency band and the processing result of the last processing frequency band.

[0011] In one possible implementation, before the image processing of each frequency band is performed in the set processing order, the method further comprises:

[0012] obtaining a processing target of the to-be-processed image;

[0013] determine configuration parameters of image processing of each frequency band based on the processing target.

[0014] In one possible implementation, before the image processing of each frequency band is sequentially performed according to the set processing sequence, the method further includes:

[0015] perform image recognition on the image to be processed to determine key features of the image to be processed;

[0016] determine a matched configuration strategy based on the key features;

[0017] determine configuration parameters of image processing of each frequency band based on the configuration strategy.

[0018] In one possible implementation, the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is respectively configured based on the processing target or the configuration strategy.

[0019] In one possible implementation, the image processing of each frequency band includes at least two different types of processing operations, the image processing of each frequency band is respectively configured based on the processing target or the configuration strategy, and the configuring includes:

[0020] determine processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations;

[0021] determine processing parameters of the processing operations that need to be enabled.

[0022] The second aspect of the present application provides an image processing device, including:

[0023] an image obtaining module configured to obtain an image to be processed;

[0024] a frequency division processing module configured to perform first processing on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, each frequency band including one first image;

[0025] an image processing module configured to sequentially perform image processing of each frequency band according to a set processing sequence, and finally obtain a target image in which image processing results of each frequency band are fused;

[0026] wherein configuration of the image processing of each frequency band corresponds to features of the first image corresponding to the frequency band.

[0027] In one possible implementation, the image processing module includes a control submodule and at least two processing submodules, and wherein:

[0028] The control submodule is configured to determine the processing submodules that need to be enabled in the at least two processing submodules, and determine the processing parameters of each processing submodule that needs to be enabled.

[0029] The third aspect of the present application provides an image processing architecture, comprising:

[0030] The hierarchical processing circuit is configured to process the obtained image to be processed into a plurality of first images containing different frequency bands;

[0031] The at least one set of hierarchical processing circuits is configured to perform image processing of each frequency band according to a set processing sequence, and each set of hierarchical processing circuits comprises a plurality of processing submodules, and the hierarchical processing circuits satisfy that the enabled states and processing parameters of different processing submodules are configurable;

[0032] The fusion processing circuit is configured to perform fusion processing on images from different sources, so that the final target image fuses the image processing results of each frequency band.

[0033] In one possible implementation, the image processing architecture further comprises:

[0034] The control circuit connected to the hierarchical processing circuit is configured to send a control signal to the at least one set of hierarchical processing circuits when performing image processing of each frequency band based on the configuration of image processing of each frequency band.

[0035] In one possible implementation, the image processing architecture comprises at least two sets of hierarchical processing circuits, and the at least two sets of hierarchical processing circuits have a multiplexed image processing module.

[0036] The fourth aspect of the present application provides an electronic device comprising at least one processor and a memory connected to the processor, wherein:

[0037] The memory is configured to store a computer program;

[0038] The processor is configured to execute the computer program to enable the electronic device to implement any of the above image processing methods. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0040] FIG. 1 is a flowchart of an image processing method disclosed in an embodiment of the present application;

[0041] FIG. 2 is a schematic diagram illustrating an implementation principle of a layer-by-layer fusion image processing method according to an embodiment of the present application;

[0042] FIG. 3 is a schematic diagram illustrating an effect comparison of different frequency bands performing the same processing operation according to an embodiment of the present application;

[0043] FIG. 4 is a schematic diagram illustrating an effect comparison of different processing configurations for specific scenes according to an embodiment of the present application;

[0044] FIG. 5 is an example diagram of a processing operation of a frequency band of a night scene image according to an embodiment of the present application;

[0045] FIG. 6 is an example diagram of an implementation process of an image post-processing according to a conventional scheme;

[0046] FIG. 7 is a schematic diagram illustrating a structure of an image processing apparatus according to an embodiment of the present application;

[0047] FIG. 8 is a schematic diagram illustrating a structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0049] The embodiments of the present application can be applied to an electronic device, and the product form of the electronic device is not limited by the present application, which can include but is not limited to a smart phone, a tablet computer, a wearable device, a personal computer (PC), a netbook, etc., and can be selected according to application requirements.

[0050] FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application. As shown in FIG. 1, the image processing method can include:

[0051] Step 101: obtaining a to-be-processed image.

[0052] The to-be-processed image can be an image collected by an image collection device in real time. For example, a user collects an image using a camera application. In the process from triggering the shutter to outputting the captured image on the display screen, the collected image is processed using the image processing scheme disclosed in the present application, and the processed image is directly displayed on the display screen. Or the user does not trigger the shutter, and the camera device continuously collects a video stream and outputs on the display screen. In the process from collecting the video stream to outputting on the display screen, each image frame in the video stream can be processed using the image processing scheme disclosed in the present application, and the processed video stream is output to the display screen for display.

[0053] Alternatively, the to-be-processed image is an existing image. In the case where the existing image has a noise reduction or optimization processing requirement, the existing image can be imported into the corresponding application of the image processing method disclosed in the present application for processing.

[0054] Step 102: performing first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image.

[0055] The first processing is used to obtain an image of a specific frequency band of the to-be-processed image, that is, the first image. Within the original frequency band span of the to-be-processed image, a plurality of first images of multiple frequency bands of the to-be-processed image can be extracted. In the application, the frequency band range and the number of frequency bands to be obtained can be determined based on actual requirements.

[0056] The first processing is based on the to-be-processed image and can include the following processing operations: image downsampling, image upsampling, image data subtraction processing, etc. How to obtain a plurality of first images of different frequency bands based on the to-be-processed image will be described in detail in the embodiments below, and will not be described in detail here.

[0057] Step 103: performing image processing of each frequency band in turn according to a set processing order, and finally obtaining a target image in which the image processing results of each frequency band are fused, and the configuration of the image processing of each frequency band corresponds to the features of the first image corresponding to the frequency band.

[0058] The frequency bands of the plurality of first images obtained as described above are different. In the application, image processing of each frequency band can be performed in turn according to a set order, such as from low to high or from high to low. The image processing results of each frequency band are fused into the final target image, ensuring that the image processing of each frequency band can only have a corresponding effect in the final target image. How the image processing results of each frequency band are fused will be described in detail in the embodiments below.

[0059] The image processing of each frequency band at least has some differences, and the differences can be any of the following: the type of enabled processing operation, the processing parameter of the same type of processing operation, and the execution order of the processing operation. That is, the image processing of each frequency band can include multiple types of processing, and the configurations of the image processing of different frequency bands are different, including the type of processing operation, the processing order of the processing operation, and the processing parameter.

[0060] The scheme provided in the application performs pyramid frequency band layering processing on the image to be processed, and then performs image processing on the first images of each frequency band (level) respectively. Different from the conventional scheme, in the scheme provided in the application, the configuration of the image processing of each frequency band corresponds to the characteristics of the first image corresponding to the frequency band. Based on different processing requirements or processing targets, the processing of each frequency band can be independently configured flexibly. For example, for the first image of a low frequency band, in order to filter out low-frequency noise as much as possible to improve the definition of the final target image, spatial domain noise reduction needs to be performed emphatically. For the first image of a high frequency band, since the details of the image need to be preserved, time domain noise reduction needs to be performed emphatically.

[0061] The image processing method provided in the embodiment only performs pyramid frequency band layering processing once on the processing thread of the image to be processed, and the area and power consumption requirements are not high; in the implementation, the characteristics of the first images of different frequency bands can be used to independently configure the processing of each frequency band, and different image processing is completed to achieve the optimization of the final target image.

[0062] In one implementation, the processing results of the first images of each frequency band can be fused layer by layer, and finally integrated into the image processing result of the last processed frequency band to obtain the target image. FIG. 2 is a schematic diagram of the implementation principle of the layer-by-layer fusion type image processing method disclosed in the embodiment of the application. The following content can be understood in combination with FIG. 2.

[0063] The architecture shown in FIG. 2 is implemented by layer-by-layer fusion from a low frequency band to a high frequency band. The image to be processed is input from the left, the first processing is performed, and four frequency band levels of first images are obtained, corresponding to the four horizontal lines to the right in the figure. The lowest line inputs the first image of the lowest frequency band, and the highest line inputs the first image of the highest frequency band. Each line (each frequency band) includes multiple processing modules (rectangular boxes on the line), which can perform multiple types of image processing on the first image.

[0064] The structure of the matrix in the upper left corner of Fig. 2, i.e. the structure of pyramid layering processing to the image to be processed, see the uppermost horizontal line in Fig. 2, after the input image is input, first a downsampling processing (the processing of the circle in the first column of the first row) is performed, then a upsampling processing (the processing of the circle in the second column of the first row) is performed, the size of the image after upsampling is the same as that of the image to be processed, then the original image to be processed is subtracted from the image after upsampling (the processing of the circle in the third column of the first row), to obtain the first image of the high frequency band; see the second horizontal line from top to bottom in Fig. 2, which is input the image after once downsampling of the image to be processed, on this basis, the same downsampling processing (the processing of the circle in the first column of the second row) is performed, then the upsampling processing (the processing of the circle in the second column of the second row) is performed, then the image after once downsampling of the image to be processed is subtracted from the image after upsampling of the second row (the processing of the circle in the third column of the third row), to obtain the first image of the next high frequency band. The method of obtaining the first image of other frequency bands is the same as the above method, which will not be introduced one by one.

[0065] In combination with Fig. 2, first, the first image of the lowest frequency band is subjected to a series of processing operations (spatial domain noise reduction, time domain noise reduction, tone mapping, sharpening, etc.), and then the processing result is upsampled to the same size as the first image of the next high frequency band, and then it is fused with the first image of the next high frequency band as the input of the next high frequency band, and so on, until the image processing of the highest frequency band is completed, to obtain the final target image processing result. Based on this, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing object of other frequency bands is the fused image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

[0066] In other implementations, the image processing of each frequency band can also be independent of each other and not associated with each other, and finally the image processing results of each frequency band are fused into the final target image. The fusion method of the image processing results of each frequency band is not fixedly limited in the present application.

[0067] Therefore, the image processing results of each frequency band are fused into the final target image, which ensures that the image processing of each frequency band can only present the corresponding effect or effect in the final target image.

[0068] The lowermost long bar in FIG. 2 corresponds to a time domain noise reduction processing module. In the example of FIG. 2, the time domain noise reduction processing module is a multiplexing module, and time domain noise reduction processing of each frequency band is completed through the time domain noise reduction processing module. The solid line between the time domain noise reduction processing module and the first image conveying line represents a default connection relationship, and the dashed line represents a switchable connection relationship. Referring to FIG. 2, taking the lowest frequency band line as an example, when the default connection relationship is adopted, time domain noise reduction processing is performed before spatial domain noise reduction processing (the first rectangular processing module on the left). When the connection position corresponding to the dashed line is adopted, spatial domain noise reduction processing is performed before time domain noise reduction processing, so that the execution order of different types of processing operations in the image processing of the same frequency band can be adjusted. Of course, in other implementations, the solid line and the dashed line between the time domain noise reduction processing module and the first image conveying line can be arranged at different positions based on application requirements, to adjust the execution order of different processing modules. In addition, other processing modules in addition to the time domain noise reduction processing module can also be arranged as multiplexing modules, and the connection positions of the solid line and the dashed line between the other processing modules and the first image conveying line can also be designed based on requirements.

[0069] In one implementation, before the image processing of each frequency band is sequentially performed according to the set processing order, the method can further include: obtaining a processing target of the image to be processed; and determining configuration parameters of the image processing of each frequency band based on the processing target.

[0070] In some scenarios, there can be a clear processing target for the image, so that the image to be processed can achieve the desired effect after processing. For example, for an image with a halo in an outdoor scene, it is necessary to focus on halo removal, and optimization processing of tone and contrast can be performed in a specific frequency band, so that the presentation effect of the halo in the final target image is greatly reduced.

[0071] FIG. 3 is a comparison diagram of effects of different frequency bands performing the same processing operation according to an embodiment of the present application. As shown in FIG. 3, a pyramid is divided into four frequency band layers, the 0th layer corresponds to the highest frequency band, and then the 2nd layer, the 4th layer, and the 8th layer. As shown in FIG. 3, the middle image is the effect of UV noise reduction processing at the bottom layer (0th layer) of the pyramid, and the right image is the effect of UV noise reduction processing at the 2nd layer of the pyramid. As can be seen, the left image has more large block color noise, and the right image has more fine noise. Therefore, in order to avoid large block color noise, the processing parameters of each frequency band can be configured based on experience.

[0072] In applications with a processing target, because there is a clear image processing effect as a guide, each frequency band can be configured according to the processing target, different frequency bands achieve different processing effects, and the target image finally fused with all frequency band processing results can also well match the processing target.

[0073] In another implementation, before the image processing of each frequency band is sequentially performed according to the preset processing sequence, the method can further include: performing image recognition on the image to be processed to determine a key feature of the image to be processed; determining a matched configuration strategy based on the key feature; and determining configuration parameters of the image processing of each frequency band based on the configuration strategy.

[0074] The key feature of the image to be processed can be, but is not limited to, an image scene and an image theme. Different configuration strategies are used for different image scenes or image themes to optimize the image processing. In the implementation, a plurality of configuration strategies can be preset, and after the key feature of the image is recognized, a most matched configuration strategy can be selected from the plurality of preset configuration strategies for application. Alternatively, after the key feature of the image to be processed is recognized, a suitable configuration strategy can also be determined in real time through an algorithm.

[0075] FIG. 4 is a comparison diagram of effects of different processing configurations for specific scenes according to the embodiment of the present application. As shown in FIG. 4, processing configuration combination 1: adjust the module intensity and other parameters of each frequency band layer; combination 2: change the order of spatial noise reduction, temporal noise reduction, and sharpening. As shown in FIG. 4, the processing configuration of combination 1 retains more details and textures, and combination 2 makes the whole image look more neat, but sacrifices some details.

[0076] In order to better understand the technical scheme of the present application, the following describes the image processing content in a night scene mode. FIG. 5 is an example diagram of processing operations of one frequency band of a night scene image according to the embodiment of the present application. As shown in FIG. 5, in a night scene, the noise is usually very large. In order to suppress the noise as much as possible, highlight the image details and contrast, and restore the real effect of the image, the first image of the highest frequency band layer G0 of the pyramid framework can be configured as follows: in the image processing of the first image of the G0 layer, spatial noise reduction is performed first, and then temporal noise reduction is performed, so as to maintain a balance between high-frequency noise and details, and make the high-frequency information more stable through temporal superposition; in addition, the sharpening parameter of the G0 layer can be set to be relatively low, so as to improve the details while avoiding excessive enhancement of residual noise.

[0077] For the image processing of other frequency band layers except the highest frequency band layer G0, the processing configuration can be correspondingly performed according to the processing requirements. For example, for the first image of a low frequency band layer, temporal noise reduction can be performed first, and then spatial noise reduction is performed, and in addition, the sharpening parameter can be set to be relatively high.

[0078] In the above embodiment implementation, the image processing of each frequency band can include image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured based on the processing target or the configuration strategy.

[0079] Specifically, if the image processing of each frequency band includes at least two different types of processing operations, the image processing of each frequency band is configured respectively based on the processing target or the configuration strategy, which can include: determining the processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; and determining the processing parameters of the processing operations that need to be enabled. That is, the configuration of each frequency band includes determining which processing operations need to be enabled and which processing operations need to be disabled, and the processing parameters of the processing operations that need to be enabled are determined.

[0080] For example, the image to be processed is divided into two layers, corresponding to a high frequency band and a low frequency band. In order to preserve more color details, no color processing is performed in the low frequency band, and color management processing is performed in the high frequency band. At the same time, based on the requirement of color details, the corresponding processing parameters are configured for the color management of the high frequency band.

[0081] FIG. 6 is an example diagram of an implementation flow of image post-processing of a conventional scheme. In combination with FIG. 6, the conventional scheme generally performs single-frame pyramid noise reduction in the second half of ISP (Image Signal Processing), then performs time-domain noise reduction, and finally performs global, local tone mapping and sharpening processing. However, these processing modules are completely serially processed, and one module is completely processed before another module is processed. Once the previous module causes some side effects, it is difficult to remedy the subsequent module. In addition, because each processing is different, the processing methods may be inconsistent in different signal frequency bands, causing some uneven side effects.

[0082] The image processing scheme described in the embodiments of the present application can flexibly switch and switch the positions of the processing modules on each frequency band from the algorithm perspective, so as to achieve the optimal algorithm effect. From the debugging perspective, each processing module can be specially set and adjusted for different layers or frequency bands, different details or noise effects can be maintained on different layers, the contrast and clarity of the image can be flexibly controlled, and the effect style of the product is greatly enriched.

[0083] For the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0084] The embodiments disclosed in the present application are described in detail above, and the method of the present application can be implemented in various forms of devices. Therefore, the present application also discloses a device, and specific embodiments are given below for detailed description.

[0085] FIG. 7 is a structural schematic diagram of an image processing device disclosed in an embodiment of the present application. Referring to FIG. 7, the image processing device 70 can include:

[0086] An image obtaining module 701 is configured to obtain a to-be-processed image.

[0087] A frequency division processing module 702 is configured to perform first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image.

[0088] An image processing module 703 is configured to perform image processing of each frequency band in a set processing sequence, and finally obtain a target image in which image processing results of each frequency band are fused.

[0089] In the image processing of each frequency band, the configuration corresponds to the feature of the first image of the corresponding frequency band.

[0090] In one implementation, the image processing module includes a control sub-module and at least two processing sub-modules, wherein the control sub-module is configured to determine the processing sub-modules to be enabled and determine the processing parameters of each processing sub-module to be enabled.

[0091] In combination with FIG. 6, in the conventional image post-processing scheme, only the noise reduction part adopts the pyramid frequency band division processing mode, and other optimization processing is performed in series after the noise reduction processing. Thus, the side effects caused by the previous processing are difficult to be eliminated in the subsequent processing, and each processing module independently performs image processing. Thus, in different signal frequency bands of the image, the inconsistent processing modes can cause the problem that the overall image effect is not uniform.

[0092] Based on the above, the image processing scheme disclosed in the embodiment performs frequency band layering processing of the pyramid only once on the processing thread of the to-be-processed image. On one hand, the area and power consumption requirements are not high, making it easier to implement the scheme. On the other hand, in the implementation, the features of the first images of different frequency bands can be independently configured for processing of each frequency band, so as to maintain different detail or noise effects on different frequency band layers, complete image processing with different focuses, and achieve the optimization of the final target image. In one implementation, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing object of other frequency bands is the fusion image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

[0093] In one implementation, the apparatus can further include a processing configuration module configured to obtain a processing target of the image to be processed before performing the image processing of each frequency band in the set processing order; and determine configuration parameters of the image processing of each frequency band based on the processing target.

[0094] In one implementation, the apparatus can further include a processing configuration module configured to perform image recognition on the image to be processed to determine a key feature of the image to be processed; determine a matched configuration strategy based on the key feature; and determine configuration parameters of the image processing of each frequency band based on the configuration strategy.

[0095] In one implementation, the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured based on the processing target or the configuration strategy.

[0096] In one implementation, the image processing of each frequency band includes at least two different types of processing operations, and the processing configuration module is specifically configured to determine processing operations to be enabled, the number of the processing operations to be enabled is not greater than the number of the at least two different types of processing operations, and determine processing parameters of the processing operations to be enabled.

[0097] The specific implementation of the image processing apparatus and each module included therein and other possible implementations can be found in the content of the corresponding part of the method embodiments, which will not be repeated here.

[0098] Further, the present application discloses an image processing architecture, including:

[0099] A hierarchical processing circuit is configured to process the obtained image to be processed into a plurality of first images including different frequency bands.

[0100] At least one set of hierarchical processing circuits is configured to perform image processing of each frequency band in a set processing order, each set of hierarchical processing circuits includes a plurality of processing sub-modules, and the hierarchical processing circuits satisfy that the enabled states and processing parameters of different processing sub-modules are configurable.

[0101] A fusion processing circuit is configured to perform fusion processing on images from different sources so that the image processing results of each frequency band are fused in a final target image.

[0102] In the implementation of only one set of hierarchical processing circuit, each processing module in the hierarchical processing circuit is a multiplexed module, which can perform processing operations in image processing of any frequency band. For example, the tone mapping module performs tone mapping processing on the first image of the G2 layer in the image processing of the frequency band layer G2, and then performs tone mapping processing on the first image of the G0 layer in the image processing of the frequency band layer G0. Such a design has the lowest overall hardware cost, but the overall image processing efficiency is relatively low.

[0103] Alternatively, the number of sets of hierarchical processing circuits is the same as the number of frequency band layers (pyramid layers) of the image to be processed, so that each frequency band layer has an independent corresponding set of hierarchical processing circuits available, and each set of hierarchical processing circuits is only responsible for image processing of the corresponding frequency band layer. Such a design has the highest overall hardware cost, but can effectively ensure processing efficiency.

[0104] Alternatively, in other implementations, the image processing architecture includes two sets of hierarchical processing circuits, so that when one set of hierarchical processing circuits performs image processing of a certain frequency band, the other set of hierarchical processing circuits can perform processing parameter setting, control of processing module enable or disable lines, line adjustment of processing module execution order, etc. in advance according to the configuration of the next processing frequency band. This design can ensure that image processing of the next frequency band can be performed immediately after image processing of the current frequency band is completed. This design has relatively low hardware cost and can ensure overall image processing efficiency.

[0105] In other implementations, the image processing architecture can further include a control circuit connected to the hierarchical processing circuit, the control circuit being configured to send control signals to the at least one set of hierarchical processing circuits based on the configuration of image processing of each frequency band.

[0106] Before image processing of each frequency band, processing parameter setting, control of processing module enable or disable lines, line adjustment of processing module execution order, etc. need to be performed according to the determined image processing configuration. This part of work can be completed by the control circuit.

[0107] In one implementation, the image processing architecture includes at least two sets of hierarchical processing circuits, and there are multiplexed image processing modules in the at least two sets of hierarchical processing circuits, for example, the time domain noise reduction processing module in FIG. 2. Such a partially hardware multiplexed design can reduce the number and size of hardware structures in the hierarchical processing circuit, saving hardware cost.

[0108] In the implementation of the present application, all processing operations that can be performed in each hierarchical processing circuit correspond to an independent processing module, and the image processing of each frequency band layer can be combined by connecting the corresponding module to start the circuit configuration, thereby forming a corresponding processing layer. That is, various image processing operations are optional designs, and the corresponding processing module is connected to the processing thread according to the type of processing operation required.

[0109] Based on the above, from the hardware perspective, the image processing architecture integrates the entire image post-processing (including noise reduction, image optimization processing, etc.) into a pyramid architecture, and the image processing of each frequency band layer of the pyramid can be flexibly configured based on the hierarchical processing circuit, so that different frequency band layers maintain different detail or noise effects, thereby achieving the optimization of the final target image.

[0110] The image processing device of any one of the above embodiments includes a processor and a memory, and the image obtaining module, the frequency division processing module, the image processing module, the control submodule, the processing submodule, etc. in the above embodiments are stored in the memory as program modules, and the corresponding functions are realized by the processor executing the above program modules stored in the memory.

[0111] The processor includes a core, and the core retrieves the corresponding program module from the memory. The core can be set to one or more, and the processing of the revisited data is realized by adjusting the core parameters.

[0112] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0113] In an exemplary embodiment, a computer readable storage medium directly loadable into the internal memory of a computer is also provided, which contains software codes. After the computer program is loaded and executed by the computer, the steps shown in any embodiment of the above image processing method can be realized.

[0114] In an exemplary embodiment, a computer program product directly loadable into the internal memory of a computer is also provided, which contains software codes. After the computer program is loaded and executed by the computer, the steps shown in any embodiment of the above image processing method can be realized.

[0115] Further, an electronic device is provided in the embodiments of the present application. Fig. 8 is a structural schematic diagram of an electronic device disclosed in the embodiments of the present application. Referring to Fig. 8, the electronic device 80 comprises at least one processor 801, at least one memory 802 connected with the processor, and a bus 803; wherein the processor and the memory complete mutual communication through the bus; the processor is used to invoke program instructions in the memory, so as to execute the image processing method described above.

[0116] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0117] It should also be noted that the relational terms herein such as first and second and the like are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by an "including a" statement does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0118] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0119] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, the method comprising: obtaining a to-be-processed image; performing first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image; performing image processing of each frequency band in a set processing order, and finally obtaining a target image in which image processing results of each frequency band are fused; wherein the configuration of the image processing of each frequency band corresponds to the characteristics of the first image corresponding to the frequency band.

2. The image processing method of claim 1, wherein, The image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing object of other frequency bands is a fused image of the first image corresponding to the current processing frequency band and the processing result of the last processing frequency band.

3. The image processing method of claim 1, before the performing image processing of each frequency band in a set processing order, further comprising: obtaining a processing target of the to-be-processed image; determining configuration parameters of the image processing of each frequency band based on the processing target.

4. The image processing method of claim 3, wherein the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured based on the processing target or a configuration strategy.

5. The image processing method of claim 4, wherein the image processing of each frequency band includes at least two different types of processing operations, the image processing of each frequency band is configured based on the processing target, including: determining processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; and determining processing parameters of the processing operations that need to be enabled.

6. The image processing method of claim 1, before the performing image processing of each frequency band in a set processing order, further comprising: performing image recognition on the to-be-processed image to determine key characteristics of the to-be-processed image; determining a matched configuration strategy based on the key characteristics; and determining configuration parameters of the image processing of each frequency band based on the configuration strategy.

7. The image processing method of claim 6, wherein the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured based on the processing target or a configuration strategy.

8. The image processing method of claim 7, wherein the image processing of each frequency band includes at least two different types of processing operations, the configuration parameters of the image processing of each frequency band are determined based on the configuration strategy, including: determining processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; and determining processing parameters of the processing operations that need to be enabled.

9. An image processing apparatus, comprising: an image obtaining module configured to obtain a to-be-processed image; a frequency division processing module configured to perform first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image; and An image processing module is configured to sequentially perform image processing of each frequency band in a set processing order, and finally obtain a target image in which image processing results of each frequency band are fused. The configuration of image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

10. The image processing apparatus of claim 9, wherein the image processing module comprises a control submodule and at least two processing submodules. The control submodule is configured to determine processing submodules that need to be enabled in the at least two processing submodules, and determine processing parameters of each processing submodule that needs to be enabled.

11. An image processing architecture, comprising: A hierarchical processing circuit is configured to process an obtained image to be processed into a plurality of first images containing different frequency bands. At least one set of hierarchical processing circuits is configured to sequentially perform image processing of each frequency band in a set processing order, and each set of hierarchical processing circuits comprises a plurality of processing submodules, and the hierarchical processing circuits are configured to support configuration of an enabled state and processing parameters of different processing submodules. A fusion processing circuit is configured to perform fusion processing on images from different sources, so that the final target image fuses image processing results of each frequency band.

12. The image processing architecture of claim 11, further comprising: A control circuit connected to the hierarchical processing circuit, the control circuit is configured to send a control signal to the at least one set of hierarchical processing circuits when performing image processing of each frequency band based on the configuration of image processing of each frequency band.

13. The image processing architecture of claim 11, comprising at least two sets of hierarchical processing circuits, and the at least two sets of hierarchical processing circuits have multiplexed image processing modules.

14. An electronic device comprising at least one processor and a memory connected to the processor, wherein: The memory is configured to store a computer program; The processor is configured to execute the computer program to enable the electronic device to perform the following: Obtain an image to be processed; Perform first processing on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, each frequency band including one first image; Sequentially perform image processing of each frequency band in a set processing order, and finally obtain a target image in which image processing results of each frequency band are fused. The configuration of image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

15. The electronic device of claim 14, wherein: The image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing object of other frequency bands is the fusion image of the first image corresponding to the current processing frequency band and the processing result of the last processing frequency band.

16. The electronic device of claim 14, wherein execution of the computer program is further configured to implement: Before the image processing of each frequency band is sequentially performed in the set processing order, obtain a processing target of the image to be processed; Determine configuration parameters of image processing of each frequency band based on the processing target.

17. The electronic device of claim 14, wherein execution of the computer program is further configured to implement: Before the image processing of each frequency band is performed in the order according to the preset processing sequence, image recognition is performed on the image to be processed to determine key features of the image to be processed; A matching configuration strategy is determined based on the key features; Configuration parameters of the image processing of each frequency band are determined based on the configuration strategy.

18. The electronic device of claim 16 or 17, wherein the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured based on the processing target or the configuration strategy.

19. The electronic device of claim 18, wherein the image processing of each frequency band includes at least two different types of processing operations, and the image processing of each frequency band is configured based on the processing target or the configuration strategy, including: determining processing operations that need to be enabled, wherein the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; and determining processing parameters of the processing operations that need to be enabled.

Images