Image processing method and device, image processing architecture and electronic equipment

Abstract

The invention discloses an image processing method and device, an image processing architecture and electronic equipment. The method comprises the steps of obtaining a to-be-processed image; performing first processing on the to-be-processed image to obtain a plurality of first images of different frequency bands of the to-be-processed image, each frequency band including one first image; according to a set processing sequence, image processing of each frequency band is carried out in sequence, and an image processing result of each frequency band is fused in a finally obtained target image; wherein the image processing configuration of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to an image processing method, device, image processing architecture and electronic device. Background Art

[0002] There is always some noise in any image taken by a mobile phone, digital camera or other device. The noise usually appears as random spots in the image. In order to make the image present a better display effect, the image will be processed through some processing modules, such as noise reduction, sharpening, tone mapping, etc. The current image denoising technology is not only resource-intensive, but also the processing effect needs to be further optimized. Summary of the invention

[0003] In view of this, this application provides the following technical solutions:

[0004] The first aspect of the present application provides an image processing method, the method comprising:

[0005] Obtaining an image to be processed;

[0006] Performing a first process on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, wherein each frequency band includes a first image;

[0007] According to the set processing order, the image processing of each frequency band is carried out in turn, and the image processing results of each frequency band are integrated into the final target image;

[0008] The configuration of the image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

[0009] In a possible implementation, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing objects of other frequency bands are the fusion image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

[0010] In a possible implementation, before performing image processing of each frequency band in sequence according to a set processing order, the method further includes:

[0011] Obtaining a processing target of the image to be processed;

[0012] Configuration parameters of image processing for each frequency band are determined based on the processing target.

[0013] In a possible implementation, before performing image processing of each frequency band in sequence according to a set processing order, the method further includes:

[0014] Performing image recognition on the image to be processed to determine key features of the image to be processed;

[0015] Determining a matching configuration strategy based on the key features;

[0016] The configuration parameters of the image processing in each frequency band are determined based on the configuration strategy.

[0017] In a possible implementation, the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured separately based on the processing target or configuration strategy.

[0018] In a possible implementation, the image processing of each frequency band includes at least two different types of processing operations, and the image processing of each frequency band is configured separately based on the processing target or configuration strategy, including:

[0019] Determining processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations;

[0020] Determine the processing parameters of the processing operation that needs to be enabled.

[0021] A second aspect of the present application provides an image processing device, comprising:

[0022] An image acquisition module, used for acquiring an image to be processed;

[0023] A frequency division processing module, used for performing a first process on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, each frequency band including a first image;

[0024] The image processing module is used to perform image processing of each frequency band in sequence according to a set processing order, and the image processing results of each frequency band are integrated into the final target image;

[0025] The configuration of the image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

[0026] In a possible implementation, the image processing module includes a control submodule and at least two processing submodules, wherein:

[0027] The control submodule is used to determine the processing submodule that needs to be enabled among the at least two processing submodules, and to determine the processing parameters of each processing submodule that needs to be enabled.

[0028] The third aspect of the present application provides an image processing architecture, including:

[0029] A layered processing circuit, used for processing the obtained image to be processed into a plurality of first images including different frequency bands;

[0030] At least one group of hierarchical processing circuits, used for performing image processing of each frequency band according to a set processing order, wherein one group of hierarchical processing circuits includes a plurality of processing submodules, and the hierarchical processing circuits satisfy that the enabling states and processing parameters of different processing submodules are configurable;

[0031] The fusion processing circuit is used to perform fusion processing on images from different sources so that the final target image is fused with the image processing results of each frequency band.

[0032] In a possible implementation, it also includes:

[0033] A control circuit connected to the hierarchical processing circuit is used to send a control signal to the at least one group of hierarchical processing circuits when performing image processing in each frequency band based on the configuration of the image processing in each frequency band.

[0034] In a possible implementation, the image processing architecture includes at least two groups of hierarchical processing circuits, and multiplexed image processing modules exist in the at least two groups of hierarchical processing circuits.

[0035] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0036] The memory is used to store computer programs;

[0037] The processor is used to execute the computer program so that the electronic device can implement any one of the above-mentioned image processing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A flowchart of an image processing method disclosed in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of the implementation principle of the layer-by-layer fusion image processing method disclosed in the embodiment of the present application;

[0041] Figure 3 A schematic diagram comparing the effects of performing the same processing operation on different frequency bands disclosed in the embodiments of the present application;

[0042] Figure 4A schematic diagram comparing the effects of different processing configurations for specific scenarios disclosed in the embodiments of the present application;

[0043] Figure 5 This is an example diagram of the processing operation of one frequency band of a night scene image disclosed in this embodiment;

[0044] Figure 6 This is an example diagram of the image post-processing implementation process of the traditional solution;

[0045] Figure 7 A schematic diagram of the structure of an image processing device disclosed in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The embodiments of the present application can be applied to electronic devices. The present application does not limit the product form of the electronic device, which can include but is not limited to smart phones, tablet computers, wearable devices, personal computers (PCs), netbooks, etc., and can be selected according to application requirements.

[0049] Figure 1 This is a flowchart of an image processing method disclosed in an embodiment of the present application. Figure 1 As shown, the image processing method may include:

[0050] Step 101: Obtain an image to be processed.

[0051] The image to be processed can be an image captured in real time by an image capture device. For example, a user uses a camera application to capture an image. When the user triggers a shutter to output the captured image to a display screen, the image processing scheme disclosed in the present application scheme is used to process the captured image, and the processed image is directly displayed on the display screen; or when the user does not trigger the shutter, the camera device continuously captures a video stream and outputs it on the display screen. When the video stream is captured and output to the display screen, the image processing scheme disclosed in the present application scheme can be used to process each image frame in the video stream, and the processed video stream can be output to the display screen for display.

[0052] Alternatively, the image to be processed is an existing image. When the existing image has a requirement for noise reduction or optimization processing, the existing image can be imported into a corresponding application of the image processing method disclosed in the present application for processing.

[0053] Step 102: performing a first process on the image to be processed to obtain a plurality of first images in different frequency bands of the image to be processed, wherein each frequency band includes a first image.

[0054] The first processing is used to obtain an image of a specific frequency band of the image to be processed, that is, the first image. Within the original frequency band span of the image to be processed, the first image of multiple frequency bands of the image to be processed can be extracted. In the application, the frequency band range and the number of frequency bands to be obtained can be determined based on actual needs.

[0055] The first processing is performed based on the image to be processed, and may include the following processing operations: image downsampling, image upsampling, image data subtraction processing, etc. How to obtain multiple first images of different frequency bands based on the image to be processed will be described in detail in the following embodiments, and will not be described in detail here.

[0056] Step 103: performing image processing of each frequency band in turn according to a set processing order, and finally obtaining a target image that incorporates image processing results of each frequency band. The configuration of image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

[0057] The frequency bands of the multiple first images obtained above are different. In the application, the image processing of each frequency band can be performed in sequence according to a set order, such as from low to high frequency bands, or from high to low frequency bands. The image processing results of each frequency band will be fused into the final target image to ensure that the image processing performed on each frequency band can only present the corresponding function or effect in the final target image. How to fuse the image processing results of each frequency band will be described in detail in the following embodiments.

[0058] The image processing performed in each frequency band is at least partially different, and the difference here can be any of the following: the type of processing operation enabled, the processing parameters of the same type of processing operation, and the execution order of the processing operations. That is, the image processing of each frequency band may include multiple processes, and the configurations of the image processing of different frequency bands are different, and the configurations include the type of processing operation, the processing order of the processing operation, the processing parameters, etc.

[0059] The present application scheme performs pyramid-style frequency band layering processing on the image to be processed, and then performs image processing on the first image of each frequency band (layer) respectively. Different from the traditional scheme, in the present application scheme, the configuration of image processing of each frequency band corresponds to the characteristics of the first image corresponding to the frequency band. Based on different processing requirements or processing goals, the processing of each frequency band can be flexibly configured independently. For example, for the first image in the low-frequency band, in order to make the final target image clear, it is necessary to filter out low-frequency noise as much as possible, so it is necessary to focus on spatial domain noise reduction; and for the first image in the high-frequency band, because the details of the image need to be retained, it is necessary to focus on time domain noise reduction.

[0060] The image processing method described in this embodiment only performs pyramid frequency band hierarchical processing once on the processing thread of the image to be processed, and has low requirements on area and power consumption. In the implementation, the features of the first image in different frequency bands can be independently configured for processing each frequency band, and image processing with different focuses can be completed to achieve optimization of the final target image.

[0061] In one implementation, the processing results of the first images of each frequency band may be fused layer by layer and finally aggregated into the image processing result of the last processed frequency band, which is the target image. Figure 2 This is a schematic diagram of the implementation principle of the layer-by-layer fusion image processing method disclosed in the embodiment of this application. Figure 2 Understand the following.

[0062] Figure 2 The architecture shown is implemented by layer-by-layer fusion from low frequency band to high frequency band. The image to be processed is input from the left side, and the first processing is performed to obtain the first image of four frequency band levels, corresponding to the four lines horizontally to the right in the figure; the line at the bottom inputs the first image of the lowest frequency band, and the line at the top inputs the first image of the highest frequency band. Each line (each frequency band) includes multiple processing modules (rectangular boxes on the line), and multiple different types of image processing can be performed on the first image.

[0063] Figure 2 The matrix structure in the upper left corner is the structure for pyramid layering of the image to be processed, see Figure 2 The top horizontal line in the figure shows that after the input image is input, it is first down-sampled (the circle in the first row and first column), and then up-sampled (the circle in the first row and second column) to the right. The size of the up-sampled image is the same as the image to be processed, and then the original image to be processed is subtracted from the up-sampled image to the right (the circle in the first row and third column) to obtain the first image in the high-frequency band; see Figure 2The second horizontal line from top to bottom takes as input the image to be processed after downsampling once, and on this basis, downsampling is first performed (processing of the circle in the second row and first column), and then upsampling is performed to the right (processing of the circle in the second row and second column), and then the image to be processed after downsampling once is subtracted from the image after upsampling in the second row (processing of the circle in the third row and third column) to obtain the first image in the next high frequency band. The method for obtaining the first image in other frequency bands is the same as the above method and will not be introduced one by one.

[0064] Combination Figure 2 , firstly, a series of processing operations (spatial noise reduction, temporal noise reduction, tone mapping, sharpening, etc.) are performed on the first image of the lowest frequency band, and then the processing result is upsampled to the same size as the first image of the second highest frequency band, and then it is fused with the first image of the second highest frequency band as the input of the second highest frequency band, and so on, until the image processing of the highest frequency band is completed, and the final target image processing result is obtained. Based on this, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing objects of other frequency bands are the fused image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

[0065] In other implementations, the image processing of each frequency band can also be independent of each other and unrelated, and the image processing results of each frequency band are finally synchronously integrated into the final target image. This application does not impose a fixed restriction on the integration method of the image processing results of each frequency band.

[0066] Thus, the image processing results of each frequency band are integrated into the final target image, ensuring that the image processing performed in each frequency band can only present corresponding functions or effects in the final target image.

[0067] Figure 2 The bottom bar corresponds to the time domain noise reduction processing module. Figure 2 In the example of , the time domain noise reduction processing module is a multiplexing module, and the time domain noise reduction processing of each frequency band is completed by the time domain noise reduction processing module. The solid line between the time domain noise reduction processing module and the first image transmission line represents the default connection relationship, and the dotted line represents the switchable connection relationship. Figure 2, taking the lowest frequency band line as an example, when the default connection relationship is adopted, the time domain noise reduction processing is performed before the spatial domain noise reduction processing (the first rectangular processing module on the left), and when the connection position corresponding to the dotted line is adopted, the spatial domain noise reduction processing is performed before the time domain noise reduction processing, thereby achieving adjustable execution order of different types of processing operations in image processing of the same frequency band. Of course, in other implementations, the solid line and dotted line between the time domain noise reduction processing module and the first image transmission line can be set at different positions based on application requirements to achieve adjustment of the execution order of different processing modules. In addition, other processing modules other than the time domain noise reduction processing module can also be set as multiplexing modules, and the connection positions of the solid lines and dotted lines between them and the first image transmission line can also be designed based on requirements.

[0068] In one implementation, before performing image processing of each frequency band in sequence according to a set processing order, the method may further include: obtaining a processing target of the image to be processed; and determining configuration parameters of image processing of each frequency band based on the processing target.

[0069] In some scenarios, there may be clear processing goals for images so that the processed images can achieve the desired effect. For example, for outdoor images with halo, it is necessary to focus on removing halo. The color tone and contrast can be optimized in a specific frequency band to greatly reduce the halo effect in the final target image.

[0070] Figure 3 The following is a schematic diagram comparing the effects of performing the same processing operation on different frequency bands disclosed in the embodiments of the present application. For example, the pyramid is divided into four frequency band layers, with the 0th layer corresponding to the highest frequency band, followed by the 2nd layer, the 4th layer, and the 8th layer. Figure 3 As shown in the figure, the middle picture shows the effect of UV noise reduction processing at the bottom layer of the pyramid (layer 0), and the right picture shows the effect of UV noise reduction processing at the second layer of the pyramid; it can be seen that the left picture has more large blocks of color noise, and the right picture may have more fine noise. In order to avoid large blocks of color noise, the processing parameters of each frequency band can be configured based on experience.

[0071] In applications with processing targets, because there are clear image processing effects as a guide, each frequency band can be configured according to the processing target. Different frequency bands can achieve different processing effects. Finally, the target image that integrates the processing results of all frequency bands can also match the processing target well.

[0072] In another implementation, before performing image processing of each frequency band in sequence according to the set processing order, it can also include: performing image recognition on the image to be processed to determine the key features of the image to be processed; determining a matching configuration strategy based on the key features; and determining configuration parameters for image processing of each frequency band based on the configuration strategy.

[0073] Among them, the key features of the image to be processed may be, but are not limited to, image scenes and image themes. Different configuration strategies are provided for different image scenes or image themes to optimize image processing. In the implementation summary, multiple configuration strategies may be preset, and after the key features of the image are identified, a configuration strategy that best matches the application may be selected from the preset multiple configuration strategies. Alternatively, in the implementation, after the key features of the image to be processed are identified, the appropriate configuration strategy may also be determined by real-time calculation through an algorithm.

[0074] Figure 4 This is a schematic diagram comparing the effects of different processing configurations for specific scenarios disclosed in the embodiments of this application. Figure 4 As shown in the figure, processing configuration combination 1: adjust the module strength and other parameters of each frequency band layer; combination 2: swap the order of spatial noise reduction / temporal noise reduction / sharpening. Figure 4 , the processing configuration of combination 1 retains more details and textures, and combination 2 makes the entire image look neater, but sacrifices some details.

[0075] In order to better understand the technical solution of the present application, the image processing content in night scene mode is introduced below. Figure 5 This is an example diagram of the processing operation of a frequency band of a night scene image disclosed in this embodiment. Figure 5 , the noise is usually very large in night scenes. In order to suppress the noise as much as possible, highlight the image details and contrast, etc., to restore the real effect of the image, the first image of the highest frequency band layer G0 of the pyramid framework can be configured as follows: In the image processing of the first image of the G0 layer, spatial noise reduction is performed first, and then temporal noise reduction is performed, so as to maintain a balance between high-frequency noise and details, and make the high-frequency information more stable through temporal superposition; in addition, the sharpening parameters of the G0 layer can be set lower to avoid excessive enhancement of residual noise while improving details.

[0076] For image processing of frequency band layers other than the highest frequency band layer G0, corresponding processing configuration can be performed according to processing requirements. For example, for the first image of the low frequency band layer, time domain noise reduction can be performed first, and then spatial domain noise reduction. In addition, its sharpening parameters can be set relatively high.

[0077] In the above embodiments, the image processing of each frequency band may include image noise reduction processing and image optimization processing. The image noise reduction processing is performed before the image optimization processing. The image processing of each frequency band is configured separately based on the processing target or configuration strategy.

[0078] Specifically, if the image processing of each frequency band includes at least two different types of processing operations, then the image processing of each frequency band is configured separately based on the processing target or configuration strategy, which may include: determining the processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; and determining the processing parameters of the processing operations that need to be enabled. That is, the configuration of each frequency band includes determining which processing operations need to be enabled and which processing operations need to be disabled, and at the same time, the processing parameters of the enabled processing operations need to be determined.

[0079] For example, the image to be processed is divided into two layers, corresponding to the high frequency band and the low frequency band. In order to retain more color details, no color processing is performed on the low frequency band, and color management is performed on the high frequency band. At the same time, based on the requirements of color details, corresponding processing parameters are configured for color management of the high frequency band.

[0080] Figure 6 This is an example of the image post-processing process of the traditional solution. Figure 6 In the second half of ISP (Image Signal Processing), traditional solutions usually perform single-frame pyramid noise reduction first, then time-domain noise reduction, and finally global and local tone mapping and sharpening. However, these processing modules are completely serial. After one module is completely processed, another module is processed. Once the previous module causes some side effects, it is difficult for the following module to remedy them. In addition, because each module processes in different signal frequency bands, the processing methods may be inconsistent, causing some uneven side effects.

[0081] The image processing solution described in the embodiment of the present application, from an algorithmic perspective, enables the processing modules on each frequency band to be flexibly switched on and off and switched positions to achieve optimal algorithmic effects; from a debugging perspective, since each processing module can be specifically set and adjusted for different layers or frequency bands, different details or noise effects can be maintained on different layers, and the contrast and clarity of the image can be flexibly controlled, greatly enriching the effect style of the product.

[0082] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0083] The method is described in detail in the embodiments disclosed in the above-mentioned application. The method of the application can be implemented by various forms of devices. Therefore, the application also discloses a device, and a specific embodiment is given below for detailed description.

[0084] Figure 7 Schematic diagram of the structure of an image processing device disclosed in an embodiment of the present application. Figure 7 As shown, the image processing device 70 may include:

[0085] The image acquisition module 701 is used to obtain an image to be processed.

[0086] The frequency division processing module 702 is used to perform a first process on the image to be processed to obtain a plurality of first images in different frequency bands of the image to be processed, each frequency band including a first image.

[0087] The image processing module 703 is used to perform image processing of each frequency band in sequence according to a set processing order, and the image processing results of each frequency band are integrated into the final target image.

[0088] The configuration of the image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

[0089] In one implementation, the image processing module includes a control submodule and at least two processing submodules, wherein: the control submodule is used to determine the processing submodule that needs to be enabled among the at least two processing submodules, and determine the processing parameters of each processing submodule that needs to be enabled.

[0090] Combination Figure 6 In traditional image post-processing solutions, only the noise reduction part adopts the pyramid frequency band processing method, and other optimization processes are executed in series after the noise reduction process. In this way, the side effects caused by the previous processing are difficult to eliminate in the subsequent processing. At the same time, each processing module performs image processing independently. In this way, in different signal frequency bands of the image, it is easy to cause the overall image effect to be uneven due to inconsistent processing methods.

[0091] Based on the above, the image processing scheme described in this embodiment only performs pyramid frequency band layering processing once on the processing thread of the image to be processed. On the one hand, the area and power consumption requirements are not high, making the scheme easier to implement; on the other hand, the features of the first image in different frequency bands can be independently configured for processing each frequency band in the implementation, so as to maintain different details or noise effects on different frequency band layers, complete image processing with different focuses, and achieve the optimization of the final target image. In one implementation, the image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing objects of other frequency bands are the fusion image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

[0092] In one implementation, the device may also include: a processing configuration module, used to obtain a processing target of the image to be processed before performing image processing of each frequency band in sequence according to a set processing order; and determine configuration parameters for image processing of each frequency band based on the processing target.

[0093] In one implementation, the device may also include: a processing configuration module, used to perform image recognition on the image to be processed and determine the key features of the image to be processed; determine a matching configuration strategy based on the key features; and determine configuration parameters for image processing in each frequency band based on the configuration strategy.

[0094] In one implementation, the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured separately based on the processing target or configuration strategy.

[0095] In one implementation, the image processing of each frequency band includes at least two different types of processing operations, and the processing configuration module can be specifically used to: determine the processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; determine the processing parameters of the processing operations that need to be enabled.

[0096] The specific implementation of the above image processing device and the modules it contains, as well as other possible implementations, can be found in the corresponding parts of the method embodiment, which will not be repeated here.

[0097] Furthermore, the present application discloses an image processing architecture, comprising:

[0098] The layered processing circuit is used to process the obtained image to be processed into a plurality of first images containing different frequency bands.

[0099] At least one group of hierarchical processing circuits is used to perform image processing in each frequency band according to a set processing order. One group of hierarchical processing circuits includes multiple processing sub-modules, and the hierarchical processing circuits meet the requirements of configurable enabling states and processing parameters of different processing sub-modules.

[0100] The fusion processing circuit is used to perform fusion processing on images from different sources so that the final target image is fused with the image processing results of each frequency band.

[0101] In the implementation with only one set of hierarchical processing circuits, each processing module in the hierarchical processing circuit is a multiplexed module, which can perform processing operations in the image processing of any frequency band. For example, after the tone mapping module performs tone mapping processing on the first image of the G2 layer in the image processing of the frequency band layer G2, it then performs tone mapping processing on the first image of the G0 layer in the image processing of the frequency band layer G0. Such a design has the lowest overall hardware cost, but the overall image processing efficiency is relatively low.

[0102] Alternatively, the number of hierarchical processing circuits is the same as the number of frequency band layers (pyramid layers) of the image to be processed, so that each frequency band layer has an independent corresponding set of hierarchical processing circuits available, and each set of hierarchical processing circuits is only responsible for image processing of its corresponding frequency band layer. This design has the highest overall hardware cost, but can effectively ensure processing efficiency.

[0103] Alternatively, in other implementations, the image processing architecture includes two groups of hierarchical processing circuits, so that when one group of hierarchical processing circuits performs image processing of a certain frequency band, the other group of hierarchical processing circuits can pre-set processing parameters, control the enabling or disabling circuits of the processing module, adjust the circuits of the execution order of the processing module, etc. according to the configuration of the next processing frequency band, so as to ensure that after the image processing of the current frequency band is completed, the image processing of the next frequency band can be performed immediately. This design has relatively low hardware cost and can ensure the overall image processing efficiency.

[0104] In other implementations, the image processing architecture may further include: a control circuit connected to the hierarchical processing circuit, wherein the control circuit is used to send a control signal to the at least one group of hierarchical processing circuits when performing image processing in each frequency band based on the configuration of image processing in each frequency band.

[0105] Before image processing is performed in each frequency band, it is necessary to set processing parameters, control the enabling or disabling lines of the processing module, and adjust the lines of the execution order of the processing module according to the determined image processing configuration. This part of the work can be completed by the control circuit.

[0106] In one implementation, the image processing architecture includes at least two groups of hierarchical processing circuits, wherein the at least two groups of hierarchical processing circuits have multiplexed image processing modules, for example, Figure 2 This partial hardware reuse design can reduce the number and volume of hardware structures in the hierarchical processing circuit and save hardware costs.

[0107] In the implementation of the present application, all processing operations that can be executed in each level of processing circuit correspond to an independent processing module, and the image processing of each frequency band layer can be combined by starting the corresponding modules through circuit configuration connection to form a corresponding processing layer. That is, various image processing operations are optional designs, and the corresponding processing module is connected to the processing thread for the type of processing operation that needs to be performed.

[0108] Based on the above, from a hardware perspective, the image processing architecture integrates the entire image post-processing (including noise reduction, image optimization processing, etc.) into a pyramid architecture. The image processing of each frequency band layer of the pyramid can be flexibly configured based on the hierarchical processing circuit, so that different details or noise effects are maintained on different frequency band layers to achieve the optimization of the final target image.

[0109] Any one of the image processing devices described in the above embodiments includes a processor and a memory. The image acquisition module, frequency division processing module, image processing module, control submodule, processing submodule, etc. in the above embodiments are all stored in the memory as program modules, and the processor executes the above program modules stored in the memory to realize corresponding functions.

[0110] The processor includes a kernel, which retrieves the corresponding program module from the memory. One or more kernels can be set, and the processing of the access data can be realized by adjusting the kernel parameters.

[0111] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0112] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above-mentioned image processing method.

[0113] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software codes. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above-mentioned image processing method.

[0114] Furthermore, an embodiment of the present application provides an electronic device. Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. Figure 8As shown, the electronic device 80 includes at least one processor 801, and at least one memory 802 and a bus 803 connected to the processor; wherein the processor and the memory communicate with each other via the bus; the processor is used to call the program instructions in the memory to execute the above-mentioned image processing method.

[0115] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0116] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0117] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0118] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, the method comprising: Obtaining an image to be processed; Performing a first process on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, wherein each frequency band includes a first image; According to the set processing order, the image processing of each frequency band is carried out in turn, and the image processing results of each frequency band are integrated into the final target image; The configuration of the image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

2. The image processing method according to claim 1, wherein: The image processing object of the lowest frequency band is the first image corresponding to the lowest frequency band, and the image processing objects of other frequency bands are the fusion image of the first image corresponding to the current processing frequency band and the processing result of the previous processing frequency band.

3. The image processing method according to claim 1, before sequentially processing the images of each frequency band according to the set processing order, further comprising: Obtaining a processing target of the image to be processed; Configuration parameters of image processing for each frequency band are determined based on the processing target.

4. The image processing method according to claim 1, before sequentially processing the images of each frequency band according to the set processing order, further comprising: Performing image recognition on the image to be processed to determine key features of the image to be processed; Determining a matching configuration strategy based on the key features; The configuration parameters of the image processing in each frequency band are determined based on the configuration strategy.

5. According to the image processing method of claim 3 or 4, the image processing of each frequency band includes image noise reduction processing and image optimization processing, the image noise reduction processing is performed before the image optimization processing, and the image processing of each frequency band is configured separately based on the processing target or configuration strategy.

6. The image processing method according to claim 5, wherein the image processing of each frequency band comprises at least two different types of processing operations, and the image processing of each frequency band is configured separately based on the processing target or configuration strategy, including: Determining processing operations that need to be enabled, the number of the processing operations that need to be enabled is not greater than the number of the at least two different types of processing operations; Determine the processing parameters of the processing operation that needs to be enabled.

7. An image processing device, comprising: An image acquisition module, used for acquiring an image to be processed; A frequency division processing module, used for performing a first process on the image to be processed to obtain a plurality of first images of different frequency bands of the image to be processed, each frequency band including a first image; The image processing module is used to perform image processing of each frequency band in sequence according to a set processing order, and the image processing results of each frequency band are integrated into the final target image; The configuration of the image processing of each frequency band corresponds to the feature of the first image corresponding to the frequency band.

8. The image processing device according to claim 7, wherein the image processing module comprises a control submodule and at least two processing submodules, wherein: The control submodule is used to determine the processing submodule that needs to be enabled among the at least two processing submodules, and to determine the processing parameters of each processing submodule that needs to be enabled.

9. An image processing architecture comprising: A layered processing circuit, used for processing the obtained image to be processed into a plurality of first images including different frequency bands; At least one group of hierarchical processing circuits, used for performing image processing of each frequency band according to a set processing order, wherein one group of hierarchical processing circuits includes a plurality of processing submodules, and the hierarchical processing circuits satisfy that the enabling states and processing parameters of different processing submodules are configurable; The fusion processing circuit is used to perform fusion processing on images from different sources so that the final target image is fused with the image processing results of each frequency band.

10. The image processing architecture according to claim 9, further comprising: A control circuit connected to the hierarchical processing circuit is used to send a control signal to the at least one group of hierarchical processing circuits when performing image processing in each frequency band based on the configuration of the image processing in each frequency band.

11. The image processing architecture according to claim 9, comprising at least two groups of hierarchical processing circuits, wherein the at least two groups of hierarchical processing circuits contain multiplexed image processing modules.

12. An electronic device comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the image processing method according to any one of claims 1 to 7.

Images