Image processing method and device, equipment and storage medium

By combining, decomposing, extracting and filtering the multi-focus image, the chromatic aberration and artifact problems during image fusion in the prior art are solved, and the visual effect is significantly improved.

CN119006304APending Publication Date: 2024-11-22HE UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411150886.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing multifocal image fusion algorithms have poor visual effects when processing images with different focal points, including chromatic aberration and artifact problems.

Method used

By combining multiple images with different focal points, decomposing them into multi-channel images, extracting depth information and smoothing using high-pass filters and mean filters to generate target images for fusing.

Benefits of technology

It significantly improves the image fusion effect, reduces chromatic aberration and artifact problems, and improves the visual effect of the final image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006304A_ABST
    Figure CN119006304A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device, equipment and a storage medium, and the method comprises the steps: carrying out the merging processing of a plurality of images with different focuses, and obtaining a multi-channel image; performing decomposition processing on the multi-channel image to obtain initial images corresponding to a plurality of channels respectively; extracting depth information of the initial images respectively corresponding to the plurality of channels to obtain initial depth images respectively corresponding to the plurality of channels, and performing smoothing processing by using a high-pass filter to obtain depth images respectively corresponding to the plurality of channels; analyzing the initial images respectively corresponding to the plurality of channels based on the depth images respectively corresponding to the plurality of channels, generating intermediate images respectively corresponding to the plurality of channels, and performing smoothing processing by using a mean filter to obtain target images respectively corresponding to the plurality of channels; and performing fusion processing on the target images corresponding to the plurality of channels to obtain an image processing result. Therefore, the image fusion effect can be remarkably improved, and the visual effect of the target image obtained through fusion is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] Multi-focus image fusion algorithm is an image processing technology that aims to fuse multiple images with different focuses into one image to obtain a clearer and more comprehensive visual effect.

[0003] However, in practical applications, existing multi-focus image fusion algorithms still have defects when processing images with different focuses, resulting in poor visual effects of the images. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, apparatus, device and storage medium to improve the visual effect of a fused image.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0006] Merge multiple images with different focuses to obtain a multi-channel image;

[0007] Decomposing the multi-channel image to obtain initial images corresponding to the multiple channels;

[0008] Extracting depth information of initial images respectively corresponding to the multiple channels to obtain initial depth images respectively corresponding to the multiple channels, and smoothing the initial depth images respectively corresponding to the multiple channels using a high-pass filter to obtain depth images respectively corresponding to the multiple channels;

[0009] Analyzing the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively, generating intermediate images corresponding to the multiple channels respectively, and smoothing the intermediate images corresponding to the multiple channels respectively by using a mean filter to obtain target images corresponding to the multiple channels respectively;

[0010] The target images corresponding to the multiple channels are fused to obtain image processing results.

[0011] Optionally, the decomposing the multi-channel image to obtain initial images corresponding to the multiple channels respectively includes:

[0012] The multi-channel image is decomposed based on multiple color channels to obtain a first initial image corresponding to a red channel, a second initial image corresponding to a green channel, and a third initial image corresponding to a blue channel.

[0013] Optionally, for each pixel position in the initial image corresponding to one of the multiple channels, extracting the depth information of the initial images respectively corresponding to the multiple channels to obtain the initial depth images respectively corresponding to the multiple channels includes:

[0014] Acquire a plurality of grayscale values ​​of each pixel position in a focus sequence, wherein the focus sequence is the plurality of images with different focuses arranged in layers, and the plurality of grayscale values ​​of each pixel in the focus sequence include the grayscale values ​​of each pixel position in the plurality of images with different focuses respectively;

[0015] Based on a plurality of grayscale values ​​of each pixel position in the focus sequence, determining the layer number of the image where the maximum grayscale value of each pixel position is located, and using the layer number as the layer number of the depth image of each pixel position;

[0016] Based on the number of layers of the depth image at each pixel position, an initial depth image corresponding to the one channel is generated.

[0017] Optionally, for each pixel position in an initial image corresponding to one of the multiple channels, analyzing the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively to generate intermediate images corresponding to the multiple channels respectively includes:

[0018] Based on the depth image corresponding to the one channel, determining the maximum grayscale value of each pixel position from the initial image corresponding to the one channel as the target grayscale value of each pixel position;

[0019] An intermediate image corresponding to the one channel is generated based on the target grayscale value.

[0020] Optionally, before decomposing the multi-channel image according to image channels, the image processing method further includes:

[0021] Reducing the size of the multi-channel image to obtain a reduced multi-channel image;

[0022] The decomposing the multi-channel image according to the image channels comprises:

[0023] The reduced multi-channel image is decomposed according to image channels.

[0024] Optionally, the multiple images with different focal points are obtained by the following steps:

[0025] Sending different focus signals to the focus-adjustable lens in sequence, so that the focus-adjustable lens responds to the different focus signals in sequence, adjusts the focus of the focus-adjustable lens and captures an image using the adjusted focus;

[0026] Images captured by the adjustable focus lens at the different focal points are acquired.

[0027] Optionally, the sending different focus signals to the focus-adjustable lens in sequence includes:

[0028] The motor movement step length and image capture amount corresponding to the adjustable focus lens are set to 0;

[0029] When the image capture amount is less than or equal to a preset maximum capture amount, a focus signal is sent to the focus-adjustable lens based on a preset motor movement step, and the image capture amount is increased by 1, and the cycle is executed until the image capture amount is greater than the maximum capture amount.

[0030] In a second aspect, an embodiment of the present application provides an image processing device, including:

[0031] An image merging module is used to merge multiple images with different focal points to obtain a multi-channel image;

[0032] An image decomposition module, used for decomposing the multi-channel image to obtain initial images corresponding to the multiple channels;

[0033] A depth extraction module, used to extract depth information of the initial images corresponding to the multiple channels, obtain the initial depth images corresponding to the multiple channels, and use a high-pass filter to smooth the initial depth images corresponding to the multiple channels, to obtain the depth images corresponding to the multiple channels;

[0034] An image analysis module, configured to analyze the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively, generate intermediate images corresponding to the multiple channels respectively, and use a mean filter to smooth the intermediate images corresponding to the multiple channels respectively, so as to obtain target images corresponding to the multiple channels respectively;

[0035] The image fusion module is used to perform fusion processing on the target images corresponding to the multiple channels to obtain an image processing result.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor, a memory, and a system bus;

[0037] The processor and the memory are connected via the system bus;

[0038] The memory is used to store a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes any implementation step of the above-mentioned image processing method.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes any implementation step of the above-mentioned image processing method.

[0040] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0041] In an embodiment of the present application, after merging multiple images with different focuses to obtain a multi-channel image, the multi-channel image can be decomposed to obtain initial images corresponding to multiple channels. Then, the depth information of the initial images corresponding to the multiple channels can be extracted to obtain initial depth images corresponding to the multiple channels, and the initial depth images corresponding to the multiple channels can be smoothed using a high-pass filter to obtain depth images corresponding to the multiple channels. Then, the initial images corresponding to the multiple channels can be analyzed based on the depth images corresponding to the multiple channels, and intermediate images corresponding to the multiple channels can be generated, and the intermediate images corresponding to the multiple channels can be smoothed using a mean filter to obtain target images corresponding to the multiple channels. Finally, the target images corresponding to the multiple channels can be fused to obtain image processing results. It can be seen that the initial depth image can be smoothed with the help of a high-pass filter to remove the low-frequency components in the initial depth image and enhance the high-frequency components, unify the image color, and thus reduce the color difference problem of the final image processing result. With the help of the mean filter, the intermediate image can be smoothed, blurring the focal area and adjacent areas of the intermediate image, thereby reducing the artifact problem of the target image. In this way, the image fusion effect can be significantly improved, and the visual effect of the final image processing result can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a process for acquiring multiple images with different focal points provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of an implementation process of sequentially sending different focus signals to a focus-adjustable lens provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a layer implementation process for obtaining a depth image provided by an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the effect of an image processing method provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] As mentioned above, in practical applications, existing multi-focus image fusion algorithms still have defects when processing images with different focuses, resulting in poor visual effects of the images.

[0049] Specifically, in existing image fusion algorithms, the color deviation between images with different focuses may not be taken into account, or unnecessary color shift may be introduced during the fusion process, resulting in image chromatic aberration problems.

[0050] In addition, the final fused image may have artifacts, which are unrealistic shadows or halos that appear around the focus switching area of ​​the image. This is usually caused by the failure of the multi-focus image fusion algorithm to fully and accurately capture and process the transition area between images with different focuses. Artifacts will make the transition area of ​​the image look unnatural, affecting the overall visual effect and realism of the image, especially in high-contrast scenes or areas with rich details, where artifacts may be more obvious and prominent.

[0051] Based on this, in order to solve the above problems, an embodiment of the present application provides an image processing method, including: after merging multiple images with different focuses to obtain a multi-channel image, the multi-channel image can be decomposed to obtain initial images corresponding to multiple channels. Then, the depth information of the initial images corresponding to the multiple channels can be extracted to obtain initial depth images corresponding to the multiple channels, and the initial depth images corresponding to the multiple channels can be smoothed using a high-pass filter to obtain depth images corresponding to the multiple channels. Then, the initial images corresponding to the multiple channels can be analyzed based on the depth images corresponding to the multiple channels to generate intermediate images corresponding to the multiple channels, and the intermediate images corresponding to the multiple channels can be smoothed using a mean filter to obtain target images corresponding to the multiple channels. Finally, the target images corresponding to the multiple channels can be fused to obtain image processing results.

[0052] It can be seen that the high-pass filter can be used to smooth the initial depth image to remove the low-frequency components in the initial depth image and enhance the high-frequency components, unify the image color, and thus reduce the color difference problem of the final image processing result. The mean filter can be used to smooth the intermediate image, blur the focal area and adjacent areas of the intermediate image, thereby reducing the artifact problem of the target image. In this way, the image fusion effect can be significantly improved, and the visual effect of the final image processing result can be improved.

[0053] It should be noted that the embodiments of the present application may not limit the execution subject of the image processing method. For example, the image processing method of the embodiments of the present application may be applied to data processing devices such as terminal devices or servers. The terminal device may be an electronic device such as a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server may be an independent server, a cluster server, or a cloud server.

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application. Figure 1 As shown, the image processing method provided in the embodiment of the present application is implemented by using an image processing device as the execution subject to describe the scheme. The image processing method may include:

[0056] S101: merging multiple images with different focal points to obtain a multi-channel image.

[0057] In the embodiment of the present application, multiple images with different focal points refer to images in which only the image distance, i.e., the distance between the lens optical center and the film plane, is changed during shooting, without changing other parameters. Based on this, for ease of understanding, the steps of obtaining the above multiple images with different focal points are explained below with examples and drawings.

[0058] Figure 2 A flowchart of a process for acquiring multiple images with different focal points provided in an embodiment of the present application. Figure 2 As shown, the above multiple images with different focal points are obtained through the following steps 1-2:

[0059] Step 1: sending different focus signals to the focus-adjustable lens in sequence, so that the focus-adjustable lens responds to the different focus signals in sequence, adjusts the focus of the focus-adjustable lens and captures an image using the adjusted focus.

[0060] In practical applications, the above-mentioned adjustable focus lens can be an adjustable focus lens that imitates the human eye lens. Among them, the adjustable focus lens that imitates the human eye lens is a liquid lens, which can set an organic film in a shell, so that the organic film, the front lens and the shell form a sealed liquid cavity for containing liquid, and the organic film, the rear lens and the shell form a sealed gas cavity, and an air inlet connected to the gas cavity is set outside the shell. In this way, gas can be injected into the gas cavity through the air inlet, so that the organic film is deformed into the liquid cavity, thereby changing the thickness of the liquid in the liquid cavity, and then realizing the change of the focal length of the lens.

[0061] Accordingly, the human eye lens-like adjustable focus lens can quickly and accurately achieve continuous zooming, thereby meeting the hardware requirements for collecting multiple images with different focal points. In addition, the human eye lens-like adjustable focus lens has better optical performance, is less likely to produce dispersion problems and optical distortion, and is more conducive to subsequent accurate image fusion.

[0062] It should be noted that the above process of sending the focus signal to the focus-adjustable lens can be directly sent to the focus-adjustable lens by the image processing device. Alternatively, the image processing device can first send a control instruction to the focus controller to trigger the focus controller to respond to the control instruction and send the focus signal to the focus-adjustable lens. This embodiment of the present application is not specifically limited.

[0063] Further, Figure 3 A schematic diagram of an implementation process of sequentially sending different focus signals to a focus-adjustable lens provided in an embodiment of the present application. Figure 3 As shown, in the implementation process of sending different focus signals to the adjustable focus lens, the parameters may be initialized first, that is, the motor moving step length λ and the image capture amount I corresponding to the adjustable focus lens are set to 0.

[0064] Next, the maximum shooting volume I of the adjustable focus lens can be set. max , and the motor movement step size for each focus adjustment.

[0065] In this way, when the image capture amount I is less than or equal to the preset maximum capture amount I max In this case, a focus signal can be sent to the focus-adjustable lens based on the above-mentioned preset motor movement step length, and the image capture amount I is increased by 1. At this time, the motor of the focus-adjustable lens will respond to the focus signal and move according to the motor movement step length, thereby controlling the movement of the focus-adjustable lens.

[0066] Next, it can be determined whether the image capture amount I is less than or equal to the maximum capture amount I max If the image capture amount I is still less than or equal to the maximum capture amount I max, the focus signal can be continuously sent to the focus lens based on the above-preset motor movement step length, and the image capture amount I can be increased by 1. Accordingly, the motor of the focus lens will continue to respond to the focus signal sent at this time, and continue to move according to the motor movement step length, thereby controlling the movement of the focus lens. This cycle is executed until it is continuously determined whether the image capture amount I is less than or equal to the maximum capture amount I max When the image capture amount I is greater than the maximum capture amount I max , end focusing.

[0067] Step 2: Obtain images captured by the tunable focus lens at different focal points.

[0068] As mentioned above, the image processing device can directly send a focus signal to the focus-adjustable lens, or the image processing device can first send a control instruction to the focus controller to trigger the focus controller to respond to the control instruction and send a focus signal to the focus-adjustable lens. Correspondingly, the focus-adjustable lens can also directly send images captured at different focuses to the image processing device, or the focus-adjustable lens first sends images captured at different focuses to the focus controller, and then the focus controller sends these images to the image processing device.

[0069] In addition, a multi-channel image refers to an image with multiple channels, each of which can represent a specific color channel in the image. For example, a red green blue (RGB) image is composed of an image corresponding to the red channel, an image corresponding to the blue channel, and an image corresponding to the green channel.

[0070] Here, the implementation process of the above step S101, that is, merging multiple images with different focal points to obtain a multi-channel image, may not be specifically limited in the present embodiment of the application. For example, any existing or future image processing algorithm capable of realizing channel merging may be used for implementation.

[0071] S102: Decompose the multi-channel image to obtain initial images corresponding to the multiple channels.

[0072] In practical applications, when performing image fusion, a large amount of image processing and computing operations are usually required. Especially when processing high-resolution images, the algorithm needs to process a large number of pixels and complex image features, so a large amount of memory and computing resources are required to support it. In addition, due to the high complexity of the algorithm, multi-focus image fusion often requires a long processing time, and the processing speed requirements may not be met.

[0073] Based on this, in the embodiment of the present application, before executing step S102, the multi-channel image may be first reduced in size to obtain a reduced multi-channel image. In a specific implementation, the size of the multi-channel image may be reduced to 640 pixels × 480 pixels, which can significantly reduce the computing resources and time required for image processing without significantly affecting the image quality, thereby improving processing efficiency and system response speed.

[0074] Next, when executing step S102, that is, in the process of decomposing the multi-channel image, the reduced multi-channel image can be decomposed according to the image channels.

[0075] Accordingly, in the embodiment of the present application, the decomposition process of the multi-channel image can be reflected as decomposing a multi-channel image into initial images corresponding to the red channel, the green channel, and the blue channel, respectively. Based on this, the above step S102 can include: decomposing the multi-channel image based on multiple color channels to obtain a first initial image corresponding to the red channel, a second initial image corresponding to the green channel, and a third initial image corresponding to the blue channel.

[0076] S103: extracting depth information of initial images corresponding to the multiple channels to obtain initial depth images corresponding to the multiple channels, and smoothing the initial depth images corresponding to the multiple channels using a high-pass filter to obtain depth images corresponding to the multiple channels.

[0077] In the embodiment of the present application, for ease of understanding, an exemplary explanation is given below by taking obtaining an initial depth image corresponding to a channel as an example.

[0078] As a possible implementation, for each pixel position in the initial image corresponding to one of the multiple channels, the above step S103, when implemented specifically, first, multiple grayscale values ​​of each pixel position in the focus sequence may be obtained. The focus sequence is a plurality of images with different focuses arranged in layers, and the multiple grayscale values ​​of each pixel in the focus sequence include the grayscale values ​​of each pixel position in the plurality of images with different focuses. That is, each pixel position has a grayscale value in the above images with different focuses.

[0079] Next, based on the multiple grayscale values ​​of each pixel position in the focus sequence, the layer number of the image where the maximum grayscale value of each pixel position is located can be determined, and the layer number is used as the layer number of the depth image of each pixel position. For ease of understanding, the following is an explanation with reference to the accompanying drawings.

[0080] Figure 4 A schematic diagram of a layer implementation process for obtaining a depth image provided by an embodiment of the present application. Figure 4As shown, taking the black part as an example to represent a pixel position, the grayscale value of the pixel position in the first layer image in the focus sequence is 0, the grayscale value of the second layer image is 43, the grayscale value of the third layer image is 58, and the grayscale value of the fourth layer image is 43. The grayscale value of the pixel position in each layer image is continuously obtained until the grayscale value of the pixel position in the nth layer image is 0. It can be determined that the layer number of the image where the maximum grayscale value 58 of the pixel position is located is 3. Therefore, the layer number of the pixel position of the black part in the depth image is also 3.

[0081] Then, an initial depth image corresponding to a channel may be generated based on the number of layers of the depth image at each pixel position. Here, the process of generating the initial depth image corresponding to the channel may be specifically embodied as synthesizing the initial depth image corresponding to the channel based on the number of layers of the depth image at each pixel position.

[0082] Furthermore, in an embodiment of the present application, after obtaining the initial depth portraits corresponding to the multiple channels, the initial depth images corresponding to the multiple channels can be smoothed using a high-pass filter to eliminate chromatic aberration, thereby obtaining depth images corresponding to the multiple channels. The kernel size of the high-pass filter is 15 pixels × 15 pixels. In this way, the image can be smoothed with the help of a high-pass filter, the low-frequency components in the image can be removed and the high-frequency components can be enhanced, the image color can be unified, and the contrast of the focus change can be improved, so that the changes in different depth planes are more obvious, which helps to more accurately obtain the depth images corresponding to the multiple channels and solve the chromatic aberration problem of the image finally fused.

[0083] S104: Analyze the initial images corresponding to the multiple channels based on the depth images corresponding to the multiple channels, generate intermediate images corresponding to the multiple channels, and use a mean filter to smooth the intermediate images corresponding to the multiple channels to obtain target images corresponding to the multiple channels.

[0084] As mentioned above, in the process of obtaining the initial depth image corresponding to each channel, the layer number of the image where the maximum pixel value of each pixel position in the initial image of each channel is located can be determined first. Based on this, in order to obtain clear images corresponding to multiple channels respectively for image fusion, in an embodiment of the present application, the intermediate image corresponding to each channel can be generated with the help of the maximum pixel value of each pixel position. For ease of understanding, the following is an exemplary explanation taking the generation of an intermediate image corresponding to a channel as an example.

[0085] As a possible implementation, for each pixel position in the initial image corresponding to one of the multiple channels, the above step S104 can be implemented by first determining the maximum grayscale value of each pixel position from the initial image corresponding to one channel based on the depth image corresponding to one channel as the target grayscale value of each pixel position. Here, the maximum grayscale value of a pixel position can represent the clearest pixel grayscale, so using it as the target grayscale value of the pixel position is helpful to generate a clear intermediate image.

[0086] Next, an intermediate image corresponding to a channel can be generated based on the target grayscale value. Here, the process of generating an intermediate image can be specifically reflected in generating a corresponding intermediate image based on the target grayscale value of each pixel position. In practical applications, when capturing an image, a clear image will be captured only when the focus is accurately aligned with a certain point in the scene, and when the focus is aligned with other positions, the image of the point will be blurred. Therefore, by using the maximum grayscale value of each pixel position as the target grayscale value and generating an intermediate image based on this, it is helpful to obtain a clear intermediate image.

[0087] It should be noted that the implementation process of generating the intermediate image corresponding to the channel based on the target grayscale value in the embodiment of the present application may not be specifically limited. For example, any existing or future algorithm that can generate an image based on grayscale values ​​may be used for implementation.

[0088] Furthermore, in the embodiment of the present application, after obtaining the intermediate images corresponding to the multiple channels, the intermediate images corresponding to the multiple channels can be smoothed using a mean filter to eliminate artifacts, thereby obtaining target images corresponding to the multiple channels. The kernel size of the mean filter is 51 pixels × 51 pixels. In this way, the focus area and the adjacent pixels can be blurred by the mean filter, thereby bridging the transition area between the two images, thereby reducing the artifact problem in image fusion.

[0089] S105: Perform fusion processing on the target images corresponding to the multiple channels to obtain an image processing result.

[0090] Here, the implementation process of the above step S105, that is, fusing the intermediate images corresponding to the multiple channels to obtain the image processing result, may not be specifically limited in the present embodiment of the application. For example, any existing or future algorithm capable of realizing image fusion may be used for implementation.

[0091] Based on the relevant contents of the above steps S101-S105, it can be known that in an embodiment of the present application, after merging multiple images with different focuses to obtain a multi-channel image, the multi-channel image can be decomposed to obtain initial images corresponding to multiple channels. Then, the depth information of the initial images corresponding to the multiple channels can be extracted to obtain initial depth images corresponding to the multiple channels, and the initial depth images corresponding to the multiple channels can be smoothed using a high-pass filter to obtain depth images corresponding to the multiple channels. Then, the initial images corresponding to the multiple channels can be analyzed based on the depth images corresponding to the multiple channels, and intermediate images corresponding to the multiple channels can be generated, and the intermediate images corresponding to the multiple channels can be smoothed using a mean filter to obtain target images corresponding to the multiple channels. Finally, the target images corresponding to the multiple channels can be fused to obtain image processing results. It can be seen that the initial depth image can be smoothed with the help of a high-pass filter to remove the low-frequency components in the initial depth image and enhance the high-frequency components, unify the image color, and thus reduce the color difference problem of the final image processing result. With the help of the mean filter, the intermediate image can be smoothed, blurring the focal area and adjacent areas of the intermediate image, thereby reducing the artifact problem of the target image. In this way, the image fusion effect can be significantly improved, and the visual effect of the final image processing result can be improved.

[0092] Further, Figure 5 A schematic diagram of the effect of an image processing method provided in an embodiment of the present application. Figure 5 As shown, 40 images at different focal positions can be fused to obtain a target image in 2.3042 seconds by using the image processing method provided in the embodiment of the present application, thereby better reducing the operation time and the requirements for computing resources. In addition, the artifact and color difference problems of the target image finally fused are reduced, which significantly improves the visual effect of the image.

[0093] Furthermore, based on the image processing method provided in the above embodiment, the embodiment of the present application may also provide an image processing device. The image processing device is described below in conjunction with the embodiments and drawings.

[0094] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Figure 6 As shown, the image processing device 600 provided in the embodiment of the present application may include:

[0095] An image merging module 601 is used to merge multiple images with different focal points to obtain a multi-channel image;

[0096] An image decomposition module 602 is used to decompose the multi-channel image to obtain initial images corresponding to multiple channels respectively;

[0097] A depth extraction module 603 is used to extract depth information of the initial images corresponding to the multiple channels, obtain the initial depth images corresponding to the multiple channels, and use a high-pass filter to smooth the initial depth images corresponding to the multiple channels, to obtain the depth images corresponding to the multiple channels;

[0098] An image analysis module 604 is used to analyze the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively, generate intermediate images corresponding to the multiple channels respectively, and use a mean filter to smooth the intermediate images corresponding to the multiple channels respectively to obtain target images corresponding to the multiple channels respectively;

[0099] The image fusion module 605 is used to perform fusion processing on the target images corresponding to the multiple channels to obtain an image processing result.

[0100] As an implementation manner, the image decomposition module 602 is specifically used for:

[0101] The multi-channel image is decomposed based on multiple color channels to obtain a first initial image corresponding to a red channel, a second initial image corresponding to a green channel, and a third initial image corresponding to a blue channel.

[0102] As an implementation manner, for each pixel position in the initial image corresponding to one of the multiple channels, the depth extraction module 603 is specifically configured to:

[0103] Acquire a plurality of grayscale values ​​of each pixel position in a focus sequence, wherein the focus sequence is the plurality of images with different focuses arranged in layers, and the plurality of grayscale values ​​of each pixel in the focus sequence include the grayscale values ​​of each pixel position in the plurality of images with different focuses respectively;

[0104] Based on a plurality of grayscale values ​​of each pixel position in the focus sequence, determining the layer number of the image where the maximum grayscale value of each pixel position is located, and using the layer number as the layer number of the depth image of each pixel position;

[0105] Based on the number of layers of the depth image at each pixel position, an initial depth image corresponding to the one channel is generated.

[0106] As an implementation manner, for each pixel position in the initial image corresponding to one of the multiple channels, the image analysis module 604 is specifically configured to:

[0107] Based on the depth image corresponding to the one channel, determining the maximum grayscale value of each pixel position from the initial image corresponding to the one channel as the target grayscale value of each pixel position;

[0108] An intermediate image corresponding to the one channel is generated based on the target grayscale value.

[0109] As an implementation manner, the image processing device 600 further includes:

[0110] An image reduction module, used for reducing the size of the multi-channel image to obtain a reduced multi-channel image;

[0111] The image decomposition module 602 is specifically used for:

[0112] The reduced multi-channel image is decomposed according to image channels.

[0113] As an implementation manner, the multiple images with different focal points are acquired through the following modules:

[0114] A signal sending module, used for sending different focus signals to the focus-adjustable lens in sequence, so that the focus-adjustable lens responds to the different focus signals in sequence, adjusts the focus of the focus-adjustable lens and captures an image using the adjusted focus;

[0115] The image acquisition module is used to acquire the images captured by the adjustable focus lens at the different focal points.

[0116] As an implementation manner, the signal sending module is specifically used to:

[0117] The motor movement step length and image capture amount corresponding to the adjustable focus lens are set to 0;

[0118] When the image capture amount is less than or equal to a preset maximum capture amount, a focus signal is sent to the focus-adjustable lens based on a preset motor movement step, and the image capture amount is increased by 1, and the cycle is executed until the image capture amount is greater than the maximum capture amount.

[0119] Furthermore, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory, and a system bus;

[0120] The processor and the memory are connected via the system bus;

[0121] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any implementation step of the above-mentioned image processing method.

[0122] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on an electronic device, any step of the above-mentioned image processing method is implemented.

[0123] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in each embodiment of the present application or certain parts of the embodiments. It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

[0124] As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0125] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0126] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: include: Merge multiple images with different focuses to obtain a multi-channel image; Decomposing the multi-channel image to obtain initial images corresponding to the multiple channels; Extracting depth information of initial images respectively corresponding to the multiple channels to obtain initial depth images respectively corresponding to the multiple channels, and smoothing the initial depth images respectively corresponding to the multiple channels using a high-pass filter to obtain depth images respectively corresponding to the multiple channels; Analyzing the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively, generating intermediate images corresponding to the multiple channels respectively, and smoothing the intermediate images corresponding to the multiple channels respectively by using a mean filter to obtain target images corresponding to the multiple channels respectively; The target images corresponding to the multiple channels are fused to obtain image processing results.

2. The image processing method according to claim 1, characterized in that: The decomposing the multi-channel image to obtain initial images corresponding to the multiple channels respectively includes: The multi-channel image is decomposed based on multiple color channels to obtain a first initial image corresponding to a red channel, a second initial image corresponding to a green channel, and a third initial image corresponding to a blue channel.

3. The image processing method according to claim 1, characterized in that: For each pixel position in the initial image corresponding to one of the multiple channels, extracting the depth information of the initial images respectively corresponding to the multiple channels to obtain the initial depth images respectively corresponding to the multiple channels includes: Acquire a plurality of grayscale values ​​of each pixel position in a focus sequence, wherein the focus sequence is the plurality of images with different focuses arranged in layers, and the plurality of grayscale values ​​of each pixel in the focus sequence include the grayscale values ​​of each pixel position in the plurality of images with different focuses respectively; Based on a plurality of grayscale values ​​of each pixel position in the focus sequence, determining the layer number of the image where the maximum grayscale value of each pixel position is located, and using the layer number as the layer number of the depth image of each pixel position; Based on the number of layers of the depth image at each pixel position, an initial depth image corresponding to the one channel is generated.

4. The image processing method according to claim 3, characterized in that: For each pixel position in the initial image corresponding to one of the multiple channels, analyzing the initial images corresponding to the multiple channels based on the depth images corresponding to the multiple channels to generate intermediate images corresponding to the multiple channels, including: Based on the depth image corresponding to the one channel, determining the maximum grayscale value of each pixel position from the initial image corresponding to the one channel as the target grayscale value of each pixel position; An intermediate image corresponding to the one channel is generated based on the target grayscale value.

5. The image processing method according to claim 1, characterized in that: Before decomposing the multi-channel image according to image channels, the image processing method further includes: Reducing the size of the multi-channel image to obtain a reduced multi-channel image; The decomposing the multi-channel image according to the image channels comprises: The reduced multi-channel image is decomposed according to image channels.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The multiple images with different focal points are obtained by the following steps: Sending different focus signals to the focus-adjustable lens in sequence, so that the focus-adjustable lens responds to the different focus signals in sequence, adjusts the focus of the focus-adjustable lens and captures an image using the adjusted focus; Images captured by the adjustable focus lens at the different focal points are acquired.

7. The image processing method according to claim 6, characterized in that: The sending different focus signals to the focus-adjustable lens in sequence includes: The motor movement step length and image capture amount corresponding to the adjustable focus lens are set to 0; When the image capture amount is less than or equal to a preset maximum capture amount, a focus signal is sent to the focus-adjustable lens based on a preset motor movement step, and the image capture amount is increased by 1, and the cycle is executed until the image capture amount is greater than the maximum capture amount.

8. An image processing device, characterized in that: include: An image merging module is used to merge multiple images with different focal points to obtain a multi-channel image; An image decomposition module, used for decomposing the multi-channel image to obtain initial images corresponding to the multiple channels; A depth extraction module, used to extract depth information of the initial images corresponding to the multiple channels, obtain the initial depth images corresponding to the multiple channels, and use a high-pass filter to smooth the initial depth images corresponding to the multiple channels, to obtain the depth images corresponding to the multiple channels; An image analysis module, configured to analyze the initial images corresponding to the multiple channels respectively based on the depth images corresponding to the multiple channels respectively, generate intermediate images corresponding to the multiple channels respectively, and smooth the intermediate images corresponding to the multiple channels respectively using a mean filter to obtain target images corresponding to the multiple channels respectively; The image fusion module is used to perform fusion processing on the target images corresponding to the multiple channels to obtain an image processing result.

9. An electronic device, characterized in that: The device includes: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the steps of the image processing method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a terminal device, the steps of the image processing method according to any one of claims 1 to 5 are implemented.