Image processing method and apparatus therefor

Abstract

The application discloses an image processing method and device, and belongs to the field of image processing. The method comprises the following steps: acquiring N first images and N second images, the N first images are images of a first region in N frames of collected images, the N second images are images of a second region in the N frames of images, the first region is a region where a static object is located in the images, the second region is a region where a moving object is located in the images, and N is a positive integer; performing convolution operation processing on each of the N second images respectively to obtain N third images; and generating a target image according to the N first images and the N third images, the target image being an image with an analog exposure time length greater than or equal to an exposure time length threshold.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to an image processing method and device thereof. BACKGROUND

[0002] Long-time exposure is a common photography method. Currently, electronic devices can simulate the effect of long-time exposure by synthesizing multiple continuous short-time exposure frames.

[0003] However, due to the influence of single-frame exposure time and frame-grabbing speed, the non-exposure accumulation time between every two short-time exposure frames in the multiple continuous short-time exposure frames may be too long. Therefore, after synthesizing the multiple continuous short-time exposure frames, the synthesized image may have defects such as breakpoints and ghosting, thereby resulting in poor simulation of the effect of long-time exposure. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide an image processing method and device, which can solve the problem of poor simulation of the effect of long-time exposure.

[0005] In a first aspect, the embodiments of the present application provide an image processing method, which comprises: acquiring N first images and N second images, the N first images are images of a first region in N frames of images collected, the N second images are images of a second region in the N frames of images, the first region is a region where a static object is located in an image, the second region is a region where a moving object is located in an image, and N is a positive integer; performing convolution operation processing on each of the N second images respectively to obtain N third images; and generating a target image according to the N first images and the N third images, the target image being an image with a simulated exposure time greater than or equal to an exposure time threshold.

[0006] In a second aspect, the embodiments of the present application provide an image processing device, which comprises an acquisition module, a processing module and a generation module; the acquisition module is configured to acquire N first images and N second images, the N first images being images of a first region in N frames of images collected, the N second images being images of a second region in the N frames of images, the first region being a region where a static object is located in an image, the second region being a region where a moving object is located in an image, and N being a positive integer; the processing module is configured to perform convolution operation processing on each of the N second images acquired by the acquisition module respectively to obtain N third images; and the generation module is configured to generate a target image according to the N first images acquired by the acquisition module and the N third images processed by the processing module, the target image being an image with a simulated exposure time greater than or equal to an exposure time threshold.

[0007] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable by the processor. The programs or instructions, when executed by the processor, implement the steps of the method according to the first aspect.

[0008] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions. The programs or instructions, when executed by a processor, implement the steps of the method according to the first aspect.

[0009] In a fifth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute programs or instructions to implement the method according to the first aspect.

[0010] In a sixth aspect, a computer program product is provided, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the method according to the first aspect.

[0011] In the embodiments of the present application, N first images and N second images can be acquired. The N first images are images of a first region in N frames of images, and the N second images are images of a second region in the N frames of images. The first region is a region where a stationary object is located in the images, and the second region is a region where a moving object is located in the images. N is a positive integer. Each of the N second images is processed by convolution operation to obtain N third images. A target image is generated according to the N first images and the N third images. The target image is an image with a simulated exposure time greater than or equal to an exposure time threshold. Through this scheme, since the electronic device can perform convolution operation on the images of the region where the moving object is located in the N frames of images, and generate a target image with a simulated exposure time greater than or equal to an exposure time threshold according to the N images obtained after processing and the images of the region where the stationary object is located in the N frames of images, and the convolution operation can obtain an image of the moving object blurred, the generated target image can have a blurred image of the region where the moving object is located, thereby avoiding defects such as breakpoints and ghosting, so that the effect of simulating long-time exposure can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a schematic diagram of a sequence of short-time exposure frames;

[0013] Figure 2 is a schematic diagram of the no-exposure accumulation time corresponding to a sequence of short-time exposure frames;

[0014] Figure 3 is a schematic diagram of the effect of directly synthesizing a sequence of short-time exposure frames;

[0015] Figure 4 is a flowchart of an image processing method provided by an embodiment of the present application;

[0016] Figure 5 is one of schematic diagrams of an image processing method provided by an embodiment of the present application;

[0017] Figure 6 is another of schematic diagrams of an image processing method provided by an embodiment of the present application;

[0018] Figure 7 is a third of schematic diagrams of an image processing method provided by an embodiment of the present application;

[0019] Figure 8 is a schematic diagram of an image processing apparatus provided by an embodiment of the present application;

[0020] Figure 9 is a schematic diagram of an electronic device provided by an embodiment of the present application;

[0021] Figure 10 is a hardware schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0023] The terms “first”, “second”, and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by “first”, “second”, and the like are generally of a kind and are not limited in number, for example, the first object can be one or more. In addition, “and / or” in the specification and claims indicates at least one of the connected objects, and the character “ / ” generally indicates that the front and rear associated objects are in an “or” relationship.

[0024] The image processing method and apparatus provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments and their application scenarios.

[0025] Long-time exposure is a common photography technique. In traditional photography, long-time exposure can be achieved by using a professional camera with manually adjustable exposure time, a stable tripod, and a neutral density filter (e.g., a light reduction lens) that can extend the exposure time. However, using too many devices for shooting can make the process more complicated, and ordinary users may find it difficult to use professional equipment for shooting.

[0026] In recent years, with the continuous enrichment of electronic device functions, technologies for simulating long-time exposure effects using multiple short-time exposure frames have appeared on digital cameras, camcorders, smartphones, and other electronic devices. Electronic devices can first capture multiple consecutive short-time exposure frames and then perform synthesis processing on the multiple short-time exposure frames through algorithms to simulate the effect of long-time exposure.

[0027] For example, assume that an electronic device captures 9 consecutive short-time exposure frames for a scene of a car driving, Figure 1 The sequence of the 9 consecutive short-time exposure frames is shown. As can be seen, the electronic device successively captures short-time exposure frame 11 to short-time exposure frame 19. Then the electronic device can perform synthesis processing on the 9 short-time exposure frames through algorithms to simulate the effect of long-time exposure of the image of the car driving.

[0028] However, due to the single-frame exposure time and the frame grabbing speed, the non-exposure accumulation time between every two short-time exposure frames in the above-mentioned 9 short-time exposure frames is too long, and there is a problem of discontinuous exposure accumulation. For example, as shown in Figure 2 assuming that the single-frame exposure time is 1 / 100 s and the frame grabbing speed is 10 frames per second, the non-exposure accumulation time will account for 90% of the total shooting time when simulating long-time exposure with multiple short-time exposure frames, and the exposure accumulation is discontinuous.

[0029] Therefore, in the image obtained after synthesis processing, there may be defects such as breakpoints and ghosting, for example, as shown in Figure 3 The image of the driving car in the image obtained by the electronic device after synthesis processing on the above-mentioned 9 short-time exposure frames has serious ghosting. Although theoretically, this problem can be improved by prolonging the single-frame exposure time and improving the frame grabbing speed, in practice, prolonging the single-frame exposure time will cause overexposure, which has a negative effect. For a given sensor, the frame grabbing speed cannot be infinitely improved, and too many frames will cause a burden on image processing, resulting in power consumption and performance problems when applied to actual shooting devices. Thus, simulating long-time exposure by the above-mentioned method will result in poor long-time exposure simulation effect.

[0030] To solve the above problems, in the image processing method provided in the embodiments of the present application, nine first images and nine second images can be acquired, the nine first images are images of the first region in the nine short-time exposure frames, the nine second images are images of the second region in the nine short-time exposure frames, the first region is a region in which an object other than a vehicle (for example, a stationary object in the embodiments of the present application) is located in an image, and the second region is a region in which a vehicle (for example, a moving object in the embodiments of the present application) is located in an image; and a convolution operation is respectively performed on each of the nine second images to obtain nine processed images (for example, N third images in the embodiments of the present application); and an image (for example, a target image in the embodiments of the present application) with an exposure time greater than or equal to an exposure time threshold is generated according to the nine first images and the nine processed images. Through this scheme, since the electronic device can perform convolution operation processing on the images of the region in which the vehicle is located in the nine short-time exposure frames, and generate a target image with an exposure time greater than or equal to an exposure time threshold according to the nine images obtained after processing and the images of the region in which the stationary object is located in the nine short-time exposure frames, and the convolution operation processing can obtain an image in which the vehicle is blurred, so that the image of the region in which the vehicle is located in the generated target image can have a blur effect, thereby avoiding the occurrence of defects such as breakpoints and ghosting, so that the effect of simulating long-time exposure can be improved, and the obtained target image is close to the result of real long-exposure shooting.

[0031] The embodiments of the present application provide an image processing method, Figure 4 A flowchart of the image processing method provided in the embodiments of the present application is shown. As shown in the figure, Figure 4 The image processing method provided in the embodiments of the present application can include the following steps 401 to 403. The method will be exemplarily described below taking that an electronic device executes the method as an example.

[0032] Step 401, the electronic device acquires N first images and N second images.

[0033] In the embodiments of the present application, the N first images are images of a first region in N frames of images collected, the N second images are images of a second region in the N frames of images, the first region is a region in which a stationary object is located in an image, the second region is a region in which a moving object is located in an image, and N is a positive integer.

[0034] In the embodiments of the present application, each of the N frames of images includes one first image and one second image.

[0035] Optionally, in the embodiments of the present application, the N frames of images can be images of the same scene collected, for example, the N frames of images are images of a continuous vehicle driving scene collected.

[0036] Optionally, in embodiments of the present application, the N frames of images can be N frames of continuously captured images.

[0037] Optionally, in embodiments of the present application, the moving object can be an object in a moving state, for example, the moving object can be a moving car, a rolling basketball, or flowing river water, etc.

[0038] Optionally, in embodiments of the present application, the step 401 can be implemented by the following steps 401a to 401d.

[0039] In step 401a, the electronic device acquires an image of a first region in each frame of image in the N frames of images, to obtain N first region images.

[0040] In step 401b, the electronic device acquires an image of a second region in each frame of image in the N frames of images, to obtain N second region images.

[0041] Optionally, in embodiments of the present application, the electronic device can use an image recognition method to acquire the image of the first region and the image of the second region in each frame of image frame by frame, to obtain the N first region images and the N second region images.

[0042] Optionally, in embodiments of the present application, the image recognition method can include a method of recognizing images by judging the difference frame by frame, or a method of recognizing images by semantics (such as vehicles, pedestrians, or water flow, etc.).

[0043] The following will be described by way of example with reference to the accompanying drawings, taking the method of acquiring the image of the first region and the image of the second region in a frame of image by the electronic device using the method of recognizing images by semantics.

[0044] For example, assuming that the frame of image is an image of a moving car, the electronic device can use the method of recognizing images by semantics to segment the car in the frame of image, so that, as shown in FIG. 5, the electronic device can acquire the image of the region 51 (i.e. the first region) where the car (i.e. the moving object) is located in the image, and the image of the region 52 (i.e. the second region) where the trees, roads, etc. (i.e. the stationary objects) are located in the image; in this way, the electronic device can obtain the first region image and the second region image of the frame of image. Figure 5

[0045] In step 401c, the electronic device performs image alignment processing on the N first region images, to obtain N first images.

[0046] In embodiments of the present application, the image alignment processing is used to compensate for the change in viewing angle caused by the shaking or movement of the electronic device during the process of capturing the N frames of images.

[0047] ​Optionally, in an embodiment of the present application, the specific methods of the above-mentioned image alignment processing may include: displacement, rotation, distortion, subspace matching, etc.

[0048] For the description of the specific method of the above-mentioned image alignment processing, reference can be made to the relevant description in the related art. In order to avoid repetition, it will not be repeated here.

[0049] Optionally, in the embodiment of the present application, if the N first region images are b1 to b n , then the i-th first image b among the above N first images i ' can be expressed by the following formula (1):

[0050] b i '=C(b i , x i ),i=1,2,…,n; (1)

[0051] Among them, x i is the spatial offset of the i-th frame image after image alignment.

[0052] Step 401d: The electronic device uses the target space offset to perform image alignment processing on the N second region images to obtain N second images.

[0053] In the embodiment of the present application, the target spatial offset is: the spatial offset of each frame of the N first images.

[0054] It can be understood that when the electronic device performs image alignment processing on the N first region images, each of the N frames of images corresponds to a spatial offset.

[0055] Optionally, in an embodiment of the present application, for one of the above-mentioned N second area images, the electronic device can use the spatial offset of a corresponding frame image to perform image alignment processing to obtain a corresponding second image; thus, after the electronic device performs image alignment processing on each second area image, it can obtain the above-mentioned N second images.

[0056] Optionally, in the embodiment of the present application, if the N second area images are a1 to a n , then the i-th second image a among the N second images above i ' can be expressed by the following formula (2):

[0057] a i '=C(a i , x i ),i=1,2,…,n; (2).

[0058] In the embodiments of the present application, since the electronic device can perform image alignment processing on the N first region images to obtain the N first images, and perform image alignment processing on the N second region images by using the spatial offset of each frame of image after the image alignment processing, it can be ensured that the obtained first images and second images are all images after image alignment, so that the viewing angles of the obtained images are the same.

[0059] In step 402, the electronic device performs convolution operation processing on each of the N second images to obtain N third images.

[0060] In the embodiments of the present application, the convolution operation processing can be used to improve the blurring effect of the processed image to approach the effect of real long-exposure shooting.

[0061] It can be understood that each of the N third images is an image obtained by performing convolution operation processing on one of the N second images.

[0062] Optionally, in the embodiments of the present application, the step 402 can be implemented by the following steps 402a and 402b.

[0063] In step 402a, the electronic device obtains a convolution kernel corresponding to each of the N second images.

[0064] Optionally, in the embodiments of the present application, the N frames of images are the first N frames of images in the N+1 frames of images collected, and the N second images are the first N second images in the N+1 second images; then the step 402a can be implemented by the following steps 402a1 and 402a2.

[0065] In step 402a1, the electronic device calculates the motion path of the same feature point in the i-th second image and the i+1-th second image for the i-th second image in the N second images.

[0066] In the embodiments of the present application, i is a positive integer less than or equal to N.

[0067] Optionally, in the embodiments of the present application, the same feature point can be any possible same point such as a same vertex, a same corner or a same center point in the i-th second image and the i+1-th second image.

[0068] Optionally, in the embodiments of the present application, the method for calculating the motion path can be a feature point matching and displacement detection method.

[0069] Optionally, in the embodiments of the present application, the motion path can be specifically represented as a corresponding vector or matrix.

[0070] The specific description of the electronic device calculating the motion path can refer to the related description in the related art, and will not be described here to avoid repetition.

[0071] Step 402a2, the electronic device determines the motion path as the convolution kernel corresponding to the i-th second image.

[0072] Optionally, in the embodiment of the present application, the above-mentioned convolution kernel can be referred to as a motion blur convolution kernel, which is used to realize the effect of motion blur.

[0073] It can be understood that the electronic device can perform the above-mentioned steps 402a1 and 402a2 on each of the N second images, so as to obtain the convolution kernel corresponding to each second image.

[0074] That is, for the N second images:

[0075] The electronic device calculates the motion path of the same feature point in the first second image and the second second image, and determines the motion path as the convolution kernel corresponding to the first second image;

[0076] The electronic device calculates the motion path of the same feature point in the second second image and the third second image, and determines the motion path as the convolution kernel corresponding to the second second image;

[0077] …

[0078] The electronic device calculates the motion path of the same feature point in the N-th second image and the N+1-th second image, and determines the motion path as the convolution kernel corresponding to the N-th second image;

[0079] In this way, the electronic device can obtain the convolution kernel corresponding to each of the N second images.

[0080] In the embodiment of the present application, since the electronic device can calculate the motion path of the same feature point in the i-th second image and the i+1-th second image, and determine the motion path as the convolution kernel corresponding to the i-th second image, the obtained convolution kernel can reflect the displacement of the moving object in the adjacent two short-time exposure frames, so that the convolution operation processing using the obtained convolution kernel can ensure the accuracy of the processed image.

[0081] Step 402b, the electronic device performs convolution operation on each second image in the N second images, and performs convolution operation on each second image and its corresponding convolution kernel.

[0082] Optionally, in the embodiment of the present application, if the N second images are a1'~a n, and the N second images correspond to the convolution kernel psf1~psf n , and the N second images correspond to the convolution kernel psf1~psf i , and the N second images correspond to the convolution kernel psf1~psf

[0083] a i , and the N second images correspond to the convolution kernel psf1~psf i ’*psf i ; (3).

[0084] a i ’ can be the pixel value matrix corresponding to the i-th second image.

[0085] Figure 6 The effect of the image after the convolution operation processing on one second image is shown, as shown in FIG. 4, the processed image includes a blurred image of the moving object, so that the effect of motion blur can be achieved. Figure 6

[0086] In the embodiment of the application, since the electronic device can perform convolution operation on each second image and the corresponding convolution kernel to perform convolution operation processing on each second image respectively, the N third images obtained by processing can include blurred images, so as to facilitate simulation of the effect of long-time exposure.

[0087] Optionally, the step 402 can be implemented by the following step 402c in the embodiment of the application.

[0088] Step 402c, the electronic device performs convolution operation processing on each of the N second images respectively in a case where the confidence of the convolution kernel corresponding to each second image is greater than or equal to the confidence threshold.

[0089] In the embodiment of the application, the confidence is used to evaluate whether the corresponding convolution kernel is accurate.

[0090] Optionally, the confidence threshold can be a system default or can be set by a user according to use requirements in the embodiment of the application.

[0091] Optionally, the size relationship between the confidence of one convolution kernel and the confidence threshold can be determined according to at least one of the following: the signal-to-noise ratio of the convolution kernel, the spatial continuity of the convolution kernel, and the time continuity of the convolution kernel.

[0092] For example, if the signal-to-noise ratio of one convolution kernel is greater than or equal to a first threshold (i.e., the confidence threshold), it can be considered that the convolution kernel is accurate.

[0093] ​For example, if the spatial continuity of a convolution kernel is greater than or equal to a second threshold (i.e., the confidence threshold), the convolution kernel can be considered accurate.

[0094] Optionally, in the embodiments of the present application, when the convolution kernel corresponding to each of the second images is greater than or equal to the confidence threshold, the electronic device performs convolution operation processing on each of the second images; and when at least one of the N convolution kernels corresponding to the N second images is less than the confidence threshold, the electronic device does not process each of the second images.

[0095] In the embodiments of the present application, since the electronic device can perform convolution operation processing on each of the second images when the confidence of the target convolution kernel is greater than or equal to the confidence threshold, the flexibility of the electronic device in performing convolution operation processing can be improved, and the accuracy of the processed image can be ensured.

[0096] In step 403, the electronic device generates a target image according to the N first images and the N third images.

[0097] In the embodiments of the present application, the target image is an image with a simulated exposure time greater than or equal to an exposure time threshold.

[0098] It can be understood that the target image is an image simulating a real long-exposure shooting effect.

[0099] Optionally, in the embodiments of the present application, the step 403 can be implemented by the following steps 403a to 403c.

[0100] In step 403a, the electronic device synthesizes the N first images to obtain a first target image.

[0101] Optionally, in the embodiments of the present application, the electronic device can synthesize the N first images by using the following algorithms: an average value synthesis algorithm, a maximum value synthesis algorithm, a minimum value synthesis algorithm, or an intermediate value synthesis algorithm.

[0102] In step 403b, the electronic device synthesizes the N third images to obtain a second target image.

[0103] Optionally, in the embodiments of the present application, the electronic device can synthesize the N third images by using the following algorithms: a multi-frame noise reduction synthesis algorithm, a super-resolution synthesis algorithm, or a High Dynamic Range Imaging (HDR) extended dynamic range synthesis algorithm.

[0104] In step 403c, the electronic device fuses the first target image and the second target image to obtain a target image.

[0105] The above algorithms and the specific description of fusing the first target image and the second target image can refer to the related description in the related art, and will not be described here again to avoid repetition.

[0106] In the embodiment of the present application, the electronic device can fuse the first target image obtained by synthesizing the N first images and the second target image obtained by synthesizing the N second images to obtain a target image, that is, the electronic device can perform multi-frame synthesis based on the image after motion blur operation (i.e., convolution operation), so that the authenticity of the long-exposure simulation image can be improved.

[0107] Figure 7 An effect diagram of an image processed by the image processing method provided in the embodiment of the present application is shown, as shown in Figure 7 The processed image significantly improves the effect of the output image, avoids the defects of breakpoints and ghosting in the conventional multi-frame short-exposure simulation long-exposure effect technology, and thus an image close to the long-exposure real shooting result can be obtained.

[0108] In the image processing method provided in the embodiment of the present application, the electronic device can perform convolution operation processing on the image of the region where the moving object is located in the N frames of images, and generate a target image with an exposure time greater than or equal to an exposure time threshold according to the N images obtained after processing and the image of the region where the stationary object is located in the N frames of images. The convolution operation processing can obtain the image of the moving object, so that the image of the region where the moving object is located in the generated target image can have a blur effect, thereby avoiding the defects of breakpoints and ghosting, and thus the effect of simulating long-exposure can be improved.

[0109] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, the image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0110] In combination with Figure 8The embodiment of the present application provides an image processing device 80, which can comprise an acquisition module 81, a processing module 82 and a generation module 83. The acquisition module 81 can be used for acquiring N first images and N second images, the N first images are images of a first region in N frames of collected images, the N second images are images of a second region in the N frames of images, the first region is a region where a static object is located in the images, the second region is a region where a moving object is located in the images, and N is a positive integer. The processing module 82 can be used for respectively performing convolution operation processing on each second image in the N second images acquired by the acquisition module 81, to obtain N third images. The generation module 83 can be used for generating a target image according to the N first images acquired by the acquisition module 81 and the N third images processed by the processing module 82, and the target image is an image with an analog exposure time greater than or equal to an exposure time threshold.

[0111] In a possible implementation, the processing module 82 can be specifically configured to acquire a convolution kernel corresponding to each second image, and perform convolution operation on a second image and the convolution kernel corresponding to the second image, to perform convolution operation processing on each second image.

[0112] In a possible implementation, the N frames of images are the first N frames of images in N+1 frames of collected images, and the N second images are the first N second images of N+1 second images. The processing module 82 can be specifically configured to calculate a motion path of a same feature point in an i-th second image and an i+1-th second image in the N second images, i is a positive integer less than or equal to N, and determine the motion path as a convolution kernel corresponding to the i-th second image.

[0113] In a possible implementation, the acquisition module 81 can be specifically configured to acquire an image of the first region in each frame of image in the N frames of images, to obtain N first region images, acquire an image of the second region in the each frame of image, to obtain N second region images, perform image alignment processing on the N first region images, to obtain the N first images, and perform image alignment processing on the N second region images by using a target space offset, to obtain the N second images, and the target space offset is a space offset of the each frame of image for obtaining the N first images.

[0114] In a possible implementation, the generation module 83 can be specifically configured to synthesize the N first images, to obtain a first target image, synthesize the N third images, to obtain a second target image, and fuse the first target image and the second target image, to obtain the target image.

[0115] In the image processing apparatus provided in the embodiments of the present application, the image processing apparatus can perform convolution operation processing on the image of the region where the moving object is located in the N frames of images, and generate a target image with an analog exposure time longer than or equal to an exposure time threshold according to the N images obtained after processing and the image of the region where the stationary object is located in the N frames of images. The convolution operation processing can obtain the image of the moving object blurred, so that the image of the region where the moving object is located in the generated target image can be blurred, thereby avoiding the occurrence of defects such as breakpoints and ghosting, and thus the effect of simulating long-time exposure can be improved.

[0116] The image processing apparatus in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than the terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like, and the embodiments of the present application are not limited thereto.

[0117] The image processing apparatus in the embodiments of the present application can be an apparatus with an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems, and the embodiments of the present application are not limited thereto.

[0118] The image processing apparatus provided in the embodiments of the present application can implement Figures 4 to 7 The method embodiments implement various processes, and to avoid repetition, details are not described herein.

[0119] For example, Figure 9As shown, the embodiments of the present application further provide an electronic device 900, comprising a processor 901 and a memory 902, wherein the memory 902 stores programs or instructions executable on the processor 901, and the programs or instructions are executed by the processor 901 to implement each step of the above image processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0120] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0121] Figure 10 A hardware structure schematic diagram of an electronic device according to an embodiment of the present application.

[0122] The electronic device 1000 includes, but is not limited to, a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc.

[0123] Those skilled in the art can understand that the electronic device 1000 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 1010 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described herein.

[0124] The processor 1010 can be configured to acquire N first images and N second images, the N first images are images of a first region in N frames of acquired images, the N second images are images of a second region in the N frames of images, the first region is a region where a stationary object is located in the image, the second region is a region where a moving object is located in the image, and N is a positive integer; and for each of the acquired N second images, convolution operation processing is performed respectively to obtain N third images; and a target image is generated according to the acquired N first images and the processed N third images, the target image is an image with an analog exposure time greater than or equal to an exposure time threshold.

[0125] In a possible implementation, the processor 1010 can be specifically configured to acquire a convolution kernel corresponding to each second image; and for each second image, a second image and its corresponding convolution kernel are subjected to convolution operation, so as to perform convolution operation processing on each second image respectively.

[0126] In a possible implementation, the N frames of images are the first N frames of N+1 frames of images collected, and the N second images are the first N second images of N+1 second images. The processor 1010 can be specifically configured to: for an i-th second image in the N second images, calculate a motion path of a same feature point in the i-th second image and an i+1-th second image, i being a positive integer less than or equal to N; and determine the motion path as a convolution kernel corresponding to the i-th second image.

[0127] In a possible implementation, the processor 1010 can be specifically configured to: obtain an image of the first region in each frame of image in the N frames of images, to obtain N first region images; obtain an image of the second region in the each frame of image, to obtain N second region images; perform image alignment processing on the N first region images, to obtain the N first images; and perform image alignment processing on the N second region images by using a target spatial offset, to obtain the N second images, the target spatial offset being a spatial offset of the each frame of image in which the N first images are obtained.

[0128] In a possible implementation, the processor 1010 can be specifically configured to: synthesize the N first images to obtain a first target image; synthesize the N third images to obtain a second target image; and fuse the first target image and the second target image to obtain a target image.

[0129] In the electronic device provided in the embodiments of the present application, because the electronic device can perform convolution operation processing on images of regions in which a moving object is located in N frames of images, and generate a target image with an analog exposure time longer than or equal to an exposure time threshold according to N images obtained after processing and images of regions in which a stationary object is located in the N frames of images, and because the moving object can be blurred by convolution operation processing, the image of the region in which the moving object is located in the generated target image can be blurred, so as to avoid defects such as breakpoints and ghosting, and thus the effect of simulating long-time exposure can be improved.

[0130] The beneficial effects of the various implementations in the embodiments of the present application can refer to the beneficial effects of the corresponding implementations in the method embodiments, and details are not described herein again to avoid repetition.

[0131] It should be understood that in the embodiments of the present application, the input unit 1004 can include a graphics processor (GPU) 10041 and a microphone 10042. The graphics processor 10041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1006 can include a display panel 10061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also referred to as a touch screen. The touch panel 10071 can include two parts of a touch detection device and a touch controller. The other input devices 10072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, and the like, which will not be described here.

[0132] The memory 1009 can be used to store software programs and various data. The memory 1009 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), etc. In addition, the memory 1009 can include a volatile memory or a non-volatile memory, or the memory 1009 can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 1009 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0133] The processor 1010 can include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1010.

[0134] The embodiments of the present application also provide a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned image processing method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0135] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0136] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the above image processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0137] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0138] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and is executed by at least one processor to realize the processes of the above image processing method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0139] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in the opposite order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0141] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining N first images and N second images, each of the N first images being an image of a first region in N frames of images collected, each of the N second images being an image of a second region in the N frames of images, the first region being a region in which a static object is located in an image, the second region being a region in which a moving object is located in an image, N being a positive integer; respectively performing convolution operation processing on each of the N second images to obtain N third images; generating a target image according to the N first images and the N third images, the target image being an image with an analog exposure duration greater than or equal to an exposure duration threshold; wherein the respective convolution operation processing on each of the N second images comprises: obtaining a convolution kernel corresponding to each second image; in a case where a confidence of the convolution kernel corresponding to each second image is greater than or equal to a confidence threshold, performing convolution operation on one second image and the convolution kernel corresponding thereto for each second image to perform the respective convolution operation processing on each of the N second images, the confidence being used to evaluate whether the corresponding convolution kernel is accurate; the N frames of images are the first N frames of images in N+1 frames of images collected, and the N second images are the first N second images in N+1 second images; the obtaining of the convolution kernel corresponding to each second image comprises: for an i-th second image in the N second images, calculating a motion path of a same feature point in the i-th second image and an i+1-th second image, i being a positive integer less than or equal to N; determining the motion path as the convolution kernel corresponding to the i-th second image.

2. The method of claim 1, wherein, the obtaining of the N first images and the N second images comprises: obtaining an image of the first region in each frame of image in the N frames of images to obtain N first region images; obtaining an image of the second region in each frame of image to obtain N second region images; performing image alignment processing on the N first region images to obtain the N first images; performing image alignment processing on the N second region images by using a target spatial offset to obtain the N second images, the target spatial offset being a spatial offset of each frame of image for obtaining the N first images.

3. The method of claim 1, wherein, the generating of the target image according to the N first images and the N third images comprises: synthesizing the N first images to obtain a first target image; synthesizing the N third images to obtain a second target image; fusing the first target image and the second target image to obtain the target image.

4. An image processing apparatus characterized by comprising: The device comprises an obtaining module, a processing module and a generating module; the obtaining module is configured to obtain N first images and N second images, each of the N first images being an image of a first region in N frames of images collected, each of the N second images being an image of a second region in the N frames of images, the first region being a region in which a static object is located in an image, the second region being a region in which a moving object is located in an image, N being a positive integer; The processing module is configured to perform convolution operation processing on each of the N second images obtained by the acquisition module respectively to obtain N third images. The generation module is configured to generate a target image according to the N first images obtained by the acquisition module and the N third images processed by the processing module, the target image being an image with an analog exposure time longer than or equal to an exposure time threshold. The processing module is specifically configured to obtain a convolution kernel corresponding to each second image, and in a case where a confidence level of the convolution kernel corresponding to each second image is greater than or equal to a confidence level threshold, perform convolution operation on one second image and the convolution kernel corresponding to the second image to perform convolution operation processing on each second image respectively, the confidence level being used to evaluate whether the corresponding convolution kernel is accurate; the N frames of images are the first N frames of images in N+1 frames of images collected, the N second images are the first N second images in N+1 second images, and for an i-th second image in the N second images, a motion path of a same feature point in the i-th second image and an i+1-th second image is calculated, i being a positive integer less than or equal to N; and the motion path is determined as the convolution kernel corresponding to the i-th second image.

5. The apparatus of claim 4, wherein The acquisition module is specifically configured to obtain images of the first region in each of the N frames of images to obtain N first region images, and obtain images of the second region in each of the N frames of images to obtain N second region images, and perform image alignment processing on the N first region images to obtain the N first images; and perform image alignment processing on the N second region images by using a target space offset to obtain the N second images, the target space offset being a space offset of each of the N frames of images in which the N first images are obtained.

6. The apparatus of claim 4, wherein The generation module is specifically configured to synthesize the N first images to obtain a first target image, and synthesize the N third images to obtain a second target image; and fuse the first target image and the second target image to obtain the target image. ​ ​

Images