Image Processing Method, Image Processing Apparatus, Terminal, and Readable Storage Medium

Abstract

The present application discloses an image processing method, an image processing apparatus, a terminal, and a readable storage medium. The image processing method includes: obtaining multiple consecutive preview images and a first exposure time of a current frame preview image; performing a first motion detection on the current frame preview image to obtain a first detection result; when a preset target object exists in the current frame preview image and the first detection result is a first result, obtaining a first image to be processed using the first exposure time; when a target object exists in the current frame preview image and the first detection result is a second result, obtaining a first image to be processed using a second exposure time; wherein the first result is used to represent no motion, the second result is used to represent motion, and the second exposure time is less than the first exposure time; performing frequency domain fusion on at least one preview image and the first image to be processed to obtain a target image. The present application uses a frequency domain fusion algorithm to improve the image quality in the presence of a target object.

Description

Technical Field

[0001] This application relates to the field of imaging technologies, and particularly to an image processing method, an image processing apparatus, a terminal, and a non-volatile computer-readable storage medium. Background Art

[0002] As more and more people like to keep pets (for example, cats or dogs), users' demand for being able to take clear images of pets is getting higher and higher. When using traditional image denoising or image enhancement algorithms with multiple frames of images, there is usually an assumption about the scene that the input frames taken are continuous, and the user or the photographed object will not have large-scale movement or displacement, so that the signal-to-noise ratio or dynamic range of the image can be improved as much as possible through more inputs. However, when there are moving objects in the photographed scene, especially in a high-frequency motion scene such as photographing a pet, there will be some areas that cannot be fully registered in the moving area. Since there is no information of other frames that can be fused in these moving areas, in order to ensure the consistency of noise, spatial domain denoising is used, which will result in some loss of signal-to-noise ratio or abrupt denoising smear, resulting in poor visual perception effects. Summary of the Invention

[0003] Embodiments of this application provide an image processing method, an image processing apparatus, a terminal, and a non-volatile computer-readable storage medium.

[0004] The image processing method provided by the embodiments of this application includes: obtaining multiple consecutive preview images and a first exposure time of the current frame of the preview images; performing a first motion detection on the current frame of the preview images to obtain a first detection result; when there is a preset target object in the current frame of the preview images and the first detection result is a first result, obtaining a first image to be processed using the first exposure time; when there is the target object in the current frame of the preview images and the first detection result is a second result, obtaining a first image to be processed using a second exposure time; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; and performing frequency domain fusion on at least one frame of the preview images and the first image to be processed to obtain a target image.

[0005] The image processing apparatus provided by the embodiment of the present application includes: a first acquisition module, a first detection module, a second acquisition module, and a first processing module. The first acquisition module is configured to acquire multiple consecutive preview images and a first exposure time of the current frame of the preview images. The first detection module is configured to perform a first motion detection on the current frame of the preview images to obtain a first detection result; the second acquisition module is configured to acquire a first image to be processed by using the first exposure time when a preset target object exists in the current frame of the preview images and the first detection result is a first result; and acquire a first image to be processed by using a second exposure time when the preset target object exists in the current frame of the preview images and the first detection result is a second result; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time. The first processing module is configured to perform frequency domain fusion on at least one frame of the preview images and the first image to be processed to obtain a target image.

[0006] The terminal according to the embodiment of the present application includes one or more processors, a memory, and one or more programs. Wherein, one or more of the programs are stored in the memory and are executed by one or more of the processors, and the programs include instructions for executing the image processing method according to the embodiment of the present application. The image processing method includes: acquiring multiple consecutive preview images and a first exposure time of the current frame of the preview images; performing a first motion detection on the current frame of the preview images to obtain a first detection result; acquiring a first image to be processed by using the first exposure time when a preset target object exists in the current frame of the preview images and the first detection result is a first result; acquiring a first image to be processed by using a second exposure time when the preset target object exists in the current frame of the preview images and the first detection result is a second result; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; and performing frequency domain fusion on at least one frame of the preview images and the first image to be processed to obtain a target image.

[0007] The non - volatile computer - readable storage medium of the embodiment of the present application contains a computer program. When the computer program is executed by one or more processors, the processors execute the following image - processing method: obtaining multiple consecutive preview images and the first exposure time of the current - frame preview image; performing a first motion detection on the current - frame preview image to obtain a first detection result; when there is a preset target object in the current - frame preview image and the first detection result is the first result, obtaining a first image to be processed using the first exposure time; when there is the target object in the current - frame preview image and the first detection result is the second result, obtaining a first image to be processed using a second exposure time; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; and performing frequency - domain fusion on at least one frame of the preview image and the first image to be processed to obtain a target image.

[0008] The image - processing method, image - processing device, terminal, and non - volatile computer - readable storage medium of the present application, by obtaining a first image to be processed using a shorter exposure time when detecting motion in the scene, can improve the shooting success rate and image clarity. At the same time, when detecting a target object (such as a pet), using the frequency - domain fusion algorithm can overcome some signal - to - noise ratio losses or abrupt noise - reduction smearing, improve the overall image quality of the finally obtained target image, and is beneficial to improving the visual perception effect.

[0009] Additional aspects and advantages of the embodiments of the present application will be given in part in the following description, become apparent in part from the following description, or be understood through the practice of the present application. Brief Description of the Drawings

[0010] The above - mentioned and / or additional aspects and advantages of the present application will become apparent and be easily understood from the description of the embodiments in conjunction with the following drawings, where:

[0011] Figure 1 is a flowchart of the image - processing method in some embodiments of the present application;

[0012] Figure 2 is a structural diagram of the image - processing device in some embodiments of the present application;

[0013] Figure 3 is a structural diagram of the terminal in some embodiments of the present application;

[0014] Figure 4 and Figure 5 is a flowchart of the image - processing method in some embodiments of the present application;

[0015] Figure 6 It is a corresponding relationship diagram of the first exposure adjustment strategy and the second exposure adjustment strategy and the first exposure time in some embodiments of the present application;

[0016] Figures 7 to 9 It is a schematic flowchart of an image processing method in some embodiments of the present application;

[0017] Figure 10 It is a schematic diagram of a region of interest box in the previous frame preview image and a region of interest box in the current frame preview image in an image processing method in some embodiments of the present application;

[0018] Figures 11 to 16 It is a schematic flowchart of an image processing method in some embodiments of the present application;

[0019] Figure 17 It is a schematic connection diagram of a non - volatile computer - readable storage medium and a processor in some embodiments of the present application. Specific Embodiments

[0020] The following details the embodiments of the present application. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and should not be construed as a limitation to the embodiments of the present application.

[0021] Please refer to Figure 1 , an embodiment of the present application provides an image processing method. The image processing method includes:

[0022] 01: Obtain multiple consecutive preview images and the first exposure time of the current frame preview image;

[0023] 02: Perform a first motion detection on the current frame preview image to obtain a first detection result;

[0024] 03: When there is a preset target object in the current frame preview image and the first detection result is the first result, obtain a first image to be processed using the first exposure time; when there is a target object in the current frame preview image and the first detection result is the second result, obtain a first image to be processed using the second exposure time; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time;

[0025] 04: Perform frequency - domain fusion on at least one frame of preview image and the first image to be processed to obtain a target image.

[0026] Please combineFigure 2 In addition, an embodiment of the present application also provides an image processing apparatus 100. The image processing apparatus 100 includes a first acquisition module 10, a first detection module 20, a second acquisition module 30, and a first processing module 40. The methods in 01, 02, 03, and 04 above can be executed and implemented by the first acquisition module 10, the first detection module 20, the second acquisition module 30, and the first processing module 40 respectively. That is to say, the first acquisition module 10 is configured to acquire multiple consecutive preview images and the first exposure time of the current frame preview image; the first detection module 20 is configured to perform a first motion detection on the current frame preview image to obtain a first detection result; the second acquisition module 30 is configured to acquire a first image to be processed using the first exposure time when a preset target object exists in the current frame preview image and the first detection result is the first result; and acquire a first image to be processed using the second exposure time when a target object exists in the current frame preview image and the first detection result is the second result; wherein the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; the first processing module 40 is configured to perform frequency domain fusion on at least one frame of preview image and the first image to be processed to obtain a target image.

[0027] Please refer to Figure 3 In addition, an embodiment of the present application also provides a terminal 1000. The terminal 1000 includes one or more processors 200, a memory 300, and one or more programs. Among them, the one or more programs are stored in the memory 300 and are executed by the one or more processors 200. The programs include instructions for executing the image processing method of the embodiment of the present application. That is, when the one or more processors 200 execute the programs, the processors 200 can implement the methods in 01, 02, 03, and 04. That is, the one or more processors 200 are configured to: acquire multiple consecutive preview images and the first exposure time of the current frame preview image; perform a first motion detection on the current frame preview image to obtain a first detection result; acquire a first image to be processed using the first exposure time when a preset target object exists in the current frame preview image and the first detection result is the first result; acquire a first image to be processed using the second exposure time when a target object exists in the current frame preview image and the first detection result is the second result; wherein the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; and perform frequency domain fusion on at least one frame of preview image and the first image to be processed to obtain a target image.

[0028] In the image processing method, image processing apparatus 100, and terminal 1000 according to the embodiments of the present application, when it is detected that there is movement in the scene, a first image to be processed is acquired with a shorter exposure time, so that the photo-taking success rate and the clarity of the image can be improved. At the same time, when a target object (such as a pet) is detected, a frequency domain fusion algorithm is used, so that some loss of signal-to-noise ratio or abrupt noise reduction smear can be overcome, and the overall image quality of the finally acquired target image can be improved, which is beneficial to improving the visual perception effect.

[0029] It should be noted that in some embodiments, before the user presses the shooting button, the processor 200 (or the first acquisition module 10) can control the imaging module (not shown in the figure) to acquire multiple frames of preview images with the same exposure time. The terminal 1000 (or the image processing apparatus 100) can predict the current scene information through the multiple frames of preview images, and adjust the exposure parameters and image processing method according to the predicted current scene information, so that a better-quality image can be obtained after the user presses the shooting button.

[0030] Specifically, the processor 200 (or the first acquisition module 10) acquires multiple frames of consecutive preview images and the first exposure time of the current frame preview image. Among them, the acquisition time of the current frame preview image is later than the acquisition times of other preview images in the multiple frames of preview images. After the current frame preview image is acquired, the processor 200 (or the first detection module 20) performs a first motion detection on the current frame preview image to obtain a first detection result. Exemplarily, please refer to Figure 4 , in some embodiments, step 02: performing a first motion detection on the current frame preview image to obtain a first detection result, includes:

[0031] 021: obtaining a detection threshold corresponding to the gyroscope parameter according to the gyroscope parameter corresponding to the current frame preview image;

[0032] 022: obtaining a local motion detection amount of the current frame preview image according to the multiple frames of preview images; and

[0033] 023: obtaining a first detection result according to the local motion detection amount and the detection threshold.

[0034] Please combine Figure 2 and Figure 3 , in some embodiments, both the first detection module 20 and the processor 200 can also be used to execute the methods in 021, 022, and 023. That is to say, both the first detection module 20 and the processor 200 can also be used to obtain a detection threshold corresponding to the gyroscope parameter according to the gyroscope parameter corresponding to the current frame preview image; obtain a local motion detection amount of the current frame preview image according to the multiple frames of preview images; and obtain a first detection result according to the local motion detection amount and the detection threshold.

[0035] After obtaining the current frame preview image, the processor 200 (or the first detection module 20) obtains a detection threshold corresponding to the gyroscope parameter according to the gyroscope parameter corresponding to the current frame preview image. It should be noted that, in some embodiments, a three-axis gyroscope (not shown in the figure) is provided in the terminal 1000 (or the image processing device 100), so that the parameter information of the terminal 1000 (or the image processing device 100) in three axial directions can be measured. Of course, other types of gyroscopes can also be used, which are not limited herein. The following embodiments will be described by taking the gyroscope in the terminal 1000 (or the image processing device 100) as a three-axis gyroscope as an example.

[0036] Specifically, in some embodiments, the correspondence between the gyroscope parameter and the detection threshold is stored in the terminal 1000 (or the image processing device 100). After obtaining the maximum gyroscope parameter in the three axial directions corresponding to the current frame preview image, according to the obtained maximum gyroscope parameter and the correspondence between the gyroscope parameter and the detection threshold, the detection threshold corresponding to the obtained maximum gyroscope parameter is obtained. For example, in some embodiments, the correspondence between the gyroscope parameter range and the detection threshold is stored in the processor 200 (or the first detection module 20): when max Gyro < 0.15, th-current = 15; when 0.15 ≤ max Gyro < 0.25, th-current = 25; when 0.25 ≤ max Gyro < 0.35, th-current = 35; when max Gyro ≥ 0.35, th-current = 50. Wherein, max Gyro is the maximum gyroscope parameter, and th-current is the detection threshold. When the maximum gyroscope parameter maxGyro corresponding to the current frame preview image in the three axial directions is 0.2, since 0.15 ≤ the maximum gyroscope parameter max Gyro < 0.25, the corresponding detection threshold th-current can be obtained as 25.

[0037] After obtaining multiple consecutive frame preview images, the processor 200 (or the first detection module 20) obtains the local motion detection amount of the current frame preview image according to the multiple frame preview images. Exemplarily, in some embodiments, the local motion detection amount can be calculated by performing full-image registration and frame difference on the current frame preview image and at least one other preview image. Of course, in some embodiments, the local motion detection amount can also be calculated by other means, which are not limited herein.

[0038] After obtaining the detection threshold and the local motion detection amount, the processor 200 (or the first detection module 20) obtains the first detection result according to the local motion detection amount and the detection threshold. Since the first detection result is jointly obtained based on the local motion detection amount and the detection threshold, it is possible to avoid local motion caused by device jitter, thereby enabling a more accurate determination of whether there is motion in the current scene.

[0039] More specifically, please refer to Figure 4 and Figure 5 , in some embodiments, step 023: obtaining the first detection result according to the local motion detection amount and the detection threshold may include:

[0040] 0231: When the local motion detection amount is less than the detection threshold, or the first exposure time is not within the preset range, determining that the first detection result is the first result; and

[0041] 0232: When the local motion detection amount is greater than the detection threshold and the first exposure time is within the preset range, determining that the first detection result is the second result.

[0042] Please refer to Figure 2 and Figure 3 , in some embodiments, both the first detection module 20 and the processor 200 can also be used to execute the methods in 0231 and 0232. That is to say, both the first detection module 20 and the processor 200 can also be used to determine that the first detection result is the first result when the local motion detection amount is less than the detection threshold, or the first exposure time is not within the preset range; and to determine that the first detection result is the second result when the local motion detection amount is greater than the detection threshold and the first exposure time is within the preset range.

[0043] After obtaining the detection threshold and the local motion detection amount, the processor 200 (or the first detection module 20) determines the first detection result by comparing the magnitude between the local motion detection amount and the detection threshold, and judging whether the first exposure time is within the preset range. When the local motion detection amount is less than the detection threshold, or the first exposure time is not within the preset range, the first detection result is determined to be the first result; when the local motion detection amount is greater than the detection threshold and the first exposure time is within the preset range, the first detection result is determined to be the second result. It should be noted that the first result is used to represent that there is no motion in the current scene, that is, when the first detection result is the first result, it means that there is no motion in the current scene; the second result is used to represent that there is motion in the current scene, that is, when the first detection result is the second result, it means that there is motion in the current scene.

[0044] Specifically, in some embodiments, the processor 200 (or the first detection module 20) compares the magnitude between the local motion detection amount and the detection threshold. When the local motion detection amount is not greater than the detection threshold, that is, when the local motion detection amount is less than or equal to the detection threshold, it indicates that although there is a local motion amount at this time, it is caused by device jitter, and there is no motion in the current scene. Therefore, at this time, the first detection result is determined to be the first result. When the local motion detection amount is greater than the detection threshold, the processor 200 (or the first detection module 20) determines whether the first exposure time is within the preset range. If the first exposure time is within the preset range, it indicates that there is motion in the current scene. Therefore, at this time, the first detection result is determined to be the second result. Particularly, when the first exposure time is not within the preset range, the first detection result is determined to be the first result. Since the judgment result of the first motion detection only takes effect when the first exposure time is within the preset range, when the first exposure time is not within the preset range, the first detection result is confirmed to be the first result. Of course, in some embodiments, it is also possible to first determine whether the first exposure time is within the preset range, and then compare the local motion detection amount with the detection threshold after the first exposure result is within the preset range, which is not limited herein. In addition, in some embodiments, the preset range is 10 ms to 100 ms, that is, when 10 ms < the first exposure time < 100 ms, the first motion detection judgment result can take effect.

[0045] Particularly, in some embodiments, the processor 200 (or the first detection module 20) can also obtain the hand jitter situation parameter shanke according to the gyroscope parameters of the current frame and the motion detection algorithm. When the first exposure time is within the preset range, if the maximum gyroscope parameter max Gyro is greater than 0.4, or the hand jitter situation parameter shanke is greater than 100, it indicates that there is global jitter in the current frame image. At this time, the first detection result can also be determined to be the second result.

[0046] In some embodiments, the image processing method may further include determining whether there is a target object in the current frame preview image. For example, in some embodiments, the current frame preview image can be input into a trained image recognition model, and the preview image is subjected to convolutional pooling processing to obtain feature information, and then whether there is a target object in the current frame preview image is recognized according to the feature information. Of course, other methods can also be used to determine whether there is a target object in the preview image, which is not limited herein. It should be noted that in some embodiments, the target object can be a pet, such as a cat, a dog, a rabbit, etc.

[0047] When there is a preset target object in the current frame preview image and the first detection result is the first result, that is, when there is a target object in the current scene and there is no movement, the processor 200 (or the second acquisition module 30) acquires the first image to be processed using the first exposure time. For example, assuming that the first exposure time of the previous frame preview image is 16 ms, when there is a preset target object in the current frame preview image and the first detection result is the first result, the processor 200 (or the second acquisition module 30) acquires the first image to be processed using the first exposure time, that is, the exposure time (i.e., duration) of the first image to be processed is 16 ms. When there is a preset target object in the current frame preview image and the first detection result is the second result, that is, when there is a target object in the current scene and there is movement, the processor 200 (or the second acquisition module 30) acquires the first image to be processed using the second exposure time, where the second exposure time is less than the first exposure time. Since there is movement in the current scene, acquiring the first image to be processed using a second exposure time shorter than the first exposure time is beneficial for obtaining a clear image.

[0048] It should be noted that if there is a preset target object in the current frame preview image, it can be predicted that there will also be a target object in the first image to be processed. In addition, in some embodiments, when there is a preset target object in the current frame preview image and the first detection result is the second result, it is not simply to acquire the first image to be processed using a shorter exposure time, but is related to the first exposure time for acquiring the current frame preview image.

[0049] Specifically, in some embodiments, when there is no target object in the current frame preview image and the first detection result is the second result, the image processing method further includes: obtaining a second exposure time corresponding to the first exposure time according to the first exposure time and a preset first exposure adjustment strategy.

[0050] Please refer to Figure 2 and Figure 3 , in some embodiments, the second acquisition module 30 and the processor 200 can also be used to obtain a second exposure time corresponding to the first exposure time according to the first exposure time and a preset first exposure adjustment strategy.

[0051] Exemplarily, as Figure 6 shown, in some embodiments, the correspondence between the first exposure time and the preset first exposure adjustment strategy is stored in the terminal 1000 (or the image processing device 100). For example, as shown in the second row of Figure 6 , if the first exposure time is in the range of [15 ms, 20 ms), the first exposure adjustment strategy is to adjust the exposure time to 8 ms; as shown in Figure 6As shown in the 3rd line of , if the first exposure time is within the range of [20 ms, 25 ms), the first exposure adjustment strategy is to adjust the exposure time to 10 ms. When there is a preset target object in the current frame preview image and the first detection result is the second result, the processor 200 (or the second acquisition module 30) can obtain the second exposure time corresponding to the first exposure time according to the first exposure time and the preset first exposure adjustment strategy, and then obtain the first image to be processed using the second exposure time. For example, assume that the first exposure time is 16 ms, and the corresponding relationship between the first exposure time and the preset first exposure adjustment strategy is: when the first exposure time is within the range of [15 ms, 20 ms), the first exposure adjustment strategy is to adjust the exposure time to 8 ms. At this time, when there is a preset target object in the current frame preview image and the first detection result is the second result, the processor 200 (or the second acquisition module 30) can obtain that the exposure time is adjusted to 8 ms according to the first exposure time and the preset first exposure adjustment strategy, that is, the second exposure time corresponding to the current first exposure time is 8 ms. Subsequently, the first image to be processed is obtained using the second exposure time, that is, the exposure of the first image to be processed is 8 ms.

[0052] In some other embodiments, a functional relationship between the first exposure time and the second exposure time is stored in the terminal 1000 (or the image processing device 100). After obtaining the first exposure time, the second exposure time can be calculated through the functional relationship between the first exposure time and the second exposure time. Exemplarily, in some embodiments, the functional relationship between the first exposure time and the second exposure time is T2 = T1 / N1, and N1 > 1. Where T1 is the first exposure time, T2 is the second exposure time, and N1 is the first strategy parameter, that is, the second exposure time is equal to the first exposure time divided by the first strategy parameter. Further, in some embodiments, the magnitude relationship between the first strategy parameter N1 and the local motion detection amount is relevant. For example, the larger the local motion detection amount, the larger the corresponding first strategy parameter N1.

[0053] After obtaining the first image to be processed, the processor 200 (or the first processing module 40) performs frequency-domain fusion on at least one frame of preview image and the first image to be processed to obtain a target image. Exemplarily, the processor 200 (or the first processing module 40) performs a two-dimensional discrete Fourier transform or a wavelet transform on at least one frame of preview image and the first image to be processed, converts the image from the image space to the frequency-domain space, and then performs image fusion processing on the image converted to the frequency-domain space (including but not limited to multi-frame denoising processing, ghost removal processing, HDR fusion processing, etc.) to obtain a fusion-processed image. After obtaining the fusion-processed image, the fusion-processed image is then converted back from the frequency-domain space to the image space to obtain a target image. Since the image containing the target object (such as a pet) may have a scene with high-frequency details, the frequency-domain fusion of the image in the scene where high-frequency details may exist in this embodiment can overcome some loss of signal-to-noise ratio or abrupt noise reduction smear, which is beneficial to improving the visual perception effect.

[0054] Please refer to Figure 7 , in some embodiments, the image processing method may further include:

[0055] 05: In the case where the target object does not exist in the current frame of preview image, perform a second motion detection on the current frame of preview image to obtain a second detection result, and the second motion detection is different from the first motion detection;

[0056] 06: Select an exposure time and an algorithm according to the first detection result and the second detection result; and

[0057] 07: Obtain a second image to be processed by using the selected exposure time, and fuse the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain a target image.

[0058] Please combine Figure 2 and Figure 3, in some embodiments, the image processing apparatus 100 further includes a second detection module 50, a selection module 60, and a second processing module 70. Both the second detection module 50 and the processor 200 can also be used to execute the method in 05, both the selection module 60 and the processor 200 can also be used to execute the method in 06, and both the second processing module 70 and the processor 200 can also be used to execute the method in 07. That is to say, when there is no target object in the current frame preview image, both the second detection module 50 and the processor 200 can also be used to perform a second motion detection on the current frame preview image to obtain a second detection result, and the second motion detection is different from the first motion detection. Both the selection module 60 and the processor 200 can also be used to select an exposure time and an algorithm according to the first detection result and the second detection result. Both the second processing module 70 and the processor 200 can also be used to obtain a second image to be processed using the selected exposure time, and fuse the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain a target image.

[0059] When there is no target object in the current frame preview image, the processor 200 (second detection module 50) performs a second motion detection on the current frame preview image to obtain a second detection result. Among them, the second motion detection is different from the first motion detection. In some embodiments, the second detection result includes a third result, a fourth result, and a fifth result. The third result is used to characterize the presence of fast motion in the current scene, the fourth result is used to characterize the presence of slight motion in the current scene, and the fifth result is used to characterize the absence of motion in the current scene.

[0060] Specifically, please refer to Figure 8 , in some embodiments, performing a second motion detection on the current frame preview image to obtain a second detection result includes:

[0061] 051: Obtain a second detection result according to at least one of the offset mean between the region of interest box in the current frame preview image and the region of interest box in the previous frame preview image, the brightness difference between multiple pixel points in the background region of the current frame preview image and the corresponding pixel points in the previous frame preview image, and the gyroscope parameters corresponding to the current frame preview image.

[0062] Please combine Figure 2 and Figure 3, in some embodiments, the second detection module 50 and the processor 200 can also be used to execute the method in 051. That is to say, the second detection module 50 and the processor 200 can also be used to obtain a second detection result according to at least one of the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image, the luminance difference between multiple pixel points in the background region of the current frame preview image and the corresponding pixel points in the previous frame preview image, and the gyroscope parameters corresponding to the current frame preview image.

[0063] In some embodiments, the processor 200 (or the second detection module 50) can obtain a second detection result according to the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image. Exemplarily, please refer to Figure 9 , perform a second motion detection on the current frame preview image to obtain a second detection result, including:

[0064] 0511: Obtain a second detection result according to the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image.

[0065] Please combine Figure 2 and Figure 3 , in some embodiments, the second detection module 50 and the processor 200 can also be used to execute the method in 0511. That is to say, the second detection module 50 and the processor 200 can also be used to obtain a second detection result according to the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image.

[0066] Specifically, in some embodiments, obtaining a second detection result according to the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image includes: obtaining the offset mean between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image; if the offset mean is greater than a preset first offset mean, determining the second detection result as a third result, that is, there is fast motion; if the offset mean is greater than a preset second offset mean and less than the first offset mean, determining the second detection result as a fourth result, that is, there is slight motion. Wherein, the second offset mean is less than the first offset mean. It should be noted that the first offset mean and the second offset mean can be determined by the manufacturer through a large amount of experimental data before the terminal 1000 (or the image processing device 100) leaves the factory, or can be set by the user according to their own needs, and are not limited herein.

[0067] In the case where the target object does not exist in the current frame preview image, the processor 200 (or the second detection module 50) performs subject recognition on the current frame preview image and the previous frame preview image, and sets the region of interest (ROI) boxes in the current frame preview image and the previous frame preview image respectively according to the subject recognition results in the images. In some embodiments, the offset mean value between the ROI box in the current frame preview image and the ROI box in the previous frame preview image may be calculated based on the coordinate changes of the ROI boxes in the two frame preview images. Exemplarily, a first ROI box is set in the current frame preview image, and the subject of the current frame preview image is included in the first ROI box. A second ROI box is set in the previous frame preview image, and the subject of the previous frame preview image is included in the second ROI box. Arbitrarily select the coordinates of multiple reference points in the first ROI box and the coordinates of the corresponding points in the second ROI that correspond to the selected reference points, calculate the coordinate changes between the multiple reference points and their corresponding points, so as to obtain the offset mean value between the ROI box in the current frame preview image and the ROI box in the previous frame preview image. For example, as Figure 10 shown Figure 10 in the right image is the current frame preview image, in which a first ROI box I1 is set; Figure 10 the left image in is the previous frame preview image, in which a second ROI box I2 is set. Select the upper left corner A point, the lower right corner B point, and the midpoint C point on the upper side of the first ROI box I1 as reference points; then the upper left corner a point of the second ROI box 12 is the corresponding point of the reference point A point; the lower right corner b point of the second ROI box I2 is the corresponding point of the reference point B point; the midpoint c point on the upper side of the second ROI box I2 is the corresponding point of the reference point C point. Establish a coordinate system with the lower left corner as the origin of coordinates in the current frame preview image, and establish a coordinate system with the lower left corner (the same as the lower left corner in the current frame preview image) as the origin of coordinates in the previous frame preview image. Then obtain the coordinates of points A, B, and C in the current frame preview image, and obtain the coordinates of points a, b, and c in the previous frame preview image; subsequently, calculate the distance H1 between the coordinates of point A and the coordinates of point a, calculate the distance H2 between the coordinates of point B and the coordinates of point b, and calculate the distance H3 between the coordinates of point C and the coordinates of point c. Then calculate the mean value of the distance H1, the distance H2, and the distance H3 to obtain the offset mean value.

[0068] It should be noted that since the previous frame preview image and the current frame preview image are two consecutive frame preview images, the subjects in the two frame preview images should be roughly the same, that is, the same subject is included in the ROI boxes in the two frame preview images. For example, the subject can be a human face, a portrait, etc. By calculating the offset amount between the ROI boxes containing the subject in the two frame preview images, the movement situation of the subject in the two frame preview images can be judged.

[0069] After obtaining the average offset between the region of interest (ROI) box in the previous frame preview image and the ROI box in the previous frame preview image, compare the magnitude relationship between the average offset and the first average offset, and compare the magnitude relationship between the average offset and the second average offset, where the second average offset is less than the first average offset. If the average offset is greater than the first average offset, it indicates that the offset between the ROI boxes in the two frame preview images is very large and there is rapid local movement in the main region, then determine the second detection result as the third result; if the average offset is greater than the second average offset and less than the first average offset, it indicates that the offset between the ROI boxes in the two frame preview images is relatively large and there is slight local movement in the main region, then determine the second detection result as the fourth result. If the average offset is not greater than the second average offset, it indicates that the offset between the ROI boxes in the two frame preview images is relatively small and it can be considered that there is no movement in the main region, then determine the second detection result as the fifth result.

[0070] In particular, in some embodiments, it is also possible to keep the positions of the ROI boxes in two consecutive frame preview images unchanged, and by calculating the information difference within the ROI boxes in the two frame preview images, determine the movement situation of the main region, thereby obtaining the second detection result. In this way, only the main body recognition of one frame preview image is required. Compared with obtaining the second detection result based on the average offset between the ROI box in the current frame preview image and the ROI box in the previous frame preview image, it can reduce power consumption and improve the detection speed of the second motion detection.

[0071] Exemplarily, in some embodiments, the processor 200 (or the second detection module 50) performs subject recognition on the current frame preview image, and sets a first region of interest (ROI) box in the current frame preview image according to the subject recognition result in the image. A second ROI box is set at the corresponding position in the previous frame preview image, and the pixel differences of the corresponding pixels in the first ROI box and the second ROI box are calculated respectively to obtain the information difference between the image in the first ROI box and the image in the second ROI box. Subsequently, the magnitude relationship between the information difference and a preset first information difference, and the magnitude relationship between the information difference and a preset second information difference are compared, wherein the second information difference is less than the first information difference. If the information difference is greater than the first information difference, it indicates that the offset between the ROI boxes in the two frame preview images is very large and the subject area has rapid movement, and the second detection result is determined as the third result; if the information difference is greater than the second information difference and less than the first information difference, it indicates that the offset between the ROI boxes in the two frame preview images is relatively large and the subject area has slight movement, and the second detection result is determined as the fourth result. If the information difference is less than the first information difference, it indicates that the offset between the ROI boxes in the two frame preview images is relatively small and the subject area has no movement, and the second detection result is determined as the fifth result. It should be noted that the first information difference and the second information difference can be determined by the manufacturer through a large amount of experimental data before the terminal 1000 (or the image processing device 100) leaves the factory, or can be set by the user according to their own needs, and no limitation is made here.

[0072] In some embodiments, the processor 200 (or the second detection module 50) can also obtain the second detection result according to the brightness difference between multiple pixel points located in the background area in the current frame preview image and the corresponding pixel points in the previous frame preview image. Exemplarily, please refer to Figure 11 , for performing a second motion detection on the current frame preview image to obtain the second detection result, further includes:

[0073] 0512: Obtain the second detection result according to the brightness difference between multiple pixel points located in the background area in the current frame preview image and the corresponding pixel points in the previous frame preview image.

[0074] Please combine Figure 2 and Figure 3 , in some embodiments, both the second detection module 50 and the processor 200 can also be used to execute the method in 0512. That is to say, both the second detection module 50 and the processor 200 can also be used to obtain the second detection result according to the brightness difference between multiple pixel points located in the background area in the current frame preview image and the corresponding pixel points in the previous frame preview image.

[0075] Specifically, in some embodiments, when there is no target object in the current frame preview image, the processor 200 (or the second detection module 50) calculates the brightness difference between multiple pixel points located in the background area of the current preview image and the corresponding pixel points in the previous frame preview image. Subsequently, the magnitudes of multiple brightness differences are compared with a preset first threshold, and the magnitudes of multiple brightness differences are compared with a preset second threshold, where the second threshold is less than the first threshold. If there is a brightness difference greater than the first threshold, it indicates that there is rapid local movement in the background area, and the second detection result is determined to be the third result; if there is a brightness difference greater than the second threshold and less than the first threshold, it indicates that there is slight local movement in the background area, and the second detection result is determined to be the fourth result; if the average value of the brightness differences corresponding to a preset number of pixel points is greater than a preset average value, it indicates that there is global movement, and the second detection result is determined to be the fourth result. If none of the multiple brightness differences is greater than the second threshold, and the average value of the brightness differences corresponding to a preset number of pixel points is also not greater than the preset average value, it can be considered that there is no movement in the background area, and the second detection result is determined to be the fifth result. It should be noted that the first threshold and the second threshold can be determined by the manufacturer through a large amount of experimental data before the terminal 1000 (or the image processing device 100) leaves the factory, or can be set by the user according to their own needs, and there is no limitation here.

[0076] In some embodiments, the processor 200 (or the second detection module 50) can also obtain the second detection result according to the gyroscope parameters corresponding to the current frame preview image. By way of example, please refer to Figure 12 for performing a second motion detection on the current frame preview image to obtain the second detection result, which further includes:

[0077] Please refer to Figure 12 for performing a second motion detection on the current frame preview image to obtain the second detection result, which further includes:

[0078] 0513: Obtain the second detection result according to the gyroscope parameters corresponding to the current frame preview image.

[0079] Please combine Figure 2 and Figure 3 In some embodiments, both the second detection module 50 and the processor 200 can also be used to execute the method in 0513. That is to say, both the second detection module 50 and the processor 200 can also be used to obtain the second detection result according to the gyroscope parameters corresponding to the current frame preview image.

[0080] Specifically, in some embodiments, when there is no target object in the current frame preview image, the processor 200 (or the second detection module 50) obtains a plurality of gyroscope parameters corresponding to the current frame preview image and calculates the mean value of the plurality of gyroscope parameters. Subsequently, the magnitudes of the plurality of gyroscope parameters are compared with a first preset value and a second preset value, where the second preset value is less than the first preset value. If the mean value of the plurality of gyroscope parameters is greater than the first preset value, it indicates that there is a fast global motion, and the second detection result is determined to be the third result; if the mean value of the plurality of gyroscope parameters is greater than the second preset value and less than the first preset value, it indicates that there is a slight global motion, and the second detection result is determined to be the fourth result. If the mean value of the plurality of gyroscope parameters is less than the second preset value, it indicates that there is no global motion, and the second detection result is determined to be the fifth result. In particular, in some embodiments, if the previous frame preview image is a fast global motion, even if it is determined according to the gyroscope that the current frame preview image is a slight global motion, the second detection result is still determined to be the third result, which is more conducive to subsequent selection of a more suitable exposure time and image processing algorithm. It should be noted that the first preset value and the second preset value can be determined by the manufacturer through a large amount of experimental data before the terminal 1000 (or the image processing device 100) leaves the factory, or can be set by the user according to their own needs, and are not limited herein.

[0081] In some embodiments, the processor 200 (or the second detection module 50) can also obtain the second detection result according to the offset mean value between the region of interest frame in the current frame preview image and the region of interest frame in the previous frame preview image, the brightness difference between a plurality of pixel points located in the background region in the current frame preview image and the corresponding pixel points in the previous frame preview image, and the gyroscope parameters corresponding to the current frame preview image. By way of example, please refer to Figure 13 , perform a second motion detection on the current frame preview image to obtain the second detection result, and further includes:

[0082] 0514: Obtain the second detection result according to the offset mean value between the region of interest frame in the current frame preview image and the region of interest frame in the previous frame preview image, the brightness difference between a plurality of pixel points located in the background region in the current frame preview image and the corresponding pixel points in the previous frame preview image, and the gyroscope parameters corresponding to the current frame preview image.

[0083] Please combine Figure 2 and Figure 3, in some embodiments, the second detection module 50 and the processor 200 can also be used to execute the method in 0514. That is to say, the second detection module 50 and the processor 200 can also be used to obtain a second detection result according to the offset mean between the region of interest box in the current frame preview image and the region of interest box in the previous frame preview image, the brightness difference between multiple pixel points in the background region of the current frame preview image and the corresponding pixel points in the previous frame preview image, and the gyroscope parameters corresponding to the current frame preview image.

[0084] Specifically, the processor 200 (or the second detection module 50) obtains the motion condition of the main body region according to the offset mean between the region of interest box in the current frame preview image and the region of interest box in the previous frame preview image; obtains the motion condition of the background region according to the brightness difference between multiple pixel points in the background region of the current frame preview image and the corresponding pixel points in the previous frame preview image; and obtains the global motion condition according to the gyroscope parameters corresponding to the preview image.

[0085] Further, the processor 200 (or the second detection module 50) obtains the mean offset between the region-of-interest box in the current frame preview image and the region-of-interest box in the previous frame preview image, and compares the mean offset with the first mean offset and the second mean offset. If the mean offset is greater than the first mean offset, it indicates that the offset between the region-of-interest boxes in the two frame preview images is very large, and there is rapid local movement in the main area. If the mean offset is greater than the second mean offset and less than the first mean offset, it indicates that the offset between the region-of-interest boxes in the two frame preview images is relatively large, and there is slight local movement in the main area. If the mean offset is not greater than the second mean offset, it indicates that the offset between the region-of-interest boxes in the two frame preview images is relatively small, and it can be considered that there is no movement in the main area. The processor 200 (or the second detection module 50) obtains the brightness differences between multiple pixel points located in the background area in the current frame preview image and the corresponding pixel points in the previous frame preview image, and compares the multiple brightness differences with the first threshold and the second threshold. If there is a brightness difference greater than the first threshold, it indicates that there is rapid local movement in the background area. If there is a brightness difference greater than the second threshold and less than the first threshold, it indicates that there is slight local movement in the background area. If the mean value of the brightness differences corresponding to a preset number of pixel points is greater than a preset mean value, it indicates that there is global movement. If the multiple brightness differences are not greater than the second threshold, and the mean value of the brightness differences corresponding to a preset number of pixel points is also not greater than the preset mean value, it can be considered that there is no movement in the background area. The processor 200 (or the second detection module 50) obtains multiple gyroscope parameters corresponding to the current frame preview image and calculates the mean value of the multiple gyroscope parameters. Subsequently, the multiple gyroscope parameters are compared with the first preset value and the second preset value. If the mean value of the multiple gyroscope parameters is greater than the first preset value, it indicates that there is rapid global movement. If the mean value of the multiple gyroscope parameters is greater than the second preset value and less than the first preset value, it indicates that there is slight global movement. If the mean value of the multiple gyroscope parameters is less than the second preset value, it indicates that there is no global movement. Particularly, in some embodiments, if the previous frame preview image is rapid global movement, even if the mean value of the gyroscope parameters is greater than the second preset value and less than the first preset value, it is still considered that there is global rapid movement.

[0086] After obtaining the motion conditions of the main body area, the background area, and the global motion, it is necessary to obtain the second detection result by combining the motion conditions of the main body area, the background area, and the global motion in the preview image. Exemplarily, as long as at least one of the three motion conditions (including the motion condition of the main body area, the motion condition of the background area, and the global motion condition) is fast motion, it is determined that the second detection result is the third result; if none of the three motion conditions is fast motion, but at least one condition is slight motion, it is determined that the second detection result is the fourth result; if none of the three motion conditions is motion, it is determined that the second detection result is the fifth result. For example, assume that it is determined that there is fast local motion in the main body area, slight local motion in the background area, and global motion, then it is determined that the second detection result is the third result; assume that it is determined that there is no local motion in the main body area, slight local motion in the background area, and global motion, then it is determined that the second detection result is the fourth result; assume that it is determined that there is no local motion in the main body area, no motion in the background area, and no global motion, then it is determined that the second detection result is the fifth result.

[0087] After obtaining the second detection result, the processor 200 (or the selection module 60) selects the exposure time and algorithm according to the first detection result and the second detection result. Subsequently, the processor 200 (or the second processing module 70) obtains the second image to be processed using the selected exposure time, and fuses the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain the target image.

[0088] Specifically, please refer to Figure 7 and Figure , in some embodiments, step 06: Select the exposure time and algorithm according to the first detection result and the second detection result, including:

[0089] 061: When the second detection result is the third result, the selected exposure time is the third exposure time, and the selected algorithm is the multi-frame noise reduction algorithm; wherein, the third result is used to characterize the existence of fast motion in the current scene, and the third exposure time is less than the first exposure time.

[0090] At this time, step 07: Obtain the second image to be processed using the selected exposure time, and fuse the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain the target image, including:

[0091] 071: Obtain the second image to be processed using the third exposure time, and perform multi-frame noise reduction processing on the second image to be processed according to at least one frame of preview image to obtain the target image.

[0092] Please combine ​ and ​, in some embodiments, both the selection module 60 and the processor 200 can also be used to execute the method in 061, and both the second processing module 70 and the processor 200 can also be used to execute the method in 071. That is to say, both the selection module 60 and the processor 200 can also be used to select an exposure time of the third exposure time and an algorithm of the multi-frame noise reduction algorithm when the second detection result is the third result; wherein, the third result is used to indicate that there is fast movement in the current scene, and the third exposure time is less than the first exposure time. Both the second processing module 70 and the processor 200 can also be used to obtain a second image to be processed using the third exposure time and perform multi-frame noise reduction processing on the second image to be processed based on at least one preview image to obtain a target image.

[0093] When there is no target object in the current frame preview image and after obtaining the second detection result, it is determined whether the second detection result is the third result. When the second detection result is the third result, it indicates that the second motion detection determines that there is fast movement in the current scene. At this time, the exposure time selected by the processor 200 (or the selection module 60) is the third exposure time, and the selected algorithm is the multi-frame noise reduction algorithm, wherein the third exposure time is less than the first exposure time. Subsequently, the processor 200 (or the second processing module 70) obtains a second image to be processed using the third exposure time and performs multi-frame noise reduction processing on the second image to be processed based on at least one preview image to obtain a target image. Of course, in some embodiments, other image fusion processing of at least one preview image and the second image to be processed in the spatio-temporal domain can also be performed (the above multi-frame noise reduction processing is also performed in the image spatial domain), which is not limited herein.

[0094] On the one hand, since the second image to be processed is obtained using the third exposure time less than the first exposure time in a scene with fast movement, it is beneficial to obtain a clear image; on the other hand, since the time-consuming of frequency domain fusion is about twice that of spatio-temporal domain fusion, in a scene where there is no target object (such as a pet) and there is fast movement, there is no obvious advantage in using frequency domain fusion compared to normal spatio-temporal domain fusion in terms of effect. Therefore, when there is no target object and the second detection result is the third result, using multi-frame noise reduction processing (or spatio-temporal domain fusion) can speed up the processing while obtaining a clear image.

[0095] Please refer to ​ and ​ , in some embodiments, step 06: Select an exposure time and an algorithm according to the first detection result and the second detection result, including:

[0096] 062: When the second detection result is the fourth result or the fifth result, and the first detection result is the second result, the selected exposure time is the third exposure time, and the selected algorithm is frequency domain fusion and network noise reduction; wherein, the fourth result is used to represent the existence of slight movement, and the fifth result is used to represent the non-existence of movement.

[0097] At this time, step 07: Obtain the second image to be processed by using the selected exposure time, and fuse the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain the target image, including:

[0098] 072: Obtain the second image to be processed by using the third exposure time, and perform frequency domain fusion on at least one frame of preview image and the second image to be processed to obtain a fused intermediate image; and

[0099] 073: Perform network noise reduction on the fused intermediate image to obtain the target image.

[0100] Please combine ​ and ​ , in some embodiments, the selection module 60 and the processor 200 can also be used to execute the method in 062, and the second processing module 70 and the processor 200 can also be used to execute the methods in 072 and 073. That is to say, the selection module 60 and the processor 200 can also be used to select the third exposure time and the selected algorithm is frequency domain fusion and network noise reduction when the second detection result is the fourth result or the fifth result, and the first detection result is the second result; wherein, the fourth result is used to represent the existence of slight movement in the current scene, and the fifth result is used to represent the non-existence of movement in the current scene. The second processing module 70 and the processor 200 can also be used to obtain the second image to be processed by using the third exposure time, and perform frequency domain fusion on at least one frame of preview image and the second image to be processed to obtain a fused intermediate image; and perform network noise reduction on the fused intermediate image to obtain the target image.

[0101] The target object does not exist in the preview image of the current frame, and a second detection result is obtained. When the second detection result is not the third result and the first detection result is the second result, that is, when the second detection result is the fourth result or the fifth result and the first detection result is the second result, it indicates that the second motion detection determines that there is no fast motion in the current scene, and the first motion detection determines that there is motion in the current scene. At this time, the exposure time selected by the processor 200 (or the selection module 60) is the third exposure time, and the selected algorithm is frequency domain fusion and network noise reduction, where the third exposure time is less than the first exposure time. Subsequently, the processor 200 (or the second processing module 70) obtains a second image to be processed using the third exposure time, and performs frequency domain fusion on at least one frame of preview image and the second image to be processed to obtain a fused intermediate image. After obtaining the intermediate image, the processor 200 (or the second processing module 70) performs network noise reduction on the fused intermediate image to obtain a target image. Among them, the specific implementation of performing frequency domain fusion on at least one frame of preview image and the second image to be processed to obtain a fused intermediate image is the same as that described in the above embodiments, and will not be elaborated here. Of course, in some embodiments, network noise reduction on the intermediate image can also be performed in the frequency domain space, which is not limited here.

[0102] On the one hand, since the second image to be processed is obtained using the third exposure time less than the first exposure time in a scene with motion, it is beneficial to obtain a clear image; on the other hand, since there is slight motion in the scene, the scene is likely to be a high-frequency detail motion scene at this time. Thus, using frequency domain fusion in such a high-frequency detail motion scene can overcome some signal-to-noise ratio losses or abrupt noise reduction smear effects, which is beneficial to improving the visual perception effect; on the other hand, if the exposure time for obtaining the second image to be processed is reduced in low light, the noise will be relatively large, and in this embodiment, the image is processed in combination with network noise reduction, so that the image quality of the finally obtained target image can be further optimized.

[0103] It should be noted that when the third exposure time is used to obtain the second image to be processed in this article, it is not simply to obtain the second image to be processed using a shorter exposure time, but is related to the first exposure time for obtaining the preview image of the current frame. Specifically, in some embodiments, when the third exposure time is used to obtain the second image to be processed, the image processing method further includes: obtaining the third exposure time corresponding to the first exposure time according to the first exposure time and a preset second exposure adjustment strategy; where the second exposure adjustment strategy is different from the first exposure adjustment strategy, and the third exposure time corresponding to the same first exposure time is less than the second exposure time.

[0104] Please combine ​ and​ , in some embodiments, the selection module 60 and the processor 200 are further configured to obtain a third exposure time corresponding to the first exposure time according to the first exposure time and a preset second exposure adjustment strategy; wherein, the second exposure adjustment strategy is different from the first exposure adjustment strategy, and the third exposure time corresponding to the same first exposure time is less than the second exposure time.

[0105] Exemplarily, as ​ shown, in some embodiments, a correspondence between the first exposure time and a preset second exposure adjustment strategy is stored in the terminal 1000 (or the image processing device 100). For example, as ​ shown in the second row of ​ , if the first exposure time is in the range of [15 ms, 20 ms), the second exposure adjustment strategy is to adjust the exposure time to 48 ms; as

[0106] shown in the third row of ​As shown in the second column of , if the first exposure time is within the range of [15 ms, 20 ms), the first exposure adjustment strategy is to adjust the exposure time to 8 ms, that is, if the first exposure time is within the range of [15 ms, 20 ms), the corresponding second exposure time is 8 ms; if the first exposure time is within the range of [15 ms, 20 ms), the second exposure adjustment strategy is to adjust the exposure time to 4 ms, that is, if the first exposure time is within the range of [15 ms, 20 ms), the corresponding third exposure time is 4 ms. Assume that the first exposure time of the current frame preview image is 18 ms, then the third exposure time corresponding to this first exposure time is 4 ms, and the second exposure time corresponding to this first exposure time is 8 ms, that is, the third exposure time corresponding to the same first exposure time is less than the second exposure time.

[0107] In some other embodiments, a functional relationship between the first exposure time and the third exposure time is stored in the terminal 1000 (or the image processing device 100). After obtaining the first exposure time, the third exposure time can be calculated through the functional relationship between the first exposure time and the third exposure time. Exemplarily, in some embodiments, the functional relationship between the first exposure time and the third exposure time is T3 = T1 / N2, and N2 > 1. Where T1 is the first exposure time, T3 is the third exposure time, and N2 is the second policy parameter, that is, the third exposure time is equal to the first exposure time divided by the second policy parameter. Further, in some embodiments, the motion speed can also be obtained according to the time difference between the current frame preview image and the previous frame preview image and the local motion detection amount. The magnitude relationship between the second policy parameter N2 and the motion speed is relevant. For example, the greater the motion speed, the greater the corresponding second policy parameter N2.

[0108] Please refer to ​ and ​ , in some embodiments, step 06: Select the exposure time and algorithm according to the first detection result and the second detection result, including:

[0109] 063: When the second detection result is the fourth result and the first detection result is the first result, the selected exposure time is the second exposure time, and the selected algorithm is multi-frame noise reduction;

[0110] At this time, step 07: Obtain the second image to be processed using the selected exposure time, and fuse the second image to be processed and at least one frame of preview image according to the selected algorithm to obtain the target image, including:

[0111] 074: Obtain the second image to be processed using the second exposure time, and perform multi-frame noise reduction processing on the second image to be processed according to at least one frame of preview image to obtain the target image.

[0112] Please combine with ​and ​ , in some embodiments, the selection module 60 and the processor 200 can also be used to execute the method in 063, and the second processing module 70 and the processor 200 can also be used to execute the method in 074. That is to say, the selection module 60 and the processor 200 can also be used to select the exposure time as the second exposure time and the algorithm as multi-frame noise reduction when the second detection result is the fourth result and the first detection result is the first result. The second processing module 70 and the processor 200 can also be used to obtain a second image to be processed by using the second exposure time, and perform multi-frame noise reduction processing on the second image to be processed according to at least one preview image to obtain a target image.

[0113] When there is no target object in the current frame preview image, a second detection result is obtained. When the second detection result is the fourth result and the first detection result is the first result, it indicates that the second motion detection determines that there is slight motion in the current scene, and the first motion detection determines that there is no motion in the current scene. At this time, the exposure time selected by the processor 200 (or the selection module 60) is the second exposure time, and the selected algorithm is multi-frame noise reduction, where the second exposure time is less than the first exposure time. Subsequently, the processor 200 (or the second processing module 70) obtains a second image to be processed by using the second exposure time, and performs multi-frame noise reduction processing on the second image to be processed according to at least one preview image to obtain a target image.

[0114] On the one hand, since the second exposure time less than the first exposure time is used to obtain the second image to be processed when the second motion detection determines that there is slight motion, it is beneficial to obtain a clear image; on the other hand, since the time-consuming of frequency domain fusion is about 2 times that of spatio-temporal domain fusion, when there is no target object (such as a pet) and the first motion detection determines that there is no motion, using frequency domain fusion has no obvious advantage in effect compared with normal spatio-temporal domain fusion. Therefore, in this embodiment, using multi-frame noise reduction processing (or spatio-temporal domain fusion) can speed up the processing speed while obtaining a clear image.

[0115] In some embodiments, when there is no target object in the current frame preview image, a second detection result is obtained. When the second detection result is the fifth result and the first detection result is the first result, it indicates that the second motion detection determines that there is no motion in the current scene, and the first motion detection determines that there is no motion in the current scene. At this time, the exposure time selected by the processor 200 (or the selection module 60) is the first exposure time, and the selected algorithm is multi-frame noise reduction. Subsequently, the processor 200 (or the second processing module 70) obtains a second image to be processed by using the first exposure time, and performs multi-frame noise reduction processing on the second image to be processed according to at least one preview image to obtain a target image.

[0116] Please refer to ​ and​ In an embodiment of the present application, a non-volatile computer-readable storage medium 400 including a computer program 401 is further provided. When the computer program 401 is executed by one or more processors 200, the processors 200 are caused to execute the image processing method described in the above embodiments. For example, the processors 200 are caused to execute the image processing methods in 01, 02, 03, 04, 021, 022, 023, 0231, 0232, 05, 06, 07, 051, 0511, 0512, 0513, 0514, 061, 071, 062, 072, 073, 063, and 074.

[0117] For example, when the computer program 401 is executed by one or more processors 200, the processors 200 are caused to execute the following method:

[0118] 01: Obtain multiple consecutive preview images and the first exposure time of the current frame preview image;

[0119] 02: Perform a first motion detection on the current frame preview image to obtain a first detection result;

[0120] 03: When a preset target object exists in the current frame preview image and the first detection result is the first result, obtain a first image to be processed using the first exposure time; when a target object exists in the current frame preview image and the first detection result is the second result, obtain a first image to be processed using the second exposure time; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time;

[0121] 04: Perform frequency domain fusion on at least one frame of preview image and the first image to be processed to obtain a target image.

[0122] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0123] Any process or method description set forth in the flowchart or otherwise described herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where functions may be performed in a substantially simultaneous manner or in an order opposite to that shown or discussed, according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0124] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that, Including: Obtaining multiple consecutive preview images and the first exposure time of the current frame of the preview image; Performing a first motion detection on the current frame of the preview image to obtain a first detection result; When there is a preset target object in the current frame of the preview image and the first detection result is the first result, obtaining a first image to be processed using the first exposure time; when there is the target object in the current frame of the preview image and the first detection result is the second result, obtaining a first image to be processed using a second exposure time; wherein, the first result is used to represent that there is no motion in the current scene, the second result is used to represent that there is motion in the current scene, and the second exposure time is less than the first exposure time; Performing frequency-domain fusion on at least one frame of the preview image and the first image to be processed to obtain a target image; When there is no such target object in the current frame of the preview image, performing a second motion detection on the current frame of the preview image to obtain a second detection result, and the second motion detection is different from the first motion detection; Selecting an exposure time and an algorithm according to the first detection result and the second detection result; and Obtaining a second image to be processed using the selected exposure time, and fusing the second image to be processed and at least one frame of the preview image according to the selected algorithm to obtain the target image.

2. The image processing method according to claim 1, wherein When there is no such target object in the current frame of the preview image and the first detection result is the second result, the image processing method further includes: Obtaining the second exposure time corresponding to the first exposure time according to the first exposure time and a preset first exposure adjustment strategy.

3. The image processing method according to claim 1, wherein The performing a first motion detection on the current frame of the preview image to obtain a first detection result includes: Obtaining a detection threshold corresponding to the gyroscope parameter according to the gyroscope parameter corresponding to the current frame of the preview image; Obtaining a local motion detection amount of the current frame of the preview image according to multiple frames of the preview image; Obtaining a first detection result according to the local motion detection amount and the detection threshold.

4. The image processing method according to claim 3, wherein The obtaining a first detection result according to the local motion detection amount and the detection threshold includes: When the local motion detection amount is less than the detection threshold, or the first exposure time is not within a preset range, determining that the first detection result is the first result; and When the local motion detection amount is greater than the detection threshold and the first exposure time is within a preset range, determining that the first detection result is the second result.

5. The image processing method according to claim 1, wherein The selecting an exposure time and an algorithm according to the first detection result and the second detection result includes: When the second detection result is the third result, the selected exposure time is the third exposure time, and the selected algorithm is a multi-frame noise reduction algorithm; wherein, the third result is used to represent the existence of fast motion, and the third exposure time is less than the first exposure time; Obtaining a second image to be processed using the selected exposure time, and fusing the second image to be processed and at least one frame of the preview images according to the selected algorithm to obtain the target image, includes: Obtaining the second image to be processed using the third exposure time, and performing multi-frame noise reduction processing on the second image to be processed according to at least one frame of the preview images to obtain the target image.

6. The image processing method according to claim 1, wherein Selecting an exposure time and an algorithm according to the first detection result and the second detection result, includes: When the second detection result is the fourth result or the fifth result, and the first detection result is the second result, the selected exposure time is the third exposure time, and the selected algorithm is frequency domain fusion and network noise reduction; wherein, the fourth result is used to represent the existence of slight movement, and the fifth result is used to represent the non-existence of movement; Obtaining a second image to be processed using the selected exposure time, and fusing the second image to be processed and at least one frame of the preview images according to the selected algorithm to obtain the target image, includes: Obtaining the second image to be processed using the third exposure time, and performing frequency domain fusion on at least one frame of the preview images and the second image to be processed to obtain a fused intermediate image; and Performing network noise reduction on the fused intermediate image to obtain the target image.

7. The image processing method according to claim 5 or 6, characterized in that, The image processing method further includes: Obtaining the third exposure time corresponding to the first exposure time according to the first exposure time and a preset second exposure adjustment strategy; wherein, the second exposure adjustment strategy is different from the first exposure adjustment strategy, and the third exposure time corresponding to the same first exposure time is less than the second exposure time.

8. The image processing method according to claim 1, wherein Selecting an exposure time and an algorithm according to the first detection result and the second detection result, includes: When the second detection result is the fourth result and the first detection result is the first result, the selected exposure time is the second exposure time, and the selected algorithm is multi-frame noise reduction; Obtaining a second image to be processed using the selected exposure time, and fusing the second image to be processed and at least one frame of the preview images according to the selected algorithm to obtain the target image, includes: Obtaining the second image to be processed using the second exposure time, and performing multi-frame noise reduction processing on the second image to be processed according to at least one frame of the preview images to obtain the target image.

9. The image processing method according to claim 1, wherein Performing a second motion detection on the current frame of the preview image to obtain a second detection result, includes: Obtaining the second detection result according to at least one of the offset mean between the region of interest box in the current frame of the preview image and the region of interest box in the previous frame of the preview image, the brightness difference between multiple pixel points located in the background region in the current frame of the preview image and the corresponding pixel points in the previous frame of the preview image, and the gyroscope parameters corresponding to the current frame of the preview image.

10. An image processing apparatus, characterized in that, Includes: A first acquisition module, configured to acquire multiple consecutive frames of preview images and the first exposure time of the current frame of the preview image; The first detection module is configured to perform a first motion detection on the preview image of the current frame to obtain a first detection result; The second acquisition module is configured to, when a preset target object exists in the preview image of the current frame and the first detection result is the first result, acquire a first image to be processed by using the first exposure time; and when the preset target object exists in the preview image of the current frame and the first detection result is the second result, acquire a first image to be processed by using a second exposure time; wherein the first result is used to represent no motion, the second result is used to represent motion, and the second exposure time is less than the first exposure time; and The first processing module is configured to perform frequency domain fusion on at least one frame of the preview image and the first image to be processed to obtain a target image; The apparatus further includes: The second detection module is configured to, when the target object does not exist in the preview image of the current frame, perform a second motion detection on the preview image of the current frame to obtain a second detection result, where the second motion detection is different from the first motion detection; The selection module is configured to select an exposure time and an algorithm according to the first detection result and the second detection result; and The second processing module is configured to acquire a second image to be processed by using the selected exposure time, and fuse the second image to be processed and at least one frame of the preview image according to the selected algorithm to obtain the target image.

11. A terminal, characterized in that, The terminal includes: One or more processors and a memory; and One or more programs, where one or more of the programs are stored in the memory and are executed by one or more of the processors, and the programs include instructions for performing the image processing method according to any one of claims 1 to 9.

12. A non-volatile computer-readable storage medium storing a computer program, which, when executed by one or more processors, implements the image processing method according to any one of claims 1 to 9.

Images