Image processing method and device, equipment and medium

Abstract

The invention discloses an image processing method and device, equipment and a medium, and belongs to the technical field of image processing. The method comprises the steps that double exposure is carried out on a target scene, a long exposure image frame and a short exposure image frame are obtained, the exposure duration of the long exposure image frame is first duration, the exposure duration of the short exposure image frame is second duration, and the second duration is smaller than the first duration; determining difference information between the definition of the long exposure image frame and the definition of the short exposure image frame; and determining the motion state of the target scene according to the difference information.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, equipment and medium. Background Art

[0002] In related technologies, image processing mainly determines the motion state of an object by comparing the pixel differences between adjacent frames. This image processing method is often affected by factors such as lighting changes, shadows, and noise, and is prone to false positives and false negatives. Therefore, this image processing method has low accuracy. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image processing method, apparatus, device and medium, which can improve the accuracy of motion state detection.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0005] Performing double exposure on the target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure time of the long-exposure image frame is a first time length, and the exposure time of the short-exposure image frame is a second time length, and the second time length is shorter than the first time length;

[0006] Determining difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame;

[0007] The motion state of the target scene is determined according to the difference information.

[0008] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0009] an exposure module, configured to perform double exposure on a target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure duration of the long-exposure image frame is a first duration, and the exposure duration of the short-exposure image frame is a second duration, and the second duration is shorter than the first duration;

[0010] A first determining module, configured to determine difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame;

[0011] The second determining module is used to determine the motion state of the target scene according to the difference information.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method provided in the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method provided in the first aspect are implemented.

[0014] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method provided in the first aspect.

[0015] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method provided in the first aspect.

[0016] In the image processing method, device, equipment and medium of the present application, the same target scene can be double-exposed to obtain long-exposure image frames and short-exposure image frames respectively, and the difference information between the clarity of the long-exposure image frame and the clarity of the short-exposure image frame can be determined to detect the motion state of the target scene. Since the clarity of the long-exposure image frame and the short-exposure image frame obtained by double exposure is significantly different in the motion area, but the difference is smaller in the static area, the quantitative analysis of this difference can effectively distinguish the motion area and the static area in the target scene, thereby detecting the motion state. In addition, this method does not rely on pixel changes in adjacent frames or complex optical flow calculations, and is therefore not affected by factors such as lighting changes, shadows, and noise, avoiding false alarms and missed alarms caused by background dynamics or rapid motion in traditional methods, thereby significantly improving the accuracy of motion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of a long exposure image frame in an image processing method provided by an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a short exposure image frame in an image processing method provided by an embodiment of the present application;

[0020] Figure 4 is a structural schematic diagram of an image processing device provided by another embodiment of the present application;

[0021] Figure 5 is a structural schematic diagram of an electronic device provided by another embodiment of the present application;

[0022] Figure 6 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0025] In order to solve the above technical problems, the present application provides an image processing method. The image processing method provided by the embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0026] like Figure 1 As shown, Figure 1 : is a flowchart of an image processing method provided by an embodiment of the present application. The embodiment of the present application provides an image processing method, which may include:

[0027] S101, performing double exposure on a target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure time of the long-exposure image frame is a first time length, and the exposure time of the short-exposure image frame is a second time length, and the second time length is shorter than the first time length;

[0028] In this embodiment, the same target scene can be photographed by the sensor using two different exposure times to obtain a long exposure image frame and a short exposure image frame. The exposure time of the long exposure image frame is a longer first time, so it can capture more light information and record more details, but it is easy to produce blur in the motion area. The exposure time of the short exposure image frame is a shorter second time, which can reduce motion blur, but may lose some details.

[0029] For example, the first duration may be a conventional exposure duration for shooting preview, and the second duration may be reduced N times based on the first duration according to the motion of the target scene. At the same time, in order to ensure the consistency of brightness between the long-exposure image frame and the short-exposure image frame, when shooting the short-exposure image frame, the sensitivity may be reduced N times based on the sensitivity of shooting the long-exposure image frame, and N may be 4.

[0030] Specifically, after setting the first time length and the second time length, the first time length and the second time length can be sent to a stagger sensor, and the stagger sensor can acquire long exposure image frames and short exposure image frames by time domain acquisition or spatial domain acquisition.

[0031] Taking the spatial domain acquisition method as an example, the interleaved sensor can realize the simultaneous acquisition of image data with different exposure information by alternately setting different exposure times for pixel areas in the same image frame. For example, the first pixel area is a long-exposure pixel, and its exposure time is set to the first time; the second pixel area is a short-exposure pixel, and its exposure time is set to the second time, alternating in sequence. Then, the long-exposure pixels and short-exposure pixels are extracted respectively. Since the alternating long-exposure pixels and short-exposure pixels are output as incomplete images, the missing pixel values ​​can be supplemented through an interpolation algorithm to generate complete long-exposure image frames and short-exposure image frames.

[0032] For example, Figure 2 is a long exposure image frame, Figure 3 is a short exposure image frame. In the target scene, a puppy runs in front of a house. Therefore, the house in the target scene is stationary, while the puppy is moving. Figure 2 The moving dog in the long exposure image frame has motion blur, while the house is clear. Figure 3 In the short-exposure image frame, both the house and the puppy are clear.

[0033] S102, determining difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame;

[0034] In this embodiment, the difference information can be obtained by calculating the clarity of the long exposure image frame and the short exposure image frame, and comparing the clarity difference between the long exposure image frame and the short exposure image frame. Specifically, in a static area, the clarity difference between the long exposure image frame and the short exposure image frame is small, because there is no motion blur in a static scene. In a moving area, the long exposure image frame will have obvious motion blur due to the long exposure time, while the short exposure image frame can better maintain clarity, so the clarity difference is significant. Therefore, the difference information can reflect whether the target scene is static or moving: the greater the difference in clarity, the more likely it is a moving area; the smaller the difference, the static area.

[0035] Specifically, in some embodiments, determining the difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame includes:

[0036] Segmenting the long-exposure image frame and the short-exposure image frame into a plurality of image blocks respectively in a target manner;

[0037] Determine difference information between sharpness of a plurality of groups of image blocks, wherein a group of the image blocks includes a pair of image blocks at corresponding positions on the long-exposure image frame and the short-exposure image frame.

[0038] In this embodiment, the long-exposure image frame and the short-exposure image frame can be divided into a plurality of image blocks according to a specific segmentation method, wherein the image blocks at the same position in the long-exposure image frame and the short-exposure image frame are paired as image blocks. Then, the clarity of each group of image blocks is compared to generate a plurality of difference information, each of which can characterize the motion state of the region where the corresponding image block is located: if the clarity of the image block of the short-exposure image frame in the image block is significantly higher than that of the image block of the long-exposure image frame, it indicates that there is motion in the region where the image block is located; and if the clarity of the two is similar, it indicates that the region is a static region.

[0039] By dividing the long and short exposure image frames into blocks and comparing the clarity block by block, the scheme can accurately detect the motion state of each area in the scene at a fine granularity.

[0040] In some embodiments, the comparison of the sharpness of the multiple groups of image blocks to obtain multiple difference information includes:

[0041] For any first image block in the plurality of groups of image blocks, determine a first clarity score of the first image block in the first group of image blocks, and a second clarity score of a second image block in the first group of image blocks, wherein the first image block is any image block in the short-exposure image frame, and the second image block is an image block having a position in the long-exposure image frame corresponding to a position of the first image block in the short-exposure image frame;

[0042] A ratio of the first clarity score to the second clarity score is determined as difference information corresponding to the first group of image blocks.

[0043] In this embodiment, the first image block and the second image block together constitute a first group of image blocks. The first image block is any image block in the short exposure image frame, and the second image block is an image block in the long exposure image frame corresponding to the position of the first image block in the short exposure image frame. The first clarity score of the first image block and the second clarity score of the second image block can be calculated respectively, and then the ratio of the two is used as the difference information corresponding to the first group of image blocks. The ratio reflects the difference between the effect of short exposure in reducing motion blur and the effect of long exposure in retaining details, so the motion state of the area can be quantified.

[0044] For example, Figure 2 is a long exposure image frame, Figure 3 is a short exposure image frame. Figure 3 Select the first image block 301 in Figure 2 The second image block 201 is determined at the corresponding position in the long exposure image frame shown, then the second image block 201 and the first image block 301 together form a first group of image blocks. A first clarity score of the first image block 301 and a second clarity score of the second image block 201 can be determined respectively, and then the ratio of the first clarity score to the second clarity score is determined as the difference information corresponding to the first group of image blocks.

[0045] In the above manner, the difference between the moving area and the static area can be accurately quantified by calculating the clarity ratio of each group of image blocks.

[0046] In some embodiments, determining a first clarity score for a first image block in the first group of image blocks comprises:

[0047] Performing convolution calculation on the first image block by using a Laplace operator to obtain a Laplace value of each pixel in the first image block, where the Laplace value is used to characterize the richness of details in the corresponding pixel;

[0048] Performing weighted average calculation on the Laplace values ​​corresponding to all pixels in the first image block to obtain a weighted average value;

[0049] The weighted average is determined as the first clarity score.

[0050] In this embodiment, the clarity score of each image block can be calculated by using the Laplace operator, which is a second-order differential operator used to measure the rate of change of pixel values ​​in an image block and detect the edge and details of the image block by detecting the grayscale change of the pixel value. The richer the edge and details, the higher the clarity of the image block.

[0051] In this embodiment, taking the first image block as an example, the Laplacian operator may be a 3×3 convolution kernel as shown below:

[0052]

[0053] The above-mentioned convolution kernel can be used to perform a convolution operation on each pixel in the first image block to calculate the Laplacian value of each pixel. The Laplacian value represents the degree of change of each pixel in the first image block, thereby reflecting the richness of the details and texture of the pixel.

[0054] Specifically, the formula for the convolution operation can be shown as follows:

[0055]

[0056] Where f(x,y) is the pixel value at the image (x,y) coordinate, Δf(x,y) is the Laplacian value at the image (x,y) coordinate calculated by the Laplacian operator, and h(i,j) is the convolution kernel.

[0057] After obtaining the Laplace value of each pixel point through the Laplace operator, the Laplace values ​​of all pixels in the first image block can be weighted averaged to obtain a weighted average value, and the weighted average value is determined as the first clarity score corresponding to the first pixel block. The formula for weighted average calculation is:

[0058]

[0059] Wherein, S represents the first clarity score, N represents the number of pixels in the first image block, n represents the nth pixel, and f(n) represents the Laplace value of the nth pixel.

[0060] In this way, the detail information of each pixel in the first image block can be extracted by the Laplacian operator, and the clarity of the entire first image block can be quantified by weighted average calculation, so that the first clarity score can accurately represent the clarity of the first image block.

[0061] S103: Determine the motion state of the target scene according to the difference information.

[0062] In the embodiment of the present application, as described above, the difference in clarity between the long-exposure image frame and the short-exposure image frame in the static area is small, while in the moving area, the difference in clarity between the long-exposure image frame and the short-exposure image frame is large. Therefore, the difference in clarity of all image blocks in the long-exposure image frame and the short-exposure image frame can be comprehensively analyzed to determine the motion state of the entire target scene.

[0063] In the present application, the same target scene can be double-exposed to obtain long-exposure image frames and short-exposure image frames respectively, and the difference information between the clarity of the long-exposure image frames and the clarity of the short-exposure image frames can be determined to detect the motion state of the target scene. Since the clarity of the long-exposure image frames and the short-exposure image frames obtained by double exposure is significantly different in the motion area, but the difference is smaller in the static area, the quantitative analysis of this difference can effectively distinguish the motion area and the static area in the target scene, thereby detecting the motion state. In addition, this method does not rely on pixel changes in adjacent frames or complex optical flow calculations, so it is not affected by factors such as lighting changes, shadows, and noise, and avoids false alarms and missed alarms caused by background dynamics or rapid motion in traditional methods, thereby significantly improving the accuracy of motion detection.

[0064] In some embodiments, determining the motion state of the target scene according to the difference information includes:

[0065] When the difference information among the plurality of difference information is greater than a first threshold, determining a region represented by an image block corresponding to the difference information as a motion region;

[0066] The motion state of the target scene is determined according to the number of motion areas in the target scene.

[0067] In this embodiment, since the difference information is the ratio of the clarity score of the first image block to the clarity score of the second image block, the ratio reflects the difference between the effect of short exposure on reducing motion blur and the effect of long exposure on retaining details. The larger the ratio, the greater the difference in details between the short exposure and the long exposure. The blurrier the long exposure image frame is, the more likely the area represented by the image block is a motion area.

[0068] Specifically, since the difference information is the ratio of the clarity score of the first image block to the clarity score of the second image block, the larger the ratio is, the greater the difference in details between the short exposure and the long exposure is. Therefore, a first threshold can be set as a critical threshold of the motion state, and each difference information is compared with the first threshold. If the difference information is greater than the first threshold, the area represented by the image block corresponding to the difference information is determined as a motion area. If the difference information is less than or equal to the first threshold, the area represented by the image block corresponding to the difference information can be determined as a static area. The first threshold can be set to 3.

[0069] Then, the motion state of the target scene can be determined based on the number of motion areas in the target scene. If there is no motion area, the target scene is considered to be in a non-motion state. If the number of motion areas in the target scene is less than a second threshold, the target scene can be determined to be in a light motion state, and if the number of motion areas in the target scene is greater than or equal to the second threshold, the target scene can be determined to be in a fast motion state.

[0070] In this way, the number of motion areas represented by each image block can be determined based on the difference information of each image block, and then the motion state of the entire target scene can be determined based on the number of motion areas in the target scene, thereby achieving a quantitative evaluation of the motion state of the target scene.

[0071] In some embodiments, after determining the ratio of the first clarity score to the second clarity score as the difference information corresponding to the first group of image blocks, the method further includes:

[0072] When the difference information among the plurality of difference information is greater than a first threshold, determining the image block corresponding to the difference information as a motion image block;

[0073] When the difference information among the plurality of difference information is less than or equal to the first threshold, determining the image block corresponding to the difference information as a still image block;

[0074] Determining a first weight value of the motion image block in the long-exposure image frame, a second weight value of the motion image block in the short-exposure image frame, a third weight value of the still image block in the long-exposure image frame, and a fourth weight value of the still image block in the short-exposure image frame, wherein the first weight value is less than the third weight value, and the second weight value is greater than the fourth weight value;

[0075] The motion image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the first weight value and the second weight value, and the still image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the third weight value and the fourth weight value.

[0076] In this embodiment, since each difference information indicates the motion of the area represented by its corresponding image block, the weight values ​​of the long exposure image frame and the short exposure image frame in the motion area and the static area during the image fusion process can be determined based on the difference information.

[0077] If a certain difference information is greater than the first threshold, the area represented by the image block corresponding to the difference information is determined as a moving area. When the difference information is less than or equal to the first threshold, the area represented by the image block corresponding to the difference information can be determined as a static area. Long-exposure image frames can retain more details and dark information in static areas, so increasing their weight in static image blocks can improve image quality. In moving image blocks, long exposure is prone to motion blur, so it is necessary to reduce the weight of moving image blocks and increase the weight of short-exposure image frames.

[0078] Therefore, the weight of the moving image block in the long exposure image frame can be set to a lower first weight value, and the weight of the still image block in the long exposure image frame can be set to a higher third weight value. Correspondingly, the weight of the moving image block in the short exposure image frame can be set to a higher second weight value, and the weight of the still image block in the short exposure image frame can be set to a lower fourth weight value.

[0079] After determining the first weight value, the second weight value, the third weight value, and the fourth weight value, the motion image blocks in the long exposure image frame and the short exposure image frame may be fused based on the first weight value and the second weight value, and the still image blocks in the long exposure image frame and the short exposure image frame may be fused based on the third weight value and the fourth weight value to obtain a target image. For example, the formula of the motion image block may be as follows:

[0080] I=(1-Weight 长曝 )×I 短曝光帧 +Weight 长曝 ×I 长曝光帧

[0081] Among them, Weight 长曝 Represents the first weight value of the long exposure image frame, (1-Weight 长曝 ) represents the second weight value of the short exposure image frame, I 长曝光帧 represents the motion image block in the long exposure image frame, I 短曝光帧 represents the motion image block in the short exposure image frame, and I represents the final target image.

[0082] In this way, the weight distribution of different areas in the long-exposure image frames and the short-exposure image frames can be dynamically adjusted according to the difference in clarity, so that the fused target image can adapt to the motion state in the target scene, thereby improving the quality and visual effect of the target image.

[0083] In some embodiments, after determining the ratio of the first clarity score to the second clarity score as the difference information corresponding to the first image block, the method further includes:

[0084] When difference information among the multiple difference information is greater than a first threshold and less than or equal to a third threshold, the difference information is determined as first difference information, wherein an area represented by an image block corresponding to the first difference information is determined as a motion area, and motion blur exists on the image blocks corresponding to the first difference information;

[0085] When the amount of first difference information in the plurality of difference information is greater than a fourth threshold, the exposure time of the short-exposure image frame is reduced.

[0086] In this embodiment, since each piece of difference information represents the ratio of the clarity of the short-exposure image frame to the clarity of the long-exposure image frame in a group of image blocks, classification can be performed based on the interval in which the ratio lies.

[0087] Specifically, if the ratio is less than or equal to the first threshold, it can be considered that the area represented by the image block corresponding to the difference information is determined to be a static area, and there is no motion blur on the image block corresponding to the difference information; if the ratio is greater than the first threshold and less than the third threshold, it can be considered that the area represented by the image block corresponding to the difference information is determined to be a moving area, and there is motion blur on the image blocks on both the short-exposure image frame and the long-exposure image frame; if the ratio is greater than the third threshold, it can be considered that the area represented by the image block corresponding to the difference information is determined to be a moving area, and there is no motion blur on the image block on the short-exposure image frame, and there is motion blur only on the image block on the long-exposure image frame. Therefore, the first difference information means that the area represented by the corresponding image block is determined to be a moving area, and there is motion blur on the image blocks corresponding to the first difference information.

[0088] For example, the first threshold value may be set to 3, and the third threshold value may be set to 5. The following formula may be listed:

[0089]

[0090] Among them, S 短曝 Indicates the clarity score of an image block in a short exposure image frame, S 长曝 Indicates the clarity score of the image block in the long exposure image frame in the image block.

[0091] If the number of first difference information in multiple difference information exceeds the fourth threshold, it means that the current short exposure time is not enough to completely reduce motion blur. Therefore, the exposure duration of the short exposure can be reduced, that is, the second duration can be reduced to further reduce motion blur to adapt to the current fast-motion scene.

[0092] In the above method, the degree of motion blur can be determined by analyzing the difference information. If the motion blur in the short exposure image frame is more serious, the length of the short exposure can be dynamically adjusted to optimize the motion blur problem and ensure the image quality in the subsequent shooting process.

[0093] Figure 4 is a structural diagram of an image processing device provided by another embodiment of the present application. Figure 4 As shown, the image processing device may include:

[0094] An exposure module 401 is used to perform double exposure on a target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure time of the long-exposure image frame is a first time length, and the exposure time of the short-exposure image frame is a second time length, and the second time length is shorter than the first time length;

[0095] A first determination module 402, configured to determine difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame;

[0096] The second determining module 403 is configured to determine the motion state of the target scene according to the difference information.

[0097] In the present application, the same target scene can be double-exposed to obtain long-exposure image frames and short-exposure image frames respectively, and the difference information between the clarity of the long-exposure image frames and the clarity of the short-exposure image frames can be determined to detect the motion state of the target scene. Since the clarity of the long-exposure image frames and the short-exposure image frames obtained by double exposure is significantly different in the motion area, but the difference is smaller in the static area, the quantitative analysis of this difference can effectively distinguish the motion area and the static area in the target scene, thereby detecting the motion state. In addition, this method does not rely on pixel changes in adjacent frames or complex optical flow calculations, so it is not affected by factors such as lighting changes, shadows, and noise, and avoids false alarms and missed alarms caused by background dynamics or rapid motion in traditional methods, thereby significantly improving the accuracy of motion detection.

[0098] In another optional example, the first determining module 402 includes:

[0099] a segmentation unit, configured to segment the long-exposure image frame and the short-exposure image frame into a plurality of image blocks;

[0100] The first determining unit is used to determine the difference information between the sharpness of multiple groups of image blocks, wherein a group of the image blocks includes a pair of image blocks at corresponding positions on the long exposure image frame and the short exposure image frame.

[0101] In another optional example, the first determining unit includes:

[0102] a first determining subunit, configured to determine, for any first group of image blocks in the plurality of groups of image blocks, a first clarity score of a first image block in the first group of image blocks, and a second clarity score of a second image block in the first group of image blocks, wherein the first image block is any image block in the short-exposure image frame, and the second image block is an image block having a position in the long-exposure image frame corresponding to a position of the first image block in the short-exposure image frame;

[0103] The second determining subunit is configured to determine a ratio of the first clarity score to the second clarity score as difference information corresponding to the first group of image blocks.

[0104] In another optional example, the first determining subunit is specifically configured to:

[0105] Performing convolution calculation on each pixel in the first image block by using a Laplace operator to obtain a Laplace value of each pixel in the first image block, where the Laplace value is used to characterize the richness of details in the corresponding pixel;

[0106] Performing weighted average calculation on the Laplace values ​​corresponding to all pixels in the first image block to obtain a weighted average value;

[0107] The weighted average is determined as the first clarity score.

[0108] In another optional example, the second determining module 403 includes:

[0109] a second determining unit, configured to determine, when difference information among the plurality of difference information is greater than a first threshold, a region represented by an image block corresponding to the difference information as a motion region;

[0110] The third determining unit is configured to determine the motion state of the target scene according to the number of motion areas in the target scene.

[0111] In another optional example, the device includes:

[0112] A third determining module is configured to determine the difference information as first difference information when the difference information among the plurality of difference information is greater than the first threshold and less than or equal to the third threshold, wherein the area represented by the image block corresponding to the first difference information is determined as a motion area, and motion blur exists on the image blocks corresponding to the first difference information;

[0113] The reducing module is configured to reduce the exposure time of the short-exposure image frame when the amount of first difference information in the plurality of difference information is greater than a fourth threshold.

[0114] In another optional example, the device includes:

[0115] In another optional example, the device includes:

[0116] A fourth determination module, configured to determine, when difference information among the plurality of difference information is greater than a first threshold, an image block corresponding to the difference information as a motion image block;

[0117] A fifth determining module, configured to determine, when the difference information among the plurality of difference information is less than or equal to a first threshold, an image block corresponding to the difference information as a still image block;

[0118] a sixth determination module, configured to determine a first weight value of the motion image block in the long-exposure image frame, a second weight value of the motion image block in the short-exposure image frame, a third weight value of the still image block in the long-exposure image frame, and a fourth weight value of the still image block in the short-exposure image frame, wherein the first weight value is less than the third weight value, and the second weight value is greater than the fourth weight value;

[0119] a fusion module, configured to fuse the motion image blocks in the long-exposure image frame and the short-exposure image frame based on the first weight value and the second weight value, and to fuse the still image blocks in the long-exposure image frame and the short-exposure image frame based on the third weight value and the fourth weight value.

[0120] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0121] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0122] The image processing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0123] Alternatively, if Figure 5 As shown, an embodiment of the present application also provides an electronic device 100, including a processor 110, a memory 119, and a program or instruction stored in the memory 119 and executable on the processor 110. When the program or instruction is executed by the processor 110, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0124] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0125] Please refer to Figure 6 , Figure 6 The hardware structure diagram of an electronic device for implementing an embodiment of the present application is shown in FIG. The electronic device 100 includes, but is not limited to, a radio frequency unit 121, a network module 122, an audio output unit 123, an input unit 124, a sensor 125, a display unit 126, a user input unit 127, an interface unit 128, a memory 129, and a processor 120.

[0126] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 120 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0127] The processor 120 is configured to perform a double exposure on the target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure time of the long-exposure image frame is a first time length, and the exposure time of the short-exposure image frame is a second time length, and the second time length is less than the first time length;

[0128] The processor 120 is configured to determine difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame;

[0129] The processor 120 is configured to determine the motion state of the target scene according to the difference information.

[0130] In the present application, the same target scene can be double-exposed to obtain long-exposure image frames and short-exposure image frames respectively, and the difference information between the clarity of the long-exposure image frames and the clarity of the short-exposure image frames can be determined to detect the motion state of the target scene. Since the clarity of the long-exposure image frames and the short-exposure image frames obtained by double exposure is significantly different in the motion area, but the difference is smaller in the static area, the quantitative analysis of this difference can effectively distinguish the motion area and the static area in the target scene, thereby detecting the motion state. In addition, this method does not rely on pixel changes in adjacent frames or complex optical flow calculations, so it is not affected by factors such as lighting changes, shadows, and noise, and avoids false alarms and missed alarms caused by background dynamics or rapid motion in traditional methods, thereby significantly improving the accuracy of motion detection.

[0131] In another optional example, the processor 120 is further configured to:

[0132] dividing the long-exposure image frame and the short-exposure image frame into a plurality of image blocks;

[0133] Determine difference information between sharpness of a plurality of groups of image blocks, wherein a group of the image blocks includes a pair of image blocks at corresponding positions on the long-exposure image frame and the short-exposure image frame.

[0134] In another optional example, the processor 120 is further configured to:

[0135] For any first group of image blocks in the multiple groups of image blocks, determine a first clarity score of a first image block in the first group of image blocks, and a second clarity score of a second image block in the first group of image blocks, wherein the first image block is any image block in the short exposure image frame, and the second image block is an image block having a position in the long exposure image frame corresponding to a position of the first image block in the short exposure image frame;

[0136] A ratio of the first clarity score to the second clarity score is determined as difference information corresponding to the first group of image blocks.

[0137] In another optional example, the processor 120 is further configured to:

[0138] Performing convolution calculation on each pixel in the first image block by using a Laplace operator to obtain a Laplace value of each pixel in the first image block, where the Laplace value is used to characterize the richness of details in the corresponding pixel;

[0139] Performing weighted average calculation on the Laplace values ​​corresponding to all pixels in the first image block to obtain a weighted average value;

[0140] The weighted average is determined as the first clarity score.

[0141] In another optional example, the processor 120 is further configured to:

[0142] When the difference information among the plurality of difference information is greater than a first threshold, determining a region represented by an image block corresponding to the difference information as a motion region;

[0143] The motion state of the target scene is determined according to the number of motion areas in the target scene.

[0144] In another optional example, the processor 120 is further configured to:

[0145] When difference information among the multiple difference information is greater than a first threshold and less than or equal to a third threshold, the difference information is determined as first difference information, wherein an area represented by an image block corresponding to the first difference information is determined as a motion area, and motion blur exists on the image blocks corresponding to the first difference information;

[0146] When the amount of first difference information in the plurality of difference information is greater than a fourth threshold, the exposure time of the short-exposure image frame is reduced.

[0147] In another optional example, the processor 120 is further configured to:

[0148] When the difference information among the plurality of difference information is greater than a first threshold, determining the image block corresponding to the difference information as a motion image block;

[0149] When the difference information among the plurality of difference information is less than or equal to the first threshold, determining the image block corresponding to the difference information as a still image block;

[0150] Determining a first weight value of the motion image block in the long-exposure image frame, a second weight value of the motion image block in the short-exposure image frame, a third weight value of the still image block in the long-exposure image frame, and a fourth weight value of the still image block in the short-exposure image frame, wherein the first weight value is less than the third weight value, and the second weight value is greater than the fourth weight value;

[0151] The motion image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the first weight value and the second weight value, and the still image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the third weight value and the fourth weight value.

[0152] It should be understood that in the embodiment of the present application, the input unit 124 may include a graphics processor (Graphics Processing Unit, GPU) 1241 and a microphone 1242, and the graphics processor 1241 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 126 may include a display panel 1261, and the display panel 1261 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 127 includes a touch panel 1271 and at least one of other input devices 1272. The touch panel 1271 is also called a touch screen. The touch panel 1271 may include two parts: a touch detection device and a touch controller. Other input devices 1272 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0153] The memory 129 can be used to store software programs and various data. The memory 129 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 129 may include a volatile memory or a non-volatile memory, or the memory 129 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 129 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0154] The processor 120 may include one or more processing units; optionally, the processor 120 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 120.

[0155] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0156] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0157] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0158] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0159] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0160] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0162] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: Performing double exposure on the target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure time of the long-exposure image frame is a first time length, and the exposure time of the short-exposure image frame is a second time length, and the second time length is shorter than the first time length; Determining difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame; The motion state of the target scene is determined according to the difference information.

2. The method according to claim 1, characterized in that The determining of difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame includes: dividing the long-exposure image frame and the short-exposure image frame into a plurality of image blocks; Determine difference information between sharpness of a plurality of groups of image blocks, wherein a group of the image blocks includes a pair of image blocks at corresponding positions on the long-exposure image frame and the short-exposure image frame.

3. The method according to claim 2, characterized in that The determining of the difference information between the sharpness of the plurality of groups of image blocks comprises: For any first group of image blocks in the multiple groups of image blocks, determine a first clarity score of a first image block in the first group of image blocks, and a second clarity score of a second image block in the first group of image blocks, wherein the first image block is any image block in the short exposure image frame, and the second image block is an image block having a position in the long exposure image frame corresponding to a position of the first image block in the short exposure image frame; A ratio of the first clarity score to the second clarity score is determined as difference information corresponding to the first group of image blocks.

4. The method according to claim 3, characterized in that: The determining a first clarity score of a first image block in the first group of image blocks comprises: Performing convolution calculation on each pixel in the first image block by using a Laplace operator to obtain a Laplace value of each pixel in the first image block, where the Laplace value is used to characterize the richness of details in the corresponding pixel; Performing weighted average calculation on the Laplace values ​​corresponding to all pixels in the first image block to obtain a weighted average value; The weighted average is determined as the first clarity score.

5. The method according to claim 3, characterized in that: The determining the motion state of the target scene according to the difference information includes: When the difference information among the plurality of difference information is greater than a first threshold, determining a region represented by an image block corresponding to the difference information as a motion region; The motion state of the target scene is determined according to the number of motion areas in the target scene.

6. The method according to claim 3, characterized in that: After determining the ratio of the first clarity score to the second clarity score as the difference information corresponding to the first group of image blocks, the method further includes: When difference information among the multiple difference information is greater than a first threshold and less than or equal to a third threshold, the difference information is determined as first difference information, wherein an area represented by an image block corresponding to the first difference information is determined as a motion area, and motion blur exists on the image blocks corresponding to the first difference information; When the amount of first difference information in the plurality of difference information is greater than a fourth threshold, the exposure time of the short-exposure image frame is reduced.

7. The method according to claim 3, characterized in that After determining the difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame, the method further includes: When the difference information among the plurality of difference information is greater than a first threshold, determining the image block corresponding to the difference information as a motion image block; When the difference information among the plurality of difference information is less than or equal to the first threshold, determining the image block corresponding to the difference information as a still image block; Determining a first weight value of the motion image block in the long-exposure image frame, a second weight value of the motion image block in the short-exposure image frame, a third weight value of the still image block in the long-exposure image frame, and a fourth weight value of the still image block in the short-exposure image frame, wherein the first weight value is less than the third weight value, and the second weight value is greater than the fourth weight value; The motion image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the first weight value and the second weight value, and the still image blocks in the long-exposure image frame and the short-exposure image frame are fused based on the third weight value and the fourth weight value.

8. An image processing device, characterized in that: include: an exposure module, configured to perform double exposure on a target scene to obtain a long-exposure image frame and a short-exposure image frame, wherein the exposure duration of the long-exposure image frame is a first duration, and the exposure duration of the short-exposure image frame is a second duration, and the second duration is shorter than the first duration; A first determining module, configured to determine difference information between the definition of the long-exposure image frame and the definition of the short-exposure image frame; The second determining module is used to determine the motion state of the target scene according to the difference information.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images