Depth map determination method and device and electronic equipment

Abstract

The invention discloses a depth map determination method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the steps that a first exposure image and a second exposure image of a first camera and a third exposure image of a second camera are acquired, the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same; target detection is carried out on the first exposure image, N image areas of the first exposure image are determined, target detection comprises at least one of brightness detection and motion detection, and N is a positive integer; according to the first exposure image and the second exposure image, fusing the N image areas to obtain a fused image; and determining a depth map according to the fused image and the third exposure image.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a depth map determination method, device and electronic device. Background Art

[0002] With the continuous development of mobile terminal technology, more and more users use mobile terminals with dual camera functions to take photos, and can perform background blurring and other processing based on the depth map of the captured image, thereby meeting the diverse shooting needs of users. At present, the shooting scenes of images are complex and diverse. For example, in high-speed motion scenes, the subject or background in the captured image may be motion blurred, resulting in poor accuracy of the depth map. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and electronic device for determining a depth map, which can solve the problem of poor accuracy of the depth map.

[0004] In a first aspect, an embodiment of the present application provides a method for determining a depth map, the method comprising:

[0005] Acquire a first exposure image and a second exposure image of the first camera, and a third exposure image of the second camera, wherein the first exposure image and the second exposure image have different exposure parameters, and the second exposure image and the third exposure image have the same exposure parameters;

[0006] Performing target detection on the first exposure image to determine N image regions of the first exposure image, where the target detection includes at least one of brightness detection and motion detection, and N is a positive integer;

[0007] According to the first exposure image and the second exposure image, N image regions are fused to obtain a fused image;

[0008] A depth map is determined based on the fused image and the third exposure image.

[0009] In a second aspect, an embodiment of the present application provides a depth map determination device, the device comprising:

[0010] an acquisition module, configured to acquire a first exposure image and a second exposure image of the first camera, and a third exposure image of the second camera, wherein the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same;

[0011] A first determination module is used to perform target detection on the first exposure image to determine N image areas of the first exposure image, where the target detection includes at least one of brightness detection and motion detection, and N is a positive integer;

[0012] A fusion module, used for fusing N image regions according to the first exposure image and the second exposure image to obtain a fused image;

[0013] The second determination module is used to determine a depth map according to the fused image and the third exposure image.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the first aspect.

[0018] In the embodiment of the present application, the first exposure image and the second exposure image of the first camera with different exposure parameters can be obtained, and based on the exposure parameters of the second exposure image, the third exposure image of the second camera can be obtained; the first exposure image is subjected to brightness detection and / or motion detection to determine N image regions of the first exposure image; the N image regions are fused according to the first exposure image and the second exposure image to obtain a fused image; a depth map is determined according to the fused image and the third exposure image. In this way, the detection results of the brightness detection and / or motion detection can be used as a reference for the fusion ratio of the image content corresponding to different exposure parameters in each image region, so as to perform fusion processing and obtain a fused image with better dynamic effect and clarity, and then a depth map with higher accuracy can be obtained based on the fused image and the third exposure image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flowchart of a depth map determination method provided in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a specific scenario embodiment of the depth map determination method provided in an embodiment of the present application;

[0021] Figure 3 is a structural schematic diagram of a depth map determination device provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0026] The depth map determination method provided in the embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0027] Figure 1 is a flow chart of a depth map determination method provided by an embodiment of the present application. The depth map determination method may include:

[0028] Step 101: Acquire a first exposure image and a second exposure image of a first camera, and a third exposure image of a second camera, wherein the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same.

[0029] In step 101, it can be understood that the electronic device may be provided with a dual camera module, one of which is a main camera, the image collected by which is used to display a preview screen on the electronic device, and the other is a secondary camera, the image collected by which is used to provide depth information. In the embodiment of the present application, the first camera can be used as the main camera and the second camera can be used as the secondary camera.

[0030] The first camera can be configured in a digital overlap (DOL) mode, in which multiple frames of images with different exposure parameters can be output simultaneously. Figure 2 As shown, the first camera can simultaneously output two image frames with different exposure parameters, such as a long exposure frame and a short burst frame. The long exposure frame and the short burst frame corresponding to the photo shooting moment can be determined as the first exposure image 201 and the second exposure image 202 of the first camera.

[0031] Based on the frame synchronization between the first camera and the second camera, the short exposure frame of the first camera can be used as a reference frame, and the image frame of the second camera at the same time can be matched by the timestamp, and the image frame of the second camera can synchronize the exposure parameters of the short exposure frame. In other words, the third exposure image 203 of the second camera can be obtained based on the timestamp and exposure parameters of the second exposure image.

[0032] Step 102: Perform target detection on the first exposure image to determine N image areas of the first exposure image. The target detection includes at least one of brightness detection and motion detection, and N is a positive integer.

[0033] In step 102, target detection may be performed on the first exposure image to obtain detection information, and the first exposure image may be divided into N image regions according to the detection information, where different image regions have different detection information.

[0034] Target detection may include brightness detection and / or motion detection. In other words, the detection information may include the detected brightness information and / or motion information. The brightness information may be brightness levels corresponding to different ranges of brightness values, such as "long exposure without overexposure", "brightness transition zone" and "long exposure with overexposure". The motion information may characterize the degree of motion of the subject, such as "still", "mild motion", "moderate motion" and "heavy motion".

[0035] In some examples, if the target detection is brightness detection, the first exposure image can be divided into different image areas according to different brightness levels. In other examples, if the target detection is motion detection, the first exposure image can be divided into different image areas according to different degrees of motion of the subject.

[0036] In some embodiments, when target detection includes brightness detection and motion detection, each image region includes brightness information and motion information, and at least one of the brightness information and the motion information is different between any two image regions.

[0037] For example, if target detection includes brightness detection and motion detection, the division of the image area can be determined by combining brightness information and motion information. In other words, different image areas may have the same brightness information but different motion information, the same motion information but different brightness information, or both the motion information and the brightness information may be different.

[0038] In this way, different image areas can be obtained by considering brightness information and motion information together, making the division of image areas more detailed, so that different image areas can be fused more finely to obtain a more accurate fused image, thereby improving the accuracy of the depth map.

[0039] Step 103: According to the first exposure image and the second exposure image, N image regions are fused to obtain a fused image.

[0040] In step 103, it can be understood that the image contents of the first exposure image and the second exposure image are consistent, and therefore, the image content in each image area in the first exposure image can correspond one-to-one to the image content in each image area in the second exposure image.

[0041] Based on this, for any image area among the N image areas, the image content of the image area in the first exposure image can be selected as the image content of the image area in the fused image. The image content of the image area in the second exposure image can also be selected as the image content of the image area in the fused image. It is also possible to fit the image content of the image area in the first exposure image and the image content of the image area in the second exposure image according to a preset fusion ratio, and use the fitted image content as the image content of the image area in the fused image. The specific selection can be determined based on the detection information in each image area, and is not specifically limited here.

[0042] like Figure 2 As shown, after traversing all image regions, the image contents of the N image regions in the fused image are spliced ​​to generate a fused image 204 .

[0043] Step 104: Determine a depth map according to the fused image and the third exposure image.

[0044] In step 104, if Figure 2 As shown, the fused image and the third exposure image may be calculated based on a dual-camera depth algorithm to obtain a depth map 205 .

[0045] For example, the fused image and the third exposure image are analyzed. Since the distances of the object from the first camera and the second camera are different, the position of the same object in the fused image and the third exposure image will be different. By calculating the difference in the position of the object in the fused image and the third exposure image, the parallax of the object can be obtained. The larger the parallax, the closer the object is to the camera; the smaller the parallax, the farther the object is from the camera. Based on the known relationship between the camera distance and the parallax, the actual distance of the object from the camera can be calculated.

[0046] By traversing all objects in the image, the depth map can be calculated.

[0047] In the embodiment of the present application, the first exposure image and the second exposure image of the first camera with different exposure parameters can be obtained, and based on the exposure parameters of the second exposure image, the third exposure image of the second camera can be obtained; the first exposure image is subjected to brightness detection and / or motion detection to determine N image regions of the first exposure image; the N image regions are fused according to the first exposure image and the second exposure image to obtain a fused image; a depth map is determined according to the fused image and the third exposure image. In this way, the detection results of the brightness detection and / or motion detection can be used as a reference for the fusion ratio of the image content corresponding to different exposure parameters in each image region, so as to perform fusion processing and obtain a fused image with better dynamic effect and clarity, and then a depth map with higher accuracy can be obtained based on the fused image and the third exposure image.

[0048] In some embodiments, fusing N image regions according to the first exposure image and the second exposure image to obtain a fused image may include:

[0049] Determine, according to a correspondence between preset brightness information and motion information and a fusion mode, a fusion mode corresponding to each of the N image regions, where the fusion mode includes using image content corresponding to the image region in the first exposure image, using image content corresponding to the image region in the second exposure image, or using image content after fitting the image regions in the first exposure image and the second exposure image;

[0050] According to the first exposure image and the second exposure image, the N image regions are fused in accordance with the fusion methods corresponding to the N image regions;

[0051] The N image regions after fusion processing are stitched together to generate a fused image.

[0052] In this embodiment, the correspondence between the preset brightness information and motion information and the fusion mode may be as shown in Table 1:

[0053] Table 1

[0054]

[0055]

[0056] Among them, brightness information can include "long exposure without overexposure", "brightness transition zone" and "long exposure with overexposure". Motion information can include "still", "mild motion", "moderate motion" and "heavy motion". Fusion methods can include "long exposure", "short burst", "long exposure priority", "short burst priority" and "fitting".

[0057] It can be understood that "long exposure priority" and "short exposure priority" may mean that when the clarity of the image content meets the conditions, "long exposure" or "short exposure" is determined as the fusion mode. "Long exposure" may refer to using the image content corresponding to the image area in the first exposure image as the image content of the image area in the fused image. "Short exposure" may refer to using the image content corresponding to the image area in the second exposure image as the image content of the image area in the fused image. "Fitting" may refer to using the image content after fitting the image area in the first exposure image and the second exposure image according to a preset fusion ratio as the image content of the image area in the fused image. Among them, the fusion ratios corresponding to different brightness information and motion information may be the same or different, and may be set according to actual needs, and are not specifically limited here.

[0058] According to the brightness information and motion information of each image area in the N image areas, a corresponding fusion mode can be matched from the preset correspondence between the brightness information and the motion information and the fusion mode. For example, when the brightness information is "long exposure and overexposure" or the motion information is "heavy motion", the fusion mode can be "short exposure". When the motion information is "mild motion" and the brightness information is "long exposure without overexposure" or "brightness transition zone", or when the motion information is "moderate motion" and the brightness information is "brightness transition zone", the fusion mode can be "fitting". When the motion information is "still" and the brightness information is "long exposure without overexposure", the fusion mode can be "long exposure". When the motion information is "still" and the brightness information is "brightness transition zone", the fusion mode can be "long exposure priority". When the motion information is "moderate motion" and the brightness information is "long exposure without overexposure", the fusion mode can be "short exposure priority".

[0059] According to the first exposure image and the second exposure image, the N image areas may be fused in accordance with the fusion mode corresponding to the N image areas. For example, according to the fusion mode, the image content of the image area in the first exposure image may be used as the image content of the image area in the fused image, or the image content of the image area in the second exposure image may be used as the image content of the image area in the fused image, or the image content of the image area in the first exposure image and the image content of the image area in the second exposure image may be fitted in accordance with a preset fusion ratio, and the fitted image content may be used as the image content of the image area in the fused image.

[0060] The N image regions after the fusion processing are stitched together to generate a fused image. For example, the fused image can be obtained by stitching together the image content of each image region in the fused image determined above.

[0061] In this way, each image area can be fused according to the fusion method corresponding to each image area and combined with the image content of the first exposure image and the second exposure image to obtain a fused image with better dynamic effect and clarity, and then a more accurate depth map can be obtained based on the fused image and the third exposure image.

[0062] In some embodiments, determining a depth map according to the fused image and the third exposure image may include:

[0063] performing exposure compensation on the third exposure image according to the fused image to obtain a fourth exposure image;

[0064] A depth map is determined according to the fused image and the fourth exposure image.

[0065] In this embodiment, the third exposure image may be subjected to exposure compensation according to the fused image to obtain a fourth exposure image. For example, the brightness values ​​of each pixel of the fused image and the third exposure image may be compared, and the exposure compensation value of each pixel in the third exposure image may be determined according to the comparison result, and the brightness value of each pixel in the third exposure image may be adjusted based on the exposure compensation value to obtain the fourth exposure image.

[0066] The fused image and the third exposure image may also be divided into regions to obtain at least one adjustment region, the average brightness value in each adjustment region is calculated, the average brightness value of a certain adjustment region in the fused image is compared with the average brightness value of the adjustment region in the third exposure image, the exposure compensation value of the adjustment region in the third exposure image is determined according to the comparison result, and the brightness value of the adjustment region in the third exposure image is adjusted based on the exposure compensation value, and after traversing all adjustment regions, a fourth exposure image is obtained.

[0067] The fused image and the fourth exposure image may be calculated based on a dual-camera depth algorithm to obtain a depth map.

[0068] In this way, exposure compensation can be performed on the third exposure image to obtain a fourth exposure image that is more consistent with the fused image in brightness and contrast, further improving the accuracy of the depth map.

[0069] In some embodiments, after determining the depth map according to the fused image and the third exposure image, the method may further include:

[0070] Determine, according to the depth map, a foreground area and a background area of ​​a first image, where the first image is an image synthesized based on the first exposure image and the second exposure image;

[0071] The background area is blurred to generate a blurred image.

[0072] In this embodiment, if Figure 2 As shown, the first exposure image and the second exposure image can be synthesized into a first image 206 captured by the first camera, and the background of the first image 206 is blurred based on the depth map 205 to obtain a blurred image 207.

[0073] The foreground area and background area of ​​the first image can be determined based on the depth map. Generally speaking, the depth map may include the distance between each object in the first image and the first camera. When the depth value is small, it means that the object is close to the first camera, and the object can be determined as the foreground. Otherwise, the object is the background. The distances of each object in the first image are traversed to determine the foreground area and background area of ​​the first image.

[0074] The background area can be blurred to generate a blurred image with a blurred background.

[0075] In this way, based on the depth map with higher accuracy, the foreground area and the background area can be identified more accurately, thereby obtaining a blurred image with more precise background blur and better effect.

[0076] The depth map determination method provided in the embodiment of the present application may be executed by a depth map determination device. In the embodiment of the present application, the depth map determination device performing the depth map determination method is taken as an example to illustrate the depth map determination device provided in the embodiment of the present application.

[0077] like Figure 3 As shown, the embodiment of the present application provides a depth map determining device 300, and the depth map determining device 300 may include:

[0078] An acquisition module 301 is used to acquire a first exposure image and a second exposure image of a first camera, and a third exposure image of a second camera, wherein the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same;

[0079] A first determination module 302 is used to perform target detection on the first exposure image to determine N image areas of the first exposure image, where the target detection includes at least one of brightness detection and motion detection, and N is a positive integer;

[0080] A fusion module 303 is used to fuse N image regions according to the first exposure image and the second exposure image to obtain a fused image;

[0081] The second determining module 304 is configured to determine a depth map according to the fused image and the third exposure image.

[0082] In the embodiment of the present application, the first exposure image and the second exposure image of the first camera with different exposure parameters can be obtained, and based on the exposure parameters of the second exposure image, the third exposure image of the second camera can be obtained; the first exposure image is subjected to brightness detection and / or motion detection to determine N image regions of the first exposure image; the N image regions are fused according to the first exposure image and the second exposure image to obtain a fused image; a depth map is determined according to the fused image and the third exposure image. In this way, the detection results of the brightness detection and / or motion detection can be used as a reference for the fusion ratio of the image content corresponding to different exposure parameters in each image region, so as to perform fusion processing and obtain a fused image with better dynamic effect and clarity, and then a depth map with higher accuracy can be obtained based on the fused image and the third exposure image.

[0083] In some embodiments, when target detection includes brightness detection and motion detection, each image region includes brightness information and motion information, and at least one of the brightness information and the motion information is different between any two image regions.

[0084] In this way, different image areas can be obtained by considering brightness information and motion information together, making the division of image areas more detailed, so that different image areas can be fused more finely to obtain a more accurate fused image, thereby improving the accuracy of the depth map.

[0085] In some embodiments, the fusion module 303 may also be used to:

[0086] Determine, according to a correspondence between preset brightness information and motion information and a fusion mode, a fusion mode corresponding to each of the N image regions, where the fusion mode includes using image content corresponding to the image region in the first exposure image, using image content corresponding to the image region in the second exposure image, or using image content after fitting the image regions in the first exposure image and the second exposure image;

[0087] According to the first exposure image and the second exposure image, the N image regions are fused in accordance with the fusion methods corresponding to the N image regions;

[0088] The N image regions after fusion processing are stitched together to generate a fused image.

[0089] In this way, each image area can be fused according to the fusion method corresponding to each image area and combined with the image content of the first exposure image and the second exposure image to obtain a fused image with better dynamic effect and clarity, and then a more accurate depth map can be obtained based on the fused image and the third exposure image.

[0090] In some embodiments, the second determining module 304 may also be used to:

[0091] performing exposure compensation on the third exposure image according to the fused image to obtain a fourth exposure image;

[0092] A depth map is determined according to the fused image and the fourth exposure image.

[0093] In this way, exposure compensation can be performed on the third exposure image to obtain a fourth exposure image that is more consistent with the fused image in brightness and contrast, further improving the accuracy of the depth map.

[0094] In some embodiments, the depth map determining apparatus 300 may further include:

[0095] a third determination module, configured to determine a foreground area and a background area of ​​a first image according to the depth map after determining the depth map according to the fused image and the third exposure image, wherein the first image is an image synthesized based on the first exposure image and the second exposure image;

[0096] The processing module is used to blur the background area to generate a blurred image.

[0097] In this way, based on the depth map with higher accuracy, the foreground area and the background area can be identified more accurately, thereby obtaining a blurred image with more precise background blur and better effect.

[0098] The depth map determination device in the embodiment of the present application can be a device or a component in the device, such as an integrated circuit or a chip. The device can be a terminal or other devices other than a terminal. Exemplarily, the device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0099] The depth map determination device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0100] The depth map determination device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0101] Alternatively, if Figure 4 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401 and a memory 402, wherein the memory 402 stores a program or instruction that can be executed on the processor 401, and when the program or instruction is executed by the processor 401, the various steps of the above-mentioned depth map determination method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0102] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0103] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0104] The electronic device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510.

[0105] Those skilled in the art will appreciate that the electronic device 500 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0106] The processor 510 may be used for:

[0107] Acquire a first exposure image and a second exposure image of the first camera, and a third exposure image of the second camera, wherein the first exposure image and the second exposure image have different exposure parameters, and the second exposure image and the third exposure image have the same exposure parameters;

[0108] Performing target detection on the first exposure image to determine N image regions of the first exposure image, where the target detection includes at least one of brightness detection and motion detection, and N is a positive integer;

[0109] According to the first exposure image and the second exposure image, N image regions are fused to obtain a fused image;

[0110] A depth map is determined based on the fused image and the third exposure image.

[0111] In the embodiment of the present application, the first exposure image and the second exposure image of the first camera with different exposure parameters can be obtained, and based on the exposure parameters of the second exposure image, the third exposure image of the second camera can be obtained; the first exposure image is subjected to brightness detection and / or motion detection to determine N image regions of the first exposure image; the N image regions are fused according to the first exposure image and the second exposure image to obtain a fused image; a depth map is determined according to the fused image and the third exposure image. In this way, the detection results of the brightness detection and / or motion detection can be used as a reference for the fusion ratio of the image content corresponding to different exposure parameters in each image region, so as to perform fusion processing and obtain a fused image with better dynamic effect and clarity, and then a depth map with higher accuracy can be obtained based on the fused image and the third exposure image.

[0112] In some embodiments, when target detection includes brightness detection and motion detection, each image region includes brightness information and motion information, and at least one of the brightness information and the motion information is different between any two image regions.

[0113] In this way, different image areas can be obtained by considering brightness information and motion information together, making the division of image areas more detailed, so that different image areas can be fused more finely to obtain a more accurate fused image, thereby improving the accuracy of the depth map.

[0114] In some embodiments, the processor 510 may also be configured to:

[0115] Determine, according to a correspondence between preset brightness information and motion information and a fusion mode, a fusion mode corresponding to each of the N image regions, where the fusion mode includes using image content corresponding to the image region in the first exposure image, using image content corresponding to the image region in the second exposure image, or using image content after fitting the image regions in the first exposure image and the second exposure image;

[0116] According to the first exposure image and the second exposure image, the N image regions are fused in accordance with the fusion methods corresponding to the N image regions;

[0117] The N image regions after fusion processing are stitched together to generate a fused image.

[0118] In this way, each image area can be fused according to the fusion method corresponding to each image area and combined with the image content of the first exposure image and the second exposure image to obtain a fused image with better dynamic effect and clarity, and then a more accurate depth map can be obtained based on the fused image and the third exposure image.

[0119] In some embodiments, the processor 510 may also be configured to:

[0120] performing exposure compensation on the third exposure image according to the fused image to obtain a fourth exposure image;

[0121] A depth map is determined according to the fused image and the fourth exposure image.

[0122] In this way, exposure compensation can be performed on the third exposure image to obtain a fourth exposure image that is more consistent with the fused image in brightness and contrast, further improving the accuracy of the depth map.

[0123] In some embodiments, the processor 510 may also be configured to:

[0124] After determining the depth map according to the fused image and the third exposure image, determining the foreground area and the background area of ​​the first image according to the depth map, the first image being an image synthesized based on the first exposure image and the second exposure image;

[0125] The background area is blurred to generate a blurred image.

[0126] In this way, based on the depth map with higher accuracy, the foreground area and the background area can be identified more accurately, thereby obtaining a blurred image with more precise background blur and better effect.

[0127] It should be understood that in the embodiment of the present application, the input unit 504 may include a graphics processor (GPU) 5041 and a microphone 5042, and the graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes a touch panel 5071 and at least one of other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0128] The memory 509 can be used to store software programs and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 509 may include a volatile memory or a non-volatile memory, or the memory 509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0129] The processor 510 may include one or more processing units; optionally, the processor 510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 510.

[0130] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned depth map determination method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0132] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned depth map determination method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0133] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0134] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned depth map determination method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0135] It should be noted that, in this article, the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for determining a depth map, characterized in that: The method comprises: Acquire a first exposure image and a second exposure image of the first camera, and a third exposure image of the second camera, wherein the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same; Performing target detection on the first exposure image to determine N image areas of the first exposure image, wherein the target detection includes at least one of brightness detection and motion detection, and N is a positive integer; According to the first exposure image and the second exposure image, the N image regions are fused to obtain a fused image; A depth map is determined according to the fused image and the third exposure image.

2. The method according to claim 1, characterized in that In the case where the target detection includes brightness detection and motion detection, each of the image regions includes brightness information and motion information, and at least one of the brightness information and the motion information is different between any two image regions.

3. The method according to claim 2, characterized in that The fusing the N image regions according to the first exposure image and the second exposure image to obtain a fused image includes: Determining a fusion mode corresponding to each of the N image regions according to a correspondence between preset brightness information and motion information and a fusion mode, wherein the fusion mode includes using image content corresponding to the image region in the first exposure image, using image content corresponding to the image region in the second exposure image, or using image content after fitting the image region in the first exposure image and the second exposure image; According to the first exposure image and the second exposure image, the N image regions are fused in accordance with the fusion modes corresponding to the N image regions; The N image regions after fusion processing are stitched together to generate a fused image.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining a depth map according to the fused image and the third exposure image includes: performing exposure compensation on the third exposure image according to the fused image to obtain a fourth exposure image; A depth map is determined according to the fused image and the fourth exposure image.

5. The method according to claim 1, characterized in that After determining the depth map according to the fused image and the third exposure image, the method further includes: Determine, according to the depth map, a foreground area and a background area of ​​a first image, wherein the first image is an image synthesized based on the first exposure image and the second exposure image; The background area is blurred to generate a blurred image.

6. A depth map determination device, characterized in that: The device comprises: an acquisition module, configured to acquire a first exposure image and a second exposure image of the first camera, and a third exposure image of the second camera, wherein the exposure parameters of the first exposure image and the second exposure image are different, and the exposure parameters of the second exposure image and the third exposure image are the same; A first determination module is used to perform target detection on the first exposure image to determine N image areas of the first exposure image, wherein the target detection includes at least one of brightness detection and motion detection, and N is a positive integer; a fusion module, configured to fuse the N image regions according to the first exposure image and the second exposure image to obtain a fused image; The second determining module is used to determine a depth map according to the fused image and the third exposure image.

7. The device according to claim 6, characterized in that In the case where the target detection includes brightness detection and motion detection, each of the image regions includes brightness information and motion information, and at least one of the brightness information and the motion information is different between any two image regions.

8. The device according to claim 7, characterized in that The fusion module is further used for: Determining a fusion mode corresponding to each of the N image regions according to a correspondence between preset brightness information and motion information and a fusion mode, wherein the fusion mode includes using image content corresponding to the image region in the first exposure image, using image content corresponding to the image region in the second exposure image, or using image content after fitting the image region in the first exposure image and the second exposure image; According to the first exposure image and the second exposure image, the N image regions are fused in accordance with the fusion modes corresponding to the N image regions; The N image regions after fusion processing are stitched together to generate a fused image.

9. The device according to any one of claims 6 to 8, characterized in that The second determining module is further used for: performing exposure compensation on the third exposure image according to the fused image to obtain a fourth exposure image; A depth map is determined according to the fused image and the fourth exposure image.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images