Photographing method and device, storage medium, terminal and computer program product

Abstract

The invention discloses a photographing method and device, a storage medium, a terminal and a computer program product, and the method comprises the steps: carrying out the synchronous multi-frame photographing of a same target scene through employing a plurality of cameras, obtaining a plurality of frames of preliminary photographing images corresponding to each camera, carrying out the fusion of the preliminary photographing images, and obtaining a first fusion image corresponding to the camera, the plurality of cameras are divided into at least one group of cameras; determining a depth map corresponding to each group of cameras, determining a depth value difference between each pixel position of the depth map and a surrounding pixel position, and determining a first fusion weight of each pixel belonging to the pixel position in each frame of first fusion image corresponding to the group of cameras at least based on the depth value difference, fusing each frame of first fusion image based on the first fusion weight to obtain a second fusion image; and carrying out image fusion on the multiple frames of second fusion images corresponding to each group of cameras to obtain a final shot image. According to the scheme, the blurring or smear problem of the moving target area in the image can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of photographing technology, and in particular to a photographing method and device, a storage medium, a terminal, and a computer program product. Background Art

[0002] In the actual shooting process, the captured image content can be preliminarily divided into static targets and moving targets. Moving targets can be further divided into global moving targets (both the shooting device and the shooting target are in motion) and local moving targets (the shooting device is fixed and the shooting target is in motion). Since the exposure time is set within a certain range during the shooting process, it is inevitable that the shooting content will be blurred or there will be a smear due to the moving target's excessive speed, affecting the captured image quality.

[0003] In the prior art, in order to solve the blur or smear problem caused by target motion, a common method is to reduce the exposure time. However, this will lead to a significant increase in the noise in the dark area of ​​the image, especially in scenes with dim light sources (for example, night scenes), resulting in poor image quality. Another method is a single-shot multi-frame fusion solution. Specifically, multiple frames of images taken by a single camera at different times are fused. However, this solution can effectively remove the temporal noise in the image area of ​​the stationary target, but it is difficult to effectively solve the blur problem for fast-moving targets in the image. Even if the moving target area is subjected to noise reduction, on the one hand, the noise reduction process will inevitably lose the details of the moving target area, and on the other hand, the effect of improving image quality is still limited. Summary of the invention

[0004] The technical problem solved by the embodiments of the present invention is how to effectively improve the blur or smear problem of the moving target area in the image when shooting a scene containing a moving target, so as to improve the quality of the shot image.

[0005] To solve the above technical problems, an embodiment of the present invention provides a photographing method, comprising the following steps: using multiple cameras to synchronously shoot multiple frames of the same target scene to obtain multiple frames of preliminary shot images corresponding to each camera, and performing image fusion on the multiple frames of preliminary shot images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras; for each group of cameras, according to the multiple groups of preliminary shot images synchronously shot by the group of cameras, a depth map of the group of cameras relative to the target scene is determined, and each pixel position of the depth map has a respective depth value, wherein each group of preliminary shot images includes at least two frames of preliminary shot images synchronously shot by each camera in the group; For each pixel position of the depth map of the group of cameras, the depth value difference between the pixel position and the surrounding pixel positions is determined, and based at least on the depth value difference, a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fused image corresponding to the group of cameras is determined, wherein the greater the depth value difference is, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position is; for each frame of the first fused image corresponding to each group of cameras, a weighted operation is performed on the pixel value of each pixel using the plurality of first fusion weights corresponding to each pixel belonging to each pixel position, so as to obtain a second fused image corresponding to the group of cameras; and image fusion is performed on multiple frames of the second fused images corresponding to each group of cameras to obtain a final captured image.

[0006] Optionally, the multiple cameras synchronously perform multi-frame shooting for the target scene using target shooting parameters; before using multiple cameras to synchronously perform multi-frame shooting for the same target scene, the method also includes: for at least one camera among the multiple cameras, determining multiple frame preview images of the target scene in preview mode of the camera, and performing motion analysis on moving targets in the multiple frame preview images to determine motion information of the moving targets; performing weighted operation on the motion information obtained by the at least one camera to obtain weighted motion information; and determining the target shooting parameters based on the weighted motion information.

[0007] Optionally, the motion information includes one or more of a motion speed, a motion direction, and an area of ​​an image region to which the moving target belongs, and the target shooting parameters include one or more of an exposure time, a camera movement direction, and a magnification ratio. Based on the weighted motion information, the target shooting parameters are determined, including: determining the exposure time corresponding to the motion speed according to the motion speed and a preset speed-exposure time mapping relationship, wherein the faster the motion speed, the shorter the corresponding exposure time; and / or, using the motion direction as the camera movement direction; and / or, determining the magnification ratio corresponding to the area of ​​the image region to which the moving target belongs according to the area of ​​the image region to which the moving target belongs and a preset area-magnification ratio mapping relationship, wherein the larger the area of ​​the image region to which the moving target belongs, the smaller the corresponding magnification ratio.

[0008] Optionally, determining a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in the first fused image of each frame corresponding to the group of cameras at least based on the depth value difference includes: determining a candidate indicator difference for each pixel belonging to the pixel position, wherein the candidate indicator difference is selected from one or more of the following: brightness value difference, contrast difference, texture difference; performing a weighted operation on the depth value difference and each candidate indicator difference to obtain a weighted difference, wherein the depth value difference has the largest weight in the weighted operation process; determining a plurality of first fusion weights corresponding to each pixel belonging to the pixel position based on the weighted difference, wherein the greater the weighted difference, the greater the difference between the plurality of first fusion weights.

[0009] Optionally, when there are differences between multiple first fusion weights corresponding to pixels belonging to the same pixel position in the first fusion image of each frame corresponding to each group of cameras, the multiple first fusion weights satisfy: the higher the image quality of the first fusion image to which the pixel belongs, the greater the first fusion weight corresponding to the pixel.

[0010] Optionally, determining, for each pixel position of the depth map of the group of cameras, the depth value difference between the pixel position and the surrounding pixel positions includes: determining, for each pixel position of the depth map of the group of cameras, the depth difference between the depth value of the pixel position and the depth values ​​of multiple pixel positions within a preset area around it, thereby obtaining multiple depth differences corresponding to the pixel position; and calculating the mean square error of the multiple depth differences, and using the mean square error as the depth value difference.

[0011] Optionally, image fusion is performed on the multiple frames of preliminary captured images to obtain a first fused image corresponding to the camera, including: determining a preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, and recording it as a reference frame image; performing one or more rounds of fusion operations based on the reference frame image and the remaining frames of preliminary captured images, and using the fusion result of the last round as the first fused image corresponding to the camera; wherein, in each round of fusion, image fusion is performed on the fusion result of the previous round and one frame of the remaining preliminary captured images that do not participate in image fusion to obtain the fusion result of the current round; wherein, the fusion result of the first round is obtained by image fusion of the reference frame image and a frame of preliminary captured image other than the reference frame image.

[0012] Optionally, in each round of fusion, the fusion result of the previous round is recorded as the first frame to be fused, and one frame of the remaining preliminary captured images that does not participate in image fusion is recorded as the second frame to be fused; in each round of fusion, the fusion result of the previous round and one frame of the remaining preliminary captured images that does not participate in image fusion are subjected to image fusion to obtain the fusion result of the current round, including: for each pair of pixels in the same position in the first frame to be fused and the second frame to be fused, determining the pixel value difference of the pair of pixels; determining the second fusion weight corresponding to the pair of pixels based on the pixel value difference, and using the second fusion weight to perform a weighted operation on the pixel values ​​of the pair of pixels to obtain a fused pixel value, wherein the greater the pixel value difference, the greater the difference between the second fusion weights corresponding to the pair of pixels; using the fused pixel value of each pair of pixels in the same position in the first frame to be fused and the second frame to be fused as the fusion result of the current round.

[0013] Optionally, before performing image fusion on the multiple frames of preliminary captured images, the method also includes: determining a preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, recording it as a reference frame image, and recording preliminary captured images other than the reference frame image as non-reference frame images; for each frame of preliminary captured image, determining an image area to which a moving target in the frame of preliminary captured image belongs, and performing feature point extraction on image areas other than the image area to obtain a feature point set of the frame of preliminary captured image; for each frame of non-reference frame image, determining an alignment matrix based on the feature point set of the reference frame image of the frame and the feature point set of the reference frame image, and using the alignment matrix to align the reference frame image of the frame with the reference frame image.

[0014] Optionally, for each group of cameras, a depth map of the group of cameras relative to the target scene is determined based on multiple groups of preliminary captured images synchronously taken by the group of cameras, including: for each group of cameras, selecting a group of preliminary captured images with the best image quality from the multiple groups of preliminary captured images synchronously taken by the group of cameras; and determining the depth map of the group of cameras relative to the target scene based on the group of preliminary captured images with the best image quality.

[0015] The embodiment of the present invention further provides a photographing device, comprising: a first fusion module, used to use multiple cameras to synchronously shoot multiple frames of the same target scene, obtain multiple frames of preliminary shot images corresponding to each camera, and perform image fusion on the multiple frames of preliminary shot images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras; a depth map determination module, used to determine, for each group of cameras, a depth map of the group of cameras relative to the target scene based on the multiple groups of preliminary shot images synchronously shot by the group of cameras, each pixel position of the depth map has its own depth value, wherein each group of preliminary shot images includes at least two frames of preliminary shot images synchronously shot by each camera in the group; a fusion weight determination module, used to determine the depth map of the group of cameras relative to the target scene based on the multiple groups of preliminary shot images synchronously shot by the group of cameras. For each pixel position of the depth map of the group of cameras, the depth value difference between the pixel position and the surrounding pixel positions is determined, and based at least on the depth value difference, a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fused image corresponding to the group of cameras is determined, wherein the greater the depth value difference is, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position is; a multi-camera fusion module is used to perform weighted operation on the pixel value of each pixel in each frame of the first fused image corresponding to each group of cameras, using the plurality of first fusion weights corresponding to each pixel belonging to each pixel position, so as to obtain the second fused image corresponding to the group of cameras; a second fusion module is used to perform image fusion on multiple frames of second fused images corresponding to each group of cameras to obtain a final captured image.

[0016] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned photographing method are executed.

[0017] An embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above-mentioned photographing method when running the computer program.

[0018] An embodiment of the present invention further provides a computer program product, including a computer program, which executes the steps of the above-mentioned photographing method when executed by a processor.

[0019] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0020] In an embodiment of the present invention, a single-shot multi-frame fusion scheme and a multi-shot multi-frame re-fusion scheme are combined to obtain a higher quality captured image. Specifically, multiple cameras are used to synchronously shoot multiple frames for the same target scene to obtain multiple frames of preliminary captured images corresponding to each camera, and the multiple frames of preliminary captured images are image fused to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras; then for each group of cameras, according to the multiple groups of preliminary captured images synchronously shot by the group of cameras, a depth map of the group of cameras relative to the target scene is determined, and based on the depth map information of the group of cameras, a fusion weight (i.e., the first fusion weight) of pixels at the same pixel position in each frame of the first fused image is adaptively allocated, and then based on the first fusion weight, multi-shot multi-frame fusion is performed (i.e., the first fused images corresponding to each of the multiple cameras are re-fused) to obtain a second fused image corresponding to each group of cameras; finally, the multiple frames of second fused images corresponding to each group of cameras are fused to obtain a captured image.

[0021] From the above, on the one hand, image fusion of multiple frames of preliminary images taken by the same camera can preliminarily improve the overall quality of the image; on the other hand, by re-fusing the first fused images corresponding to multiple cameras, not only can depth information be established from different perspectives, but also the weight of multi-camera multi-frame fusion can be adjusted using depth information to avoid blur in the time domain. Furthermore, the images taken synchronously between multiple cameras can compensate for the missing information from different perspectives, and can also effectively reduce the blur problem in the moving area, further improving the quality of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a photographing method in an embodiment of the present invention;

[0023] Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11;

[0024] Figure 3 yes Figure 1 A flowchart of a specific implementation of step S13;

[0025] Figure 4 is a partial flow chart of another photographing method in an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the architecture of a photographing system for implementing the photographing method described in an embodiment of the present invention;

[0027] Figure 6 It is a structural schematic diagram of a photographing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and understandable, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0029] Reference Figure 1 , Figure 1 1 is a flow chart of a method for taking photos in an embodiment of the present invention. The method can be applied to various terminal devices with a multi-camera synchronous shooting function, such as mobile phones, tablet computers, computers, smart wearable devices (such as smart watches), and autonomous driving vehicles equipped with multiple cameras.

[0030] The method may include steps S11 to S14:

[0031] Step S11: using multiple cameras to synchronously shoot multiple frames of the same target scene to obtain multiple frames of preliminary shot images corresponding to each camera, and performing image fusion on the multiple frames of preliminary shot images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras;

[0032] Step S12: for each group of cameras, according to a plurality of groups of preliminary captured images synchronously captured by the group of cameras, determining a depth map of the group of cameras relative to the target scene, wherein each pixel position of the depth map has a respective depth value, wherein each group of preliminary captured images includes at least two frames of preliminary captured images synchronously captured by each camera in the group;

[0033] Step S13: for each pixel position of the depth map of the group of cameras, determine the depth value difference between the pixel position and the surrounding pixel positions, and determine a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fusion image corresponding to the group of cameras at least based on the depth value difference, wherein the greater the depth value difference, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position;

[0034] Step S14: for each frame of the first fused image corresponding to each camera group, a plurality of first fusion weights corresponding to each pixel at each pixel position are used to perform a weighted operation on the pixel value of each pixel, so as to obtain a second fused image corresponding to the camera group;

[0035] Step S15: performing image fusion on multiple frames of second fused images corresponding to each group of cameras to obtain a final captured image.

[0036] It can be understood that, in a specific implementation, the method can be implemented in the form of a software program, which runs in a processor integrated inside a chip or a chip module (for example, an image signal processing (Image Signal Processor, ISP) chip or chip module); or, the method can be implemented in hardware or a combination of hardware and software.

[0037] In the specific implementation of step S11, taking a smartphone as an example, the multiple cameras may be cameras arranged on the back of the smartphone, and the multiple frames of preliminary captured images obtained by the multiple cameras at the same time (i.e., one synchronous capture) for the target scene have overlapping areas. The target scene may include stationary targets and / or moving targets. The moving targets may be, for example, people in a running or walking state, vehicles in a driving state, animals in a moving state, etc.

[0038] The method of using multiple cameras to synchronously shoot multiple frames of the same target scene may specifically refer to: using the multiple cameras to synchronously shoot the target scene multiple times, each time the multiple cameras shoot a frame of preliminary shot image, and after multiple synchronous shots, each camera obtains multiple frames of preliminary shot images.

[0039] As a nonlinear example, three cameras (respectively denoted as camera A, camera B, and camera C) are used to synchronously shoot multiple frames of the same target scene. The multiple frames of preliminary images shot by each camera have their own shooting time. For example, cameras A, B, and C are at time t1, time t2, and so on. n At each time, a synchronous shooting is performed to obtain preliminary shooting images a1, b1, c1 obtained by cameras A, B, and C at time t1, preliminary shooting images a2, b2, c2 obtained by cameras A, B, and C at time t2, and so on. n The initial image a obtained by synchronous shooting at all times n 、b n 、c n .

[0040] Among them, for camera A, the corresponding multiple frames of preliminary captured images are a1, a2...a n ; For camera B, the corresponding multiple frames of preliminary captured images are b1, b2...b n ; For camera C, the corresponding multiple frames of preliminary captured images are c1, c2...c n .

[0041] It is understandable that, in practical applications, in order to improve the blur or smear problem of the moving target as much as possible, multiple synchronous shooting times t1, t2, ..., t nThe time interval between the two images is as small as possible, for example, multiple consecutive snapshots are used to achieve multiple synchronous shootings.

[0042] In a specific implementation, the manner of grouping the multiple cameras may be selected from but not limited to the following.

[0043] Method 1: Take every N cameras as a group (for example, they can be randomly combined or all possible combinations can be exhausted) to obtain at least one group of cameras, where N is greater than or equal to 2 and less than or equal to the total number of the multiple cameras, and N is a positive integer (when N is 2, each group has two cameras; when N is the total number of the multiple cameras, the multiple cameras are all divided into one group of cameras).

[0044] Method 2: Select a camera with the best shooting quality from the multiple cameras as a reference camera, and group the remaining cameras with every M cameras (for example, they can be randomly combined or all possible combinations can be exhausted) to obtain at least one group of initial cameras; then group each group of initial cameras and the reference camera as a group to obtain the at least one group of cameras, where M is greater than or equal to 1 and less than or equal to the total number of the multiple cameras minus 1, and M is a positive integer (when M is 1, each group has two cameras, one of which is the reference camera; when M is the total number of the multiple cameras minus 1, the multiple cameras are all divided into one group of cameras).

[0045] Furthermore, in a specific implementation manner, the multiple frames of preliminary captured images corresponding to each camera (for example, the multiple frames of preliminary captured images a1, a2, ... a corresponding to camera A) are n ) The image fusion scheme can be adopted Figure 2 Fusion scheme shown.

[0046] Reference Figure 2 , Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11 in the embodiment, wherein in step S11, image fusion is performed on the multiple frames of preliminary captured images to obtain a first fused image corresponding to the camera, which may specifically include steps S111 to S112.

[0047] In step S111 , a preliminary captured image with the highest image quality is determined from the multiple frames of preliminary captured images and recorded as a reference frame image.

[0048] In a specific implementation, the method for determining the preliminary captured image with the highest image quality may be a manual image quality assessment method or a machine learning method.

[0049] In step S112, one or more rounds of fusion operations are performed based on the reference frame image and the remaining preliminary captured images, and the fusion result of the last round is used as the first fused image corresponding to the camera.

[0050] Among them, in each round of fusion, the fusion result of the previous round and one frame of the remaining preliminary captured images that did not participate in the image fusion are image fused to obtain the fusion result of the current round; wherein, the fusion result of the first round is obtained by image fusion of the reference frame image and a frame of preliminary captured image other than the reference frame image.

[0051] Furthermore, in each round of fusion, the fusion result of the previous round is recorded as the first frame to be fused, and one frame of the remaining preliminary captured images that does not participate in image fusion is recorded as the second frame to be fused; in each round of fusion, the fusion result of the previous round and one frame of the remaining preliminary captured images that does not participate in image fusion are subjected to image fusion to obtain the fusion result of the current round, including: for each pair of pixels in the same position in the first frame to be fused and the second frame to be fused, determining the pixel value difference of the pair of pixels; determining the second fusion weight corresponding to the pair of pixels based on the pixel value difference, and using the second fusion weight to perform a weighted operation on the pixel values ​​of the pair of pixels to obtain a fused pixel value, wherein the greater the pixel value difference, the greater the difference between the second fusion weights corresponding to the pair of pixels; using the fused pixel value of each pair of pixels in the same position in the first frame to be fused and the second frame to be fused as the fusion result of the current round.

[0052] Specifically, if the pixel difference of the pixel pair at the same position is less than a first preset threshold (for example, it can be set to 0, or a smaller value close to 0), then the two second fusion weights corresponding to the pair of pixels can be equal (for example, they can both be 0.5); if the pixel difference of the pixel pair at the same position is greater than or equal to the first preset threshold, then there is a difference between the two second fusion weights corresponding to the pair of pixels, and the greater the difference in pixel values, the greater the difference between the two second fusion weights corresponding to the pair of pixels.

[0053] Furthermore, for each pair of pixels in the same position, when there is a difference between the two second fusion weights corresponding to the pair of pixels, the second fusion weight corresponding to the pixels in the pair of pixels belonging to the frame to be fused with higher image quality is greater than the second fusion weight corresponding to the pixels belonging to the frame to be fused with lower image quality.

[0054] In a specific implementation, the sum of the two second fusion weights corresponding to each pair of pixels in the same position in the first frame to be fused and the second frame to be fused can be 1, that is, 1 can be weighted according to the pixel difference of the pair of pixels to determine the two second fusion weights corresponding to the pair of pixels.

[0055] As a non-limiting embodiment, when the sum of the two second fusion weights corresponding to the pair of pixels is 1 and there is a difference between the two, the second fusion weight corresponding to the pixels of the frame to be fused with higher image quality is, for example, 0.6 (or 0.7), and the second fusion weight corresponding to the pixels of the frame to be fused with lower image quality is, for example, 0.4 (or 0.3).

[0056] Regarding the specific method of weight allocation, it can be appropriately allocated in combination with the actual application scenario requirements, under the premise that the difference between the two second fusion weights corresponding to each pair of pixels is positively correlated with the difference in the pixel values ​​of the pair of pixels, and the embodiment of the present invention does not make specific limitations.

[0057] It can be understood that, for each pair of pixels in the same position in the first frame to be fused and the second frame to be fused, the greater the difference in pixel values ​​of the pair of pixels, the greater the possibility that the target to which the pair of pixels belongs is a moving target, and the greater the difference in pixel values ​​often also means a faster movement speed; the smaller the difference in pixel values ​​of the pair of pixels, the greater the possibility that the target to which the pair of pixels belongs is a stationary target. Therefore, in an embodiment of the present invention, for a moving target, there is a difference in the two second fusion weights corresponding to the pixels, that is, the pixels of one of the two frames to be fused are used as the main factors affecting the weighted pixel values, and the pixels of the other frame are used as the secondary factors affecting the weighted pixel values; and for a stationary target, the two second fusion weights corresponding to the pixels are set as consistent as possible, that is, the influence of the two pixels at the same position in the two frames to be fused on the weighted fusion is close to the same. Thereby, it is helpful to improve the effect of image fusion, especially to improve the clarity of the moving target area after fusion.

[0058] Furthermore, for each pair of pixels at the same position, when there is a difference between the two second fusion weights corresponding to the pair of pixels (i.e., it means that the pair of pixels has a greater probability of belonging to a moving target), this embodiment also specifically sets the second fusion weight corresponding to the pixel of the pair of pixels belonging to the frame to be fused with higher image quality to be greater than the second fusion weight corresponding to the pixel of the frame to be fused with lower image quality. Thus, for moving targets, the pixel features of the frame to be fused with higher image quality can be mainly retained, further improving the image fusion quality.

[0059] It should be noted that, in a specific implementation, in addition to the above-mentioned solution, other appropriate methods may be used to fuse the multiple frames of preliminary captured images corresponding to each camera. For example, the following steps may be used to fuse the images:

[0060] Step (1): determining a preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, recording it as a reference frame image, and recording the remaining preliminary captured images as non-reference frame images;

[0061] Step (2): for each non-reference frame image, determining the difference in pixel values ​​between each pair of pixels at the same position in the non-reference frame image and the reference frame image, and determining the third fusion weight corresponding to the pair of pixels based on the pixel value difference, wherein the greater the pixel value difference, the greater the difference between the two third fusion weights corresponding to the pair of pixels;

[0062] Step (3): normalizing the third fusion weights corresponding to the pairs of pixels at the same position in each non-reference frame image and the reference frame image to obtain target weights corresponding to all pixels at the same position in each non-reference frame image and the reference frame image;

[0063] Step (4): For all pixels belonging to the same position in each non-reference frame image and the reference frame image, use their corresponding target weights to perform a weighted operation on the pixel values ​​of all pixels at the same position to obtain the weighted pixel value corresponding to the position, thereby obtaining the first fused image corresponding to the camera.

[0064] Furthermore, before performing image fusion on the multiple frames of preliminary captured images, the method also includes: determining the preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, recording it as the reference frame image, and recording the preliminary captured images other than the reference frame image as non-reference frame images; for each frame of preliminary captured image, determining the image area to which the moving target in the frame of preliminary captured image belongs, and performing feature point extraction on the image area other than the image area to obtain a feature point set of the frame of preliminary captured image; for each frame of non-reference frame image, determining an alignment matrix based on the feature point set of the reference frame image of the frame and the feature point set of the reference frame image, and using the alignment matrix to align the reference frame image of the frame with the reference frame image.

[0065] In an embodiment of the present invention, performing an image alignment operation based on feature points of an image area outside the image area to which the moving target belongs helps to reduce the impact of the blurred moving area itself on the alignment result and improve the alignment accuracy, thereby improving the image fusion effect of multiple frames of preliminary captured images taken by each camera.

[0066] Continue to refer to Figure 1 In the specific implementation of step S12, for each group of cameras, each group of preliminary captured images synchronously captured by the group of cameras includes: at least two frames of preliminary captured images obtained by synchronously capturing each camera in the group.

[0067] Continuing with the example of using three cameras A, B, and C to perform multiple synchronous shooting in the above description of step S11, it is assumed that the three cameras are divided into a group, and the cameras in the group are at time t1, time t2, and so on. nThe multiple groups of preliminary images taken synchronously by the group of cameras are as follows:

[0068] The preliminary images a1, b1, c1 taken by cameras A, B, and C at time t1 are obtained synchronously; the preliminary images a2, b2, c2 taken by cameras A, B, and C at time t2 are obtained synchronously... The preliminary images a1, b1, c1 taken by cameras A, B, and C at time t1 are obtained synchronously; the preliminary images a2, b2, c2 taken by cameras A, B, and C at time t2 are obtained synchronously n The initial image a obtained by synchronous shooting at all times n 、b n 、c n .

[0069] Furthermore, in step S12, the method for determining the depth map of each camera group relative to the target scene may include: for each camera group, selecting a group of preliminary images with the best image quality from multiple groups of preliminary images synchronously taken by the camera group; and determining the depth map of the camera group relative to the target scene based on the group of preliminary images with the best image quality. This helps to obtain a more accurate depth map, thereby improving the accuracy of the subsequent determination of pixel fusion weights based on the depth map.

[0070] In the specific implementation of step S13, the following method can be used to determine the depth value difference between each pixel position of the depth map and the surrounding pixel positions: for each pixel position of the depth map of the group of cameras, the depth difference between the depth value of the pixel position and the depth values ​​of multiple pixel positions within a preset area around it is determined respectively, so as to obtain multiple depth differences corresponding to the pixel position; and the mean square error of the multiple depth differences is calculated, and the mean square error is used as the depth value difference.

[0071] In a specific implementation, in addition to using the mean square error of the depth difference between each pixel position and its surrounding pixel positions as the depth difference, other appropriate indicators can also be used as the depth difference:

[0072] For example, for each pixel position in the depth map, the absolute value of the difference between the depth values ​​of the pixel position and multiple surrounding pixel positions is calculated respectively; and the sum of the calculated multiple absolute values ​​is used as the depth value difference.

[0073] For another example, for each pixel position in the depth map, the average value of the depth values ​​of multiple pixel positions around the pixel position is calculated; and the difference between the depth value of the pixel position and the average value is used as the depth value difference.

[0074] In a specific implementation, multiple first fusion weights corresponding to pixels at the same pixel position in the first fusion images of each frame corresponding to the group of cameras may be determined directly based on a single difference item, namely, the depth value difference.

[0075] In another specific implementation, in addition to considering the depth value difference, other appropriate candidate indicator differences can also be introduced, and based on the weighted results of the depth value difference and the candidate indicator differences, multiple first fusion weights corresponding to each pixel belonging to the same pixel position in each frame of the first fused image corresponding to the group of cameras are determined. In this way, it is helpful to improve the accuracy of each first fusion weight and improve the fusion effect, especially for the moving target area, it is more helpful to improve the blur problem and enhance the clarity of the corresponding image area after fusion. For details about the other specific implementation, please refer to the following about Figure 3 Description.

[0076] Reference Figure 3 , Figure 3 yes Figure 1 FIG. 4 is a flowchart of a specific implementation of step S13 in FIG. 4 . In this implementation, step S13 may specifically include steps S131 to S133.

[0077] In step S131, for each pixel belonging to the pixel position, a candidate indicator difference of each pixel is determined, and the candidate indicator difference is selected from one or more of the following: brightness value difference, contrast difference and texture difference.

[0078] In a specific implementation, the candidate indicator difference of each pixel belonging to the same pixel position can be determined in the following manner: taking the brightness value difference as an example, for each pixel belonging to the same pixel position, determine the initial brightness difference between the pixel and the surrounding pixels (for example, the mean square error of the difference between the brightness values ​​of the pixel and the surrounding pixels can be used); average the initial brightness differences corresponding to each pixel belonging to the same pixel position to obtain the brightness value difference of each pixel belonging to the same pixel position. Alternatively, other appropriate methods can be used to determine the candidate indicator difference of each pixel belonging to the same pixel position, as long as the candidate indicator difference can be used to characterize the possibility that the pixels at the pixel position and the surrounding pixel positions belong to the same target.

[0079] In step S132, a weighted operation is performed on the depth value difference and the differences of each candidate indicator to obtain a weighted difference, wherein the depth value difference has the largest weight in the weighted operation process.

[0080] In step S133, a plurality of first fusion weights corresponding to the pixels belonging to the pixel position are determined based on the weighted difference. The greater the weighted difference is, the greater the difference between the plurality of first fusion weights is.

[0081] It can be understood that, for each pixel position of the first fused image of each frame corresponding to each group of cameras, the greater the difference (or weighted difference) in depth value between the pixel position and the surrounding pixel positions, the smaller the possibility that the pixel position and the surrounding pixel positions belong to the same target (that is, the greater the probability that the pixel position belongs to the boundary pixel between two different targets). Accordingly, in this embodiment, the greater the difference between the multiple first fusion weights corresponding to the pixels belonging to the pixel position in the first fused image of each frame, that is, the pixels of one of the first fused images of each frame are used as the main factors affecting the weighted pixel value, and the pixels of the remaining first fused images are used as secondary factors affecting the weighted pixel value.

[0082] For pixel positions with smaller depth value differences from surrounding pixel positions (which means the pixel position and surrounding pixel positions are more likely to belong to the same target), multiple first fusion weights corresponding to each pixel belonging to the pixel position in each frame's first fusion image are set as consistent as possible, that is, the influence of pixels at the same position in each frame's first fusion image on weighted fusion is nearly consistent. This helps to improve the noise removal effect on the moving target area and enhance the clarity of the fused image.

[0083] Specifically, if the weighted difference is less than a second preset threshold (for example, it can be set to 0, or a smaller value close to 0), the multiple first fusion weights corresponding to the pixels belonging to the pixel position can be equal; if the weighted difference is greater than or equal to the second preset threshold, there are differences in the multiple first fusion weights corresponding to the pixels belonging to the pixel position, and the greater the weighted difference, the greater the difference between the multiple first fusion weights corresponding to the pixels belonging to the pixel position.

[0084] Furthermore, in the case where there are differences between the multiple first fusion weights corresponding to the pixels belonging to the same pixel position in the first fusion images of each frame corresponding to each camera group, the multiple first fusion weights satisfy the following: the higher the image quality of the first fusion image to which the pixel belongs, the greater the first fusion weight corresponding to the pixel. In this way, for pixels with a low probability of belonging to the same target (i.e., boundary pixels between different targets), the pixel features of the first fusion image with higher image quality can be mainly retained, further improving the image fusion quality.

[0085] In a specific implementation, the sum of the multiple first fusion weights corresponding to the pixels belonging to the same pixel position can be 1, that is, 1 can be weighted according to the weighted difference to determine the multiple first fusion weights corresponding to the pixels belonging to the same pixel position.

[0086] In a non-limiting embodiment, each group has 2 cameras, and in the two frames of first fused images corresponding to the group of cameras, the sum of the two first fusion weights corresponding to two pixels belonging to the same pixel position is 1, and when there is a difference between the two, the first fusion weight corresponding to the pixels belonging to the first fused image with higher image quality is, for example, 0.6 (or 0.7), and the first fusion weight corresponding to the pixels belonging to the first fused image with lower image quality is, for example, 0.4 (or 0.3).

[0087] In another non-limiting embodiment, each group has 3 cameras, and the sum of the 3 first fusion weights corresponding to the 3 pixels at the same pixel position in the 3 frames of the first fusion images corresponding to the cameras in the group is 1, and there are differences between the first fusion weights, the first fusion weight corresponding to the pixel of the first fusion image with the highest image quality is, for example, 0.5, the first fusion weight corresponding to the pixel of the first fusion image with the second highest image quality is, for example, 0.3, and the first fusion weight corresponding to the pixel of the first fusion image with the lowest image quality is, for example, 0.2. Alternatively, the first fusion weights corresponding to the pixels of the first fusion images with the second highest image quality and the lowest image quality may be the same, for example, both are 0.25.

[0088] Regarding the specific method of weight allocation, it can be appropriately allocated in combination with the actual application scenario requirements, under the premise that the difference between the multiple first fusion weights corresponding to each pixel is positively correlated with the weighted difference, and the embodiment of the present invention does not make specific limitations.

[0089] Continue to refer to Figure 1 In the specific implementation of step S14, for each frame of the first fused image corresponding to each group of cameras, a plurality of first fusion weights corresponding to each pixel belonging to each pixel position are used to perform weighted operation on the pixel value of each pixel, so as to obtain the second fused image corresponding to the group of cameras.

[0090] In the specific implementation of step S15, the multiple frames of second fused images corresponding to each group of cameras are fused to obtain a final captured image. The detailed implementation scheme for fusing the multiple frames of second fused images can be referred to the above implementation scheme for fusing the multiple frames of preliminary captured images corresponding to each camera in step S11, which will not be repeated here.

[0091] Reference Figure 4 , Figure 4 is a partial flow chart of another photographing method in an embodiment of the present invention. In this embodiment, the multiple cameras use target photographing parameters to synchronously photograph multiple frames for the target scene; the another photographing method may include Figure 1The steps S11 to S15 shown may further include the following steps S41 to S43, wherein S41 to S43 may be performed before step S11.

[0092] In step S41, for at least one camera among the multiple cameras, a plurality of preview images of the camera for the target scene in preview mode is determined, and motion analysis is performed on moving targets in the plurality of preview images to determine motion information of the moving targets.

[0093] The at least one camera may be one or more cameras with higher shooting quality among the multiple cameras.

[0094] The preview mode (or preview state) may refer to a mode or state before the camera function key is triggered. In the preview mode, the terminal device usually collects and stores the preview frames collected by each camera for the target scene in real time. For each camera, the multiple preview frames of the camera for the target scene in the preview mode may be selected from the preview frames collected by the camera for the target scene. For example, multiple preview frames collected within a preset time period before the camera function key is triggered may be selected.

[0095] Among them, the method for performing motion analysis on the moving targets in the multi-frame preview images can adopt existing conventional methods, such as a motion analysis algorithm based on optical flow, a motion analysis method based on fast corner detection and binary basic feature extraction algorithm (Oriented FAST and Rotated BRIEF, ORB).

[0096] In step S42, a weighted operation is performed on the motion information obtained by the at least one camera to obtain weighted motion information.

[0097] In step S43, the target shooting parameters are determined based on the weighted motion information.

[0098] Compared with performing motion analysis based on multiple preview frames captured by a single camera to obtain motion information and determine target shooting parameters, this implementation uses weighted motion information determined by multiple cameras to obtain more accurate and reliable target shooting parameters.

[0099] Further, the motion information may include one or more of the motion speed, the motion direction, and the area of ​​the image region to which the moving target belongs, and the target shooting parameter may include one or more of the exposure time, the camera moving direction, and the magnification. In step S43, determining the target shooting parameter based on the weighted motion information may specifically include:

[0100] According to the movement speed and a preset speed-exposure duration mapping relationship, the exposure duration corresponding to the movement speed is determined, wherein the faster the movement speed, the shorter the corresponding exposure duration; and / or, the movement direction is used as the camera movement direction; and / or, according to the area of ​​the image area to which the moving target belongs and a preset area-magnification ratio mapping relationship, the magnification ratio (i.e., zoom factor) corresponding to the area of ​​the image area to which the moving target belongs is determined, wherein the larger the area of ​​the image area to which the moving target belongs, the smaller the corresponding magnification ratio.

[0101] In a specific implementation, the speed-exposure duration mapping relationship and the area-magnification mapping relationship may be empirical data determined in advance based on experiments, and may be specifically expressed in the form of a mapping relationship table or a fitted mapping relationship function.

[0102] In a specific implementation, after the target shooting parameters are determined, they can be displayed on a display screen (or user interaction screen) of a terminal device, in the form of text, arrow graphics, etc. The user can choose whether to apply the target shooting parameters to the multiple cameras of the terminal device according to actual needs, so as to synchronously shoot multiple frames for the target scene.

[0103] In another specific implementation, after the target shooting parameters are determined, they can be automatically applied to the multiple cameras of the terminal device to synchronously perform multi-frame shooting for the target scene.

[0104] In the embodiment of the present invention, on the basis of adopting a single-shot multi-frame (i.e., multiple frames of preliminary captured images taken by a single camera) fusion scheme to obtain the first fused image corresponding to each camera, and adopting a multi-shot multi-frame fusion scheme to re-fuse the first fused images corresponding to multiple cameras, motion analysis of the moving target is additionally performed based on the preview image of the target scene for each camera, and target shooting parameters suitable for shooting the target scene (including but not limited to exposure time, camera movement direction, and magnification) are determined based on the analysis results. Thus, a better quality preliminary captured image (which may be called preliminary data) can be obtained, especially for scenes containing fast-moving targets, the blur and smear problems of the image can be improved as much as possible from the preliminary data collection stage, and the quality of the captured image obtained by the final fusion can be further improved.

[0105] Reference Figure 5 , Figure 5 It is a schematic diagram of the architecture of a photographing system for implementing the photographing method described in an embodiment of the present invention.

[0106] The photographing system may include two cameras, a main camera 50 and an auxiliary camera 51 (the number of cameras is only a non-limiting example), a first multi-frame capture module 52, a second multi-frame capture module 53, a preview image acquisition module 54, a motion analysis module 55, a strategy analysis module 56, a user interaction module 57, a depth information calculation module 58, a first multi-frame alignment and fusion module 59, a second multi-frame alignment and fusion module 510, and a multi-camera compensation fusion module 511.

[0107] Specifically, the first multi-frame capture module 52 and the second multi-frame capture module 53 are used to control the main camera 50 and the auxiliary camera 51 to use target shooting parameters to synchronously perform multi-frame shooting for the same target scene, and obtain multiple frames of preliminary captured images corresponding to the main camera 50 and multiple frames of preliminary captured images corresponding to the auxiliary camera 51.

[0108] The main camera may be a camera with higher shooting quality, and the auxiliary camera may be a camera with lower shooting quality. It should be noted that the number of cameras included in the camera system is not limited to two.

[0109] The preview image acquisition module 54 is used to acquire multiple frames of preview images of the target scene from the main camera 50 in the preview mode.

[0110] The motion analysis module 55 is used to perform motion analysis on the moving targets in the multiple frames of preview images to determine the motion information of the moving targets.

[0111] The strategy analysis module 56 is used to determine the target shooting parameters based on the motion information.

[0112] The user interaction module 57 is used to display the target shooting parameters on a display screen (or user interaction screen) of the terminal device, so that the user can choose whether to apply the target shooting parameters to the main camera 50 and the auxiliary camera 51 according to actual needs.

[0113] The depth information calculation module 58 is used to determine the depth map of the main camera 50 and the auxiliary camera 51 relative to the target scene according to multiple groups of preliminary captured images synchronously taken by the main camera 50 and the auxiliary camera 51, wherein each pixel position of the depth map has a respective depth value, wherein each group of preliminary captured images includes two frames of preliminary captured images obtained by synchronously taking the main camera 50 and the auxiliary camera 51.

[0114] The first multi-frame alignment and fusion module 59 is used to align the multiple frames of preliminary captured images corresponding to the main camera 50, and then perform image fusion processing on the multiple frames of preliminary captured images after the alignment processing to obtain a first fused image corresponding to the main camera 50; further, the first alignment information provided by the motion analysis module 55 can also be used for alignment processing. The first alignment information specifically refers to the image area information (for example, the position, area, etc. of the image area) to which the moving target belongs determined by the motion analysis module 55 on each frame of the preliminary captured image from the first multi-frame capture module 52.

[0115] The second multi-frame alignment module 510 is used to align the multiple frames of preliminary captured images corresponding to the auxiliary camera 51, and then perform image fusion processing on the multiple frames of preliminary captured images after the alignment processing to obtain the first fused image corresponding to the auxiliary camera 51; further, the second alignment information provided by the motion analysis module 55 can also be used for alignment processing, and the second alignment information specifically refers to the image area information (for example, the position, area, etc. of the image area) to which the moving target belongs determined by the motion analysis module 55 on each frame of the preliminary captured image from the second multi-frame capture module 53.

[0116] Regarding the specific implementation scheme for performing alignment processing using the first alignment information and the second alignment information, please refer to the previous description of the related scheme for extracting feature points from image areas other than the image area to which the moving target belongs, and performing alignment processing based on the extracted feature point set, which will not be repeated here.

[0117] The multi-shot compensation fusion module 511 performs multi-shot multi-frame fusion processing on the two frames of first fused images output by the first multi-frame alignment fusion module 59 and the second multi-frame alignment module 510 based on the depth map determined by the depth information calculation module 58, and outputs the fusion result (i.e., the final captured image).

[0118] For more information on the implementation, principles, technical effects, etc. of each module of the camera system, please refer to the previous article and Figures 1 to 4 The relevant description of any embodiment will not be repeated here.

[0119] Reference Figure 6 , Figure 6 1 is a schematic diagram of the structure of a photographing device in an embodiment of the present invention. The photographing device may include:

[0120] A first fusion module 61 is used to use multiple cameras to synchronously shoot multiple frames of the same target scene to obtain multiple frames of preliminary shooting images corresponding to each camera, and perform image fusion on the multiple frames of preliminary shooting images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras;

[0121] A depth map determining module 62 is used to determine, for each camera group, a depth map of the camera group relative to the target scene according to a plurality of groups of preliminary captured images synchronously captured by the cameras group, wherein each pixel position of the depth map has a respective depth value, wherein each group of preliminary captured images includes at least two frames of preliminary captured images synchronously captured by each camera group;

[0122] a fusion weight determination module 63, configured to determine, for each pixel position of the depth map of the group of cameras, a depth value difference between the pixel position and surrounding pixel positions, and determine, based at least on the depth value difference, a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fused image corresponding to the group of cameras, wherein the greater the depth value difference, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position;

[0123] The multi-camera fusion module 64 is used to perform a weighted operation on the pixel value of each pixel of each frame of the first fused image corresponding to each camera group by using a plurality of first fusion weights corresponding to each pixel belonging to each pixel position, so as to obtain a second fused image corresponding to the camera group;

[0124] The second fusion module 65 is used to fuse the multiple frames of second fused images corresponding to each group of cameras to obtain a final captured image.

[0125] For the principle, specific implementation and beneficial effects of the camera device, please refer to the previous article and Figures 1 to 4 The relevant description about the photographing method shown in any embodiment will not be repeated here.

[0126] The embodiment of the present invention further provides a storage medium, such as a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor. Figures 1 to 4 The steps of the photographing method shown in any embodiment. The computer-readable storage medium may include a non-volatile memory or a non-transitory memory, and may also include an optical disk, a mechanical hard disk, a solid-state hard disk, etc.

[0127] Specifically, in the embodiment of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0128] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0129] The embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the above-mentioned Figures 1 to 4The steps of the photographing method shown in any embodiment.

[0130] The embodiment of the present invention also provides a computer program product, including a computer program, wherein the computer program is executed by a processor Figures 1 to 4 The steps of the photographing method shown in any embodiment.

[0131] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship.

[0132] The "plurality" appearing in the embodiments of the present application refers to two or more.

[0133] The first, second, etc. descriptions appearing in the embodiments of the present application are only used for illustration and distinction of the description objects. There is no order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0134] It should be pointed out that the serial numbers of the steps in this embodiment do not limit the execution order of the steps.

[0135] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A photographing method, characterized in that: include: Using multiple cameras to synchronously shoot multiple frames of the same target scene to obtain multiple frames of preliminary shooting images corresponding to each camera, and performing image fusion on the multiple frames of preliminary shooting images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras; For each group of cameras, a depth map of the group of cameras relative to the target scene is determined based on multiple groups of preliminary captured images synchronously captured by the group of cameras, wherein each pixel position of the depth map has a respective depth value, wherein each group of preliminary captured images includes at least two frames of preliminary captured images obtained by synchronously capturing each camera in the group; For each pixel position of the depth map of the group of cameras, determine a depth value difference between the pixel position and surrounding pixel positions, and determine a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fusion image corresponding to the group of cameras based at least on the depth value difference, wherein the greater the depth value difference, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position; For each frame of the first fused image corresponding to each group of cameras, a plurality of first fusion weights corresponding to each pixel at each pixel position are used to perform a weighted operation on the pixel value of each pixel, so as to obtain a second fused image corresponding to the group of cameras; The multiple frames of second fused images corresponding to each group of cameras are fused to obtain a final captured image.

2. The method according to claim 1, characterized in that: The multiple cameras synchronously perform multi-frame shooting for the target scene using target shooting parameters; Before using multiple cameras to synchronously shoot multiple frames of the same target scene, the method further includes: For at least one camera among the multiple cameras, determining multiple frames of preview images of the camera for the target scene in a preview mode, and performing motion analysis on a moving target in the multiple frames of preview images to determine motion information of the moving target; Performing a weighted operation on the motion information obtained by the at least one camera to obtain weighted motion information; Based on the weighted motion information, the target shooting parameter is determined.

3. The method according to claim 2, characterized in that The motion information includes one or more of the motion speed, the motion direction, and the area of ​​the image region to which the moving target belongs, and the target shooting parameters include one or more of the exposure time, the camera moving direction, and the magnification; Determining the target shooting parameters based on the weighted motion information includes: Determine the exposure time corresponding to the movement speed according to the movement speed and a preset speed-exposure time mapping relationship, wherein the faster the movement speed, the shorter the corresponding exposure time; and / or, Using the movement direction as the camera movement direction; and / or, The magnification ratio corresponding to the area of ​​the image region to which the moving target belongs is determined according to the area of ​​the image region to which the moving target belongs and a preset area-magnification ratio mapping relationship, wherein the larger the area of ​​the image region to which the moving target belongs, the smaller the corresponding magnification ratio.

4. The method according to claim 1, characterized in that: The determining, at least based on the depth value difference, a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fusion image corresponding to the group of cameras includes: For each pixel belonging to the pixel position, determine a candidate indicator difference of each pixel, wherein the candidate indicator difference is selected from one or more of the following: brightness value difference, contrast difference, and texture difference; Performing a weighted operation on the depth value difference and the differences of various candidate indicators to obtain a weighted difference, wherein the depth value difference has the largest weight in the weighted operation process; A plurality of first fusion weights corresponding to the pixels belonging to the pixel position are determined based on the weighted difference, and the greater the weighted difference is, the greater the difference between the plurality of first fusion weights is.

5. The method according to claim 1 or 4, characterized in that: When there are differences between multiple first fusion weights corresponding to pixels belonging to the same pixel position in each frame of the first fusion image corresponding to each group of cameras, the multiple first fusion weights satisfy: the higher the image quality of the first fusion image to which the pixel belongs, the greater the first fusion weight corresponding to the pixel.

6. The method according to claim 1, characterized in that Determining, for each pixel position of the depth map of the group of cameras, a depth value difference between the pixel position and surrounding pixel positions, comprises: For each pixel position of the depth map of the group of cameras, respectively determine the depth difference between the depth value of the pixel position and the depth values ​​of multiple pixel positions within a preset area around the pixel position, thereby obtaining multiple depth differences corresponding to the pixel position; A mean square error of the plurality of depth differences is calculated, and the mean square error is used as the depth value difference.

7. The method according to claim 1, characterized in that Performing image fusion on the multiple frames of preliminary captured images to obtain a first fused image corresponding to the camera includes: Determine a preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, and record it as a reference frame image; Perform one or more rounds of fusion operations based on the reference frame image and the remaining preliminary captured images, and use the fusion result of the last round as the first fused image corresponding to the camera; In each round of fusion, the fusion result of the previous round and one frame of the remaining preliminary captured images that do not participate in the image fusion are fused to obtain the fusion result of the current round; The first round of fusion result is obtained by fusing the reference frame image and a frame of preliminary captured image other than the reference frame image.

8. The method according to claim 7, characterized in that In each round of fusion, the fusion result of the previous round is recorded as the first frame to be fused, and one frame of the remaining preliminary captured images that does not participate in image fusion is recorded as the second frame to be fused; In each round of fusion, the fusion result of the previous round and one frame of the remaining preliminary captured images that do not participate in the image fusion are fused to obtain the fusion result of the current round, including: For each pair of pixels in the same position in the first frame to be fused and the second frame to be fused, determining a difference in pixel values ​​of the pair of pixels; Determining a second fusion weight corresponding to the pair of pixels based on the pixel value difference, and performing a weighted operation on the pixel values ​​of the pair of pixels using the second fusion weight to obtain a fused pixel value, wherein the greater the pixel value difference is, the greater the difference between the second fusion weights corresponding to the pair of pixels is; The fused pixel values ​​of each pair of pixels at the same position in the first frame to be fused and the second frame to be fused are used as the fusion result of the current round.

9. The method according to claim 1, characterized in that: Before performing image fusion on the multiple frames of preliminary captured images, the method further includes: Determine a preliminary captured image with the highest image quality from the multiple frames of preliminary captured images, record it as a reference frame image, and record preliminary captured images other than the reference frame image as non-reference frame images; For each frame of preliminary captured image, determine the image region to which the moving target in the frame of preliminary captured image belongs, and extract feature points from image regions other than the image region to obtain a feature point set for the frame of preliminary captured image; For each non-reference frame image, an alignment matrix is ​​determined based on the feature point set of the reference frame image and the feature point set of the reference frame image, and the reference frame image is aligned with the reference frame image using the alignment matrix.

10. The method according to claim 1, characterized in that For each group of cameras, determining a depth map of the group of cameras relative to the target scene according to multiple groups of preliminary images synchronously captured by the group of cameras includes: For each camera group, a group of preliminary images with the best image quality is selected from multiple groups of preliminary images synchronously taken by the cameras; Determine a depth map of the group of cameras relative to the target scene based on the group of preliminary captured images with the best image quality.

11. A photographing device, characterized in that: include: A first fusion module is used to use multiple cameras to synchronously shoot multiple frames for the same target scene to obtain multiple frames of preliminary shooting images corresponding to each camera, and perform image fusion on the multiple frames of preliminary shooting images to obtain a first fused image corresponding to the camera, wherein the multiple cameras are divided into at least one group of cameras, and each group has at least two cameras; A depth map determination module is used to determine, for each camera group, a depth map of the camera group relative to the target scene based on multiple groups of preliminary captured images synchronously captured by the camera group, wherein each pixel position of the depth map has a respective depth value, wherein each group of preliminary captured images includes at least two frames of preliminary captured images synchronously captured by each camera in the group; a fusion weight determination module, configured to determine, for each pixel position of the depth map of the group of cameras, a depth value difference between the pixel position and surrounding pixel positions, and determine, based at least on the depth value difference, a plurality of first fusion weights corresponding to each pixel belonging to the pixel position in each frame of the first fused image corresponding to the group of cameras, wherein the greater the depth value difference, the greater the difference between the plurality of first fusion weights corresponding to each pixel belonging to the pixel position; A multi-camera fusion module is used to perform a weighted operation on the pixel value of each pixel of each frame of the first fused image corresponding to each camera group by using a plurality of first fusion weights corresponding to each pixel belonging to each pixel position, so as to obtain a second fused image corresponding to the camera group; The second fusion module is used to fuse multiple frames of second fused images corresponding to each group of cameras to obtain a final captured image.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the photographing method according to any one of claims 1 to 10 are executed.

13. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the photographing method according to any one of claims 1 to 10 are performed.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the photographing method according to any one of claims 1 to 10 are executed.

Images