Depth information generation method, image blurring method and video blurring method

Abstract

The embodiments of the present application disclose a depth information generation method, an image blurring method, and a video blurring method. The embodiments of the method include: acquiring a monocular image, and determining the first monocular depth information corresponding to the monocular image; determining the first binocular depth information corresponding to the monocular image based on a predetermined depth information conversion parameter and the first monocular depth information; the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, and the monocular image and the first image are acquired by a camera located at the same position to a focus object at the same position. This implementation takes into account both the generation efficiency and accuracy of the depth information.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and specifically to a depth information generating method, an image blurring method, and a video blurring method. Background Art

[0002] The depth information of an image can be understood as the distance from the image collector to each point in the scene corresponding to the image. It can usually be represented in the form of a depth map, vector, matrix, etc.

[0003] In the prior art, the depth information of an image needs to be calculated through binocular images or estimated through monocular images. Calculating depth information through binocular images requires a complex data processing process, which has low computational efficiency and is not suitable for scenes with high real-time requirements such as video blurring. The depth information estimated through monocular images is not real depth information, so the accuracy is low. Summary of the invention

[0004] The embodiments of the present application propose a depth information generation method, an image blurring method and a video blurring method to solve the technical problem in the prior art that it is impossible to take into account both the generation efficiency and accuracy of image depth information.

[0005] In a first aspect, an embodiment of the present application provides a depth information generating method, the method comprising: acquiring a monocular image, and determining first monocular depth information corresponding to the monocular image; determining first binocular depth information corresponding to the monocular image based on a predetermined depth information conversion parameter and the first monocular depth information; the depth information conversion parameter is determined based on second binocular depth information corresponding to a pre-acquired binocular image and second monocular depth information corresponding to a first image in the binocular image, the monocular image and the first image being acquired by a camera located at the same position with respect to a focus object at the same position.

[0006] In a second aspect, an embodiment of the present application provides an image blurring method, the method comprising: obtaining an image to be processed, and using the depth information generation method described in the first aspect to determine first binocular depth information corresponding to the image to be processed; based on the first binocular depth information, blurring the image to be processed to obtain a blurred image.

[0007] In a third aspect, an embodiment of the present application provides a video blurring method, the method comprising: obtaining a sequence of images to be processed, and using the depth information generation method described in the first aspect to determine the first binocular depth information corresponding to each image to be processed in the sequence of images to be processed; based on each of the first binocular depth information, blurring the sequence of images to be processed to obtain a blurred video.

[0008] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0011] The depth information generation method, image blurring method and video blurring method provided in the embodiments of the present application first obtain a monocular image and determine the first monocular depth information corresponding to the monocular image; then determine the first binocular depth information corresponding to the monocular image based on the predetermined depth information conversion parameter and the first monocular depth information, the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, the monocular image and the first image are acquired by the camera located at the same position to the focus object at the same position. Thus, after obtaining the depth information conversion parameter through the binocular image, only the monocular image can be obtained, and the monocular depth information with lower accuracy can be fitted into binocular depth information with higher accuracy through the depth information conversion parameter, and accurate binocular depth information can be obtained without complex data processing of the binocular image, thereby improving the generation efficiency of the depth information while ensuring the accuracy of the depth information. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0013] Figure 1 is a flow chart of an embodiment of a method for generating depth information according to the present application;

[0014] Figure 2 is a flowchart of a second binocular depth information determination process according to the depth information generation method of the present application;

[0015] Figure 3 is a flow chart of an embodiment of an image blurring method according to the present application;

[0016] Figure 4 is a flow chart of an embodiment of a video blurring method according to the present application;

[0017] Figure 5 is a schematic structural diagram of an embodiment of a depth information generating device according to the present application;

[0018] Figure 6 is a structural schematic diagram of an embodiment of an image blurring device according to the present application;

[0019] Figure 7 is a structural schematic diagram of an embodiment of a video blurring device according to the present application;

[0020] Figure 8 It is a schematic diagram of the structure of a computer system for implementing an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In recent years, research on computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence has made important progress. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving many types of technologies such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks, etc. Computer vision, as an important branch of artificial intelligence, specifically allows machines to recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, automatic driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, and smart temperature measurement.

[0024] In the field of image processing, the depth information of the image is usually needed. For example, the depth can be used to perform image blur processing, video blur processing, and estimate the distance in the autonomous driving scenario. In related technologies, the depth information of the image needs to be calculated through binocular images. First, stereo correction is performed to obtain a corrected binocular image; then, stereo matching is performed using the corrected binocular image to obtain disparity information; finally, the depth information is determined based on the disparity information. This method requires data processing processes such as stereo matching and stereo correction, and has low computational efficiency. It is not easy to apply to scenes with high real-time requirements such as video blur. In related technologies, depth information can also be estimated through monocular images. This depth information is not the true depth of the image and has low accuracy. The embodiments of the present application can take into account both the generation efficiency and accuracy of depth information.

[0025] Please refer to Figure 1, which shows a process 100 of an embodiment of a depth information generation method according to the present application. The execution subject of the depth information generation method can be various electronic devices with shooting functions. For example, it can include but is not limited to: smart phones, tablet computers, laptop computers, car computers, handheld computers, desktop computers, set-top boxes, smart TVs, cameras, wearable devices, etc.

[0026] The depth information generation method comprises the following steps:

[0027] Step 101: Acquire a monocular image and determine first monocular depth information corresponding to the monocular image.

[0028] In this embodiment, the execution subject of the depth information generation method may be equipped with a camera, and a monocular image may be captured by the camera. The camera may be an independent single camera or one of the binocular cameras. The execution subject may estimate the first monocular depth information corresponding to the monocular image using monocular vision depth perception technology.

[0029] In some optional implementations, a pre-trained depth information detection model may be stored in the above-mentioned execution subject. The above-mentioned execution subject may input the monocular image into the pre-trained depth information detection model to obtain the first monocular depth information. Among them, the depth information detection model may be pre-trained by a machine learning method (such as a supervised learning method). The basic model used to train the depth information detection model may be a convolutional neural network, etc. During training, one of the binocular images (such as the left image) may be first input into the model as a sample image. Then, the model loss value is determined based on the depth information predicted by the model and the binocular depth information of the binocular image (that is, the real depth information, which can be used as a label). The model parameters are then updated based on the loss value. The above steps are iterated until the end condition is met. The end condition may be that the loss value converges, or the number of iterations reaches a preset number, the accuracy of the depth information output by the model is greater than the target value, etc., which are not specifically limited here.

[0030] Step 102: Determine first binocular depth information corresponding to the monocular image based on a predetermined depth information conversion parameter and the first monocular depth information.

[0031] In this embodiment, the execution entity may store depth information conversion parameters. The depth information conversion parameters may be parameters in a preset conversion relationship. The conversion relationship may characterize the conversion relationship between binocular depth information and monocular depth information. The conversion relationship may be preset as needed. The execution entity may substitute the value of the depth information conversion parameter and the first monocular depth information into the conversion relationship, thereby solving the conversion relationship to obtain the first binocular depth information of the monocular image. The first binocular depth information is real depth information, which is more accurate than the first monocular depth information.

[0032] In this embodiment, the depth information conversion parameter can be determined based on the binocular depth information corresponding to the pre-acquired binocular image (which may be referred to as the second binocular depth information) and the monocular depth information corresponding to the first image in the binocular image (which may be referred to as the second monocular depth information). Among them, the above-mentioned binocular image can be two images taken by a binocular camera at the same time of the same focus object. The first image is one of the images in the binocular image. The above-mentioned monocular image and the first image can be acquired by a camera located at the same position for the focus object at the same position, thereby ensuring the accuracy of the calculation of the first binocular depth information. The cameras located at the same position can be the same camera (such as a specified camera in the binocular camera) or different cameras, which is not limited here.

[0033] As an example, the above conversion relationship can be: Dt=a×Ds+b. Wherein, Dt is the binocular depth information, Ds is the monocular depth information, and Dt and Ds can be expressed as matrices. a and b are depth information conversion parameters. a and b can be two constants, that is, when the relative position of the camera and the focus object does not change, a and b are fixed values. At this time, a and b can be used to convert the binocular depth information to the monocular depth information. The above execution entity can substitute the second binocular depth information and the second monocular depth information of the acquired binocular image into the above relationship, solve the above conversion relationship, and obtain the values ​​of a and b. It should be noted that in addition to the above examples, other conversion relationships can be set as needed, and the number of depth information conversion parameters in the conversion relationship may not be limited to two, and is not specifically limited here.

[0034] In this embodiment, the second binocular depth information can be determined by the disparity information of the two images in the binocular image. As an example, the binocular image can be firstly stereo corrected to obtain a corrected binocular image. After that, the corrected binocular image can be used for stereo matching to obtain disparity information. Finally, the conversion relationship between disparity and depth can be used to determine the second binocular depth information based on the disparity information. Among them, stereo correction can be performed using a stereo correction algorithm such as Bouguet. Stereo matching can be performed using a global method or a local method. The global method may include but is not limited to the DeepPruner algorithm, etc., and the local method may include but is not limited to the NCC (Normalized Cross Correlation) algorithm, the SSD (Sum of Squared Defferences, the sum of squared differences) algorithm, the SAD (Sum of Absolute Defferences, the sum of absolute differences) algorithm, etc., which are not limited here. The method for determining the second monocular depth information can refer to the method for determining the first monocular depth information, which will not be repeated here.

[0035] The method provided by the above-mentioned embodiment of the present application first acquires a monocular image and determines the first monocular depth information corresponding to the monocular image; then, based on the predetermined depth information conversion parameter and the first monocular depth information, determines the first binocular depth information corresponding to the monocular image, the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, the monocular image and the first image are acquired by the camera located at the same position to the focus object at the same position. Thus, after obtaining the depth information conversion parameter through the binocular image, only the monocular image can be acquired, and the monocular depth information with lower accuracy can be fitted into the binocular depth information with higher accuracy through the depth information conversion parameter, and accurate binocular depth information can be obtained without complex data processing of the binocular image, thereby improving the generation efficiency of the depth information while ensuring the accuracy of the depth information.

[0036] In some optional embodiments, the conversion equation may include a first depth information conversion parameter (such as parameter a in the conversion equation in the above example) and a second depth information conversion parameter (such as parameter b in the conversion equation in the above example). The above execution subject may first convert the conversion equation into an overdetermined equation, and then substitute the first binocular depth information and the first monocular depth information into the overdetermined equation, and solve the overdetermined equation to obtain a two-dimensional parameter vector. For example, the overdetermined equation may be expressed as A×D s =D t . Where A = [a, b], which is the two-dimensional parameter vector to be solved. In practice, the direct method can be used to solve A. The solution formula is: in, Yes D s The transposed matrix of . Thus, A can be obtained. Where a=A[0], b=A[1]. By solving the overdetermined equation, the depth information conversion parameters can be conveniently and quickly fitted.

[0037] In some optional embodiments, the execution subject may obtain the first frame image captured by the binocular camera when starting photo or video shooting to obtain a binocular image, and then determine the second binocular depth information and the second monocular depth information corresponding to the binocular image, and determine the depth information conversion parameters based on the second binocular depth information and the second monocular depth information.

[0038] In some optional embodiments, the above-mentioned execution subject can obtain the first frame image captured by the binocular camera after the relative position changes when the relative position between the binocular camera and the focus object changes during the photo or video shooting process to obtain the binocular image. Then, the second binocular depth information and the second monocular depth information corresponding to the binocular image can be determined, and the depth information conversion parameters can be determined based on the second binocular depth information and the second monocular depth information.

[0039] Among them, the relative position of the binocular camera and the focus object can change in many cases. For example, when the position of the binocular camera moves and the position of the focus object does not move, the above relative position can be changed; when the position of the binocular camera does not move and the position of the focus object moves, the above relative position can be changed; when the binocular camera and the focus object both move but the movement is not synchronized, the above relative position can be changed; when the focus object changes (such as switching the focus object from A to B), the above relative position can be changed; when refocusing on the same shooting object, since the relative position may also change, the above relative position can also be changed. Thus, during the shooting process, whenever the relative position of the binocular camera and the focus object changes, the binocular image captured by the binocular camera can be obtained, and the second binocular depth information and the second monocular depth information of the binocular image can be determined, and then the depth information conversion parameters are updated based on the second binocular depth information and the second monocular depth information. Thus, the dynamic update of the depth information conversion parameters can be achieved to ensure the accuracy of the depth information conversion parameters.

[0040] In some optional embodiments, the above-mentioned execution subject may obtain a monocular image captured by a monocular camera when the current relative position of the monocular camera and the focus object does not change compared to the target relative position during the photo or video shooting process. Wherein, the target relative position is the relative position of the monocular camera and the focus object at the time of the above-mentioned binocular image acquisition. The monocular camera may be one of the binocular cameras that capture the binocular image. In this way, the depth information conversion parameters can be avoided from changing, thereby avoiding inaccurate depth information calculation. It should be noted that if the current relative position of the monocular camera and the focus object is different from the target relative position, it is necessary to re-acquire the binocular image, and based on the binocular depth information corresponding to the re-acquired binocular image and the monocular depth information corresponding to the first image, the depth information conversion parameters are re-determined to ensure the accuracy of the depth information calculation. This implementation method can generate accurate depth information in real time during the photo or video shooting process, so as to facilitate the processing of photos or videos based on the depth information, thereby improving the processing efficiency of photos or videos.

[0041] In some optional embodiments, the binocular image further includes a second image, and the second image and the first image are obtained by photographing the focus object at different angles. Figure 2 The second binocular depth information is determined by the following sub-steps S11 to S14:

[0042] Sub-step S11, determining feature points in the first image.

[0043] Specifically, the execution subject may perform feature point detection on the first image to determine the feature points in the first image. After determining the feature points, the execution subject may obtain the coordinates of each feature point in the first image. Here, feature point detection may be performed using the ORB (Oriented Fast and Rotated Brief) algorithm. The determined feature points are ORB feature points. Among them, Fast and Brief are feature detection algorithms and vector creation algorithms, respectively.

[0044] When detecting feature points using the ORB algorithm, you first look for special areas in the first image, called keypoints. Keypoints are small areas of the image that stand out, such as corners. Then, you calculate a corresponding feature vector for each keypoint. The feature vector created by the ORB algorithm contains only 1s and 0s and is called a binary feature vector. The order of 1s and 0s varies depending on the specific keypoint and the area of ​​pixels around it. This vector represents the intensity pattern around the keypoint.

[0045] Sub-step S12, determining corresponding points of each feature point in the second image.

[0046] Specifically, for each feature point in the first image, its corresponding point in the second image can be determined. In practice, the LK (Lucas–Kanade) optical flow algorithm can be used to find the corresponding points of each feature point on the second image. After determining the corresponding points, the above execution entity can obtain the coordinates of each feature point in the second image.

[0047] In practice, optical flow is a motion pattern, which refers to the apparent movement of an object, surface, or edge between an observer (such as eyes, camera, etc.) and the background under a certain viewing angle. Optical flow studies the use of temporal changes and correlations in pixel intensities in an image sequence to determine the movement of each pixel position, that is, to study the relationship between the temporal changes in image grayscale and the structure of objects in the scene and their movement. The LK optical flow algorithm is a two-frame differential optical flow estimation algorithm, and feature point matching can be performed using the LK optical flow algorithm. Therefore, the first image and the second image can be regarded as two frames, and the corresponding points of each feature point in the first image can be matched on the second image using the LK optical flow algorithm. It should be noted that in addition to using the LK optical flow algorithm, other optical flow algorithms can also be used to determine the corresponding points of each feature point in the second image, which is not specifically limited here.

[0048] Sub-step S13, determining the disparity value of each feature point based on the coordinates of each feature point in the first image and the coordinates of each corresponding point in the second image.

[0049] Here, for each feature point, the disparity relationship is: d i =x li -x ri Among them, d i is the disparity value of feature point i, x li is the row coordinate of feature point i in the first image, x ri is the row coordinate of feature point i on the second image.

[0050] Sub-step S14: determining first binocular depth information of the binocular image based on the disparity value.

[0051] Here, based on the disparity value, the first binocular depth information of the binocular image can be determined by a conversion formula between disparity and depth. As an example, the conversion relationship between disparity and depth can be: depth i =(f×baseline) / d i Among them, depth irepresents the depth value of feature point i in the first image, f represents the normalized focal length, which can be obtained from the internal reference, and baseline is the distance between the optical centers of the two cameras of the binocular camera, which is called the baseline distance. Through the above conversion formula, the depth value of each feature point in the first image can be determined. Based on the depth value of each feature point, the depth value of other points in the first image can be determined by difference or other methods. Thus, the depth values ​​of each point in the first image can be summarized to obtain the depth information of the binocular image, which can be called the first binocular depth information.

[0052] Since the feature points are sparse points in the first image, the first binocular depth information can be fitted based on the sparse points, and then the binocular depth information of the monocular image can be obtained based on the depth information conversion parameters. While ensuring the accuracy of the depth information, the amount of data calculation is further reduced, the efficiency of generating depth information is improved, and the power consumption of the device is reduced.

[0053] Further references Figure 3 , which shows a process 300 of an embodiment of the image blur generation method according to the present application. The execution subject of the image blur generation method can be various electronic devices with shooting functions. For example, it can include but is not limited to: smart phones, tablet computers, laptop computers, car computers, handheld computers, desktop computers, set-top boxes, smart TVs, cameras, wearable devices, etc.

[0054] The image blur generation method comprises the following steps:

[0055] Step 301 : obtaining an image to be processed, and determining first binocular depth information corresponding to the image to be processed by using a depth information generation method.

[0056] In this embodiment, the image to be processed may be the monocular image in the above embodiment, which may be acquired by a monocular camera or one of the binocular cameras. The depth information generation method may refer to the description in the above embodiment, which will not be repeated here.

[0057] Step 302: based on the first binocular depth information, blur the image to be processed to obtain a blurred image.

[0058] In this embodiment, blur processing is an image processing method that makes the focused object clear and the background area blurred. The image obtained after blur processing is performed on the image to be processed is a blurred image.

[0059] The method provided by the above-mentioned embodiment of the present application can improve the image blurring efficiency and reduce the power consumption of the device in the image blurring preview scenario because accurate depth information of the image can be obtained without complex binocular image data processing.

[0060] Further references Figure 4 , which shows a process 400 of an embodiment of a method for generating video blur according to the present application. The execution subject of the method for generating video blur can be various electronic devices with shooting functions. For example, it can include but is not limited to: smart phones, tablet computers, laptop computers, car computers, handheld computers, desktop computers, set-top boxes, smart TVs, cameras, wearable devices, etc.

[0061] The video blur generation method comprises the following steps:

[0062] Step 401 : obtaining a sequence of images to be processed, and determining first binocular depth information corresponding to each image to be processed in the sequence of images to be processed by a depth information generation method.

[0063] In this embodiment, the image to be processed in the image sequence to be processed can be a video frame in the video to be processed, which can be used as the monocular image in the above embodiment, and is collected by a monocular camera or one of the binocular cameras. The depth information generation method can refer to the description in the above embodiment, which will not be repeated here.

[0064] Step 402: based on each first binocular depth information, blur the image sequence to be processed to obtain a blurred video.

[0065] In this embodiment, blur processing is an image processing method that makes the focused object clear and the background area blurred. The image obtained after blur processing is performed on the image to be processed is a blurred image.

[0066] The method provided by the above-mentioned embodiment of the present application can improve the frame rate of the video blur preview and reduce the power consumption of the device for the video blur preview scenario because accurate depth information of each image can be obtained without complex binocular image data processing.

[0067] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a depth information generating device, which can be specifically applied to various electronic devices.

[0068] like Figure 5As shown, the depth information generating device 500 of this embodiment includes: an acquisition unit 501, used to acquire a monocular image and determine the first monocular depth information corresponding to the monocular image; a determination unit 502, used to determine the first binocular depth information corresponding to the monocular image based on a predetermined depth information conversion parameter and the first monocular depth information; the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, and the monocular image and the first image are acquired by a camera located at the same position to a focus object at the same position.

[0069] In some optional implementations, the binocular image is acquired through the following steps: when starting photo or video shooting, the first frame image captured by the binocular camera is acquired to obtain the binocular image; or, during the photo or video shooting process, when the relative position between the binocular camera and the focused object changes, the first frame image captured by the binocular camera after the relative position changes is acquired to obtain the binocular image.

[0070] In some optional implementations, the acquisition unit 501 is further used to: during the photo or video shooting process, when the current relative position of the monocular camera and the focus object has not changed compared to the target relative position, acquire a monocular image captured by the monocular camera, the target relative position is the relative position of the monocular camera and the focus object at the moment of binocular image capture, and the monocular camera is one of the binocular cameras.

[0071] In some optional implementations, the binocular image also includes a second image, and the second image and the first image are obtained by photographing the focus object at different angles; the second binocular depth information is determined by the following steps: determining feature points in the first image; determining corresponding points of each feature point in the second image; determining a disparity value of each feature point based on the coordinates of each feature point in the first image and the coordinates of each corresponding point in the second image; and determining the second binocular depth information of the binocular image based on the disparity value.

[0072] In some optional implementations, the depth information conversion parameters are determined by the following steps: obtaining a preset conversion relationship, the conversion relationship is used to characterize the conversion relationship between monocular depth information and binocular depth information, the conversion relationship includes a first depth information conversion parameter and a second depth information conversion parameter; substituting the second binocular depth information and the second monocular depth information into the conversion relationship, solving the conversion relationship, and obtaining parameter values ​​of the first depth information conversion parameter and the second depth information conversion parameter.

[0073] The device provided by the above-mentioned embodiment of the present application first acquires a monocular image and determines the first monocular depth information corresponding to the monocular image; then, based on the predetermined depth information conversion parameter and the first monocular depth information, the first binocular depth information corresponding to the monocular image is determined, and the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, and the monocular image and the first image are acquired by the camera located at the same position to the focus object at the same position. Thus, after obtaining the depth information conversion parameter through the binocular image, only the monocular image can be acquired, and the monocular depth information with low accuracy can be fitted into the binocular depth information with high accuracy through the depth information conversion parameter, and accurate binocular depth information can be obtained without complex data processing of the binocular image, thereby improving the generation efficiency of the depth information while ensuring the accuracy of the depth information.

[0074] Further references Figure 6 As an implementation of the methods shown in the above figures, the present application provides an embodiment of an image blurring device, which can be specifically applied to various electronic devices.

[0075] like Figure 6 As shown, the image blurring device 600 of this embodiment includes: an acquisition unit 601, used to acquire the image to be processed, and determine the first binocular depth information corresponding to the image to be processed by using the depth information generation method in the above embodiment; a blurring processing unit 602, used to blur the image to be processed based on the first binocular depth information to obtain a blurred image.

[0076] The device provided by the above-mentioned embodiment of the present application can improve the image blurring efficiency and reduce the power consumption of the device in the image blurring preview scenario because accurate depth information of the image can be obtained without complex binocular image data processing.

[0077] Further references Figure 7 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a video blurring device, which can be specifically applied to various electronic devices.

[0078] like Figure 7 As shown, the video blurring device 700 of this embodiment includes: an acquisition unit 701, used to acquire a sequence of images to be processed, and to determine the first binocular depth information corresponding to each image to be processed in the sequence of images to be processed by using the depth information generation method in the above embodiment; a blurring processing unit 702, used to blur the sequence of images to be processed based on the above first binocular depth information to obtain a blurred video.

[0079] The device provided by the above-mentioned embodiment of the present application can improve the frame rate of the video blur preview and reduce the power consumption of the device for the video blur preview scene because it can obtain accurate depth information of each image without complex binocular image data processing.

[0080] An embodiment of the present application also provides an electronic device, including one or more processors and a storage device on which one or more programs are stored. When the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned depth information generation method.

[0081] Reference below Figure 8 , which shows a schematic structural diagram of an electronic device for implementing some embodiments of the present application. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0082] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0083] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a disk, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 8 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0084] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned depth information generation method when executed by a processor.

[0085] In particular, according to some embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of some embodiments of the present application are executed.

[0086] The embodiment of the present application further provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned depth information generating method is implemented.

[0087] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0088] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0089] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: acquires a monocular image and determines the first monocular depth information corresponding to the monocular image; determines the first binocular depth information corresponding to the monocular image based on the predetermined depth information conversion parameter and the first monocular depth information; the depth information conversion parameter is determined based on the second binocular depth information corresponding to the pre-acquired binocular image and the second monocular depth information corresponding to the first image in the binocular image, and the monocular image and the first image are acquired by a camera located at the same position to a focus object at the same position.

[0090] Computer program code for performing the operations of some embodiments of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++; and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, or may be connected to an external computer (e.g., through the Internet using an Internet service provider), including a local area network (LAN) or a wide area network (WAN).

[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The units described in some embodiments of the present application may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a first determination unit, a second determination unit, a selection unit, and a third determination unit. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] The above description is only some preferred embodiments of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with (but not limited to) the technical features with similar functions disclosed in the embodiments of the present application to form a technical solution.

Claims

1. A method for generating depth information, characterized in that: The method comprises: Acquire a monocular image, and determine first monocular depth information corresponding to the monocular image; Determine first binocular depth information corresponding to the monocular image based on a predetermined depth information conversion parameter and the first monocular depth information; The depth information conversion parameter is determined based on second binocular depth information corresponding to a pre-acquired binocular image and second monocular depth information corresponding to a first image in the binocular image, wherein the monocular image and the first image are acquired by a camera located at the same position and focusing on a focus object at the same position; The depth information conversion parameters are determined by the following steps: Acquire a preset conversion relationship, where the conversion relationship is used to characterize the conversion relationship between monocular depth information and binocular depth information, and the conversion relationship includes a first depth information conversion parameter and a second depth information conversion parameter; Substituting the second binocular depth information and the second monocular depth information into the conversion equation, solving the conversion equation, and obtaining parameter values ​​of the first depth information conversion parameter and the second depth information conversion parameter; The determining, based on a predetermined depth information conversion parameter and the first monocular depth information, first binocular depth information corresponding to the monocular image comprises: Substituting the value of the depth information conversion parameter and the first monocular depth information into the conversion equation, solving the conversion equation, and determining the first binocular depth information.

2. The method according to claim 1, characterized in that The binocular image is obtained by the following steps: When starting photo or video shooting, obtain the first frame image captured by the binocular camera to obtain a binocular image; or, During the process of taking a photo or video, when the relative position between the binocular camera and the focused object changes, the first frame image captured by the binocular camera after the relative position changes is acquired to obtain a binocular image.

3. The method according to claim 2, characterized in that The step of acquiring a monocular image comprises: During the photo or video shooting process, when the current relative position of the monocular camera and the focus object does not change compared to the target relative position, a monocular image captured by the monocular camera is obtained, and the target relative position is the relative position of the monocular camera and the focus object at the moment of binocular image capture, and the monocular camera is one of the binocular cameras.

4. The method according to claim 1, characterized in that: The binocular image also includes a second image, where the second image and the first image are obtained by photographing the focus object at different angles; The second binocular depth information is determined by the following steps: determining feature points in the first image; Determining corresponding points of each feature point in the second image; Determine a disparity value of each feature point based on the coordinates of each feature point in the first image and the coordinates of each corresponding point in the second image; Based on the disparity value, second binocular depth information of the binocular image is determined.

5. An image blurring method, characterized in that: The method comprises: Acquire an image to be processed, and determine first binocular depth information corresponding to the image to be processed using the depth information generation method according to any one of claims 1 to 4; Based on the first binocular depth information, the image to be processed is blurred to obtain a blurred image.

6. A video blurring method, characterized in that: The method comprises: Acquire a sequence of images to be processed, and determine first binocular depth information corresponding to each image to be processed in the sequence of images to be processed by using the depth information generation method according to any one of claims 1 to 4; Based on each of the first binocular depth information, the image sequence to be processed is blurred to obtain a blurred video.

7. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images