Automatic focusing method and electronic equipment

Abstract

The invention discloses an automatic focusing method and electronic equipment, and relates to the field of terminals, and the method can comprise the steps that the electronic equipment collects a first image and a second image through a camera; wherein the first image comprises the first depth of the target object, the second image comprises the second depth of the target object, and the collection time of the second image is later than the collection time of the first image. The electronic device may predict a third depth of the target object in the first target image based on the first depth and the second depth. Wherein the acquisition time of the first target image is after the acquisition time of the second image. Then, the electronic equipment determines a first focusing position of a lens in the camera based on the third depth; before the electronic equipment collects a first target image, a lens in a camera is moved to a first focusing position through a motor. The electronic device collects a first target image at a first focusing position through a camera.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to an automatic focusing method and electronic equipment. Background Art

[0002] With the continuous development of terminal technology, the functions of electronic devices are becoming increasingly diversified. Nowadays, many electronic devices on the market are equipped with cameras, which can realize the functions of taking pictures or videos. During the shooting process, if the electronic device wants to capture a clear target object through the camera, it is necessary to focus on the target object. When the target object is a moving object, the camera may be in a state of always focusing. During the shooting process of this dynamic scene, the camera may fail to focus successfully when focusing, resulting in the problem that the captured image is not clear enough and the film effect is not good. Summary of the invention

[0003] The present application provides an automatic focusing method and electronic device, which realizes that the electronic device pre-estimates the depth of the target object at a preset time point (i.e., the time point corresponding to the target image acquisition) through the collected N frames of images, and pushes the lens in the camera to the corresponding focus position through the motor before the preset time point. In this way, in a scene where the target object moves at high speed, when the preset time point is reached, the electronic device can reduce the gap between the focus area and focus depth of the lens and the actual position and depth of the target object, and the electronic device can successfully focus on the target object, thereby capturing a clear target object and improving the quality of the image capture effect.

[0004] In a first aspect, the present application provides an autofocus method, comprising: acquiring a first image and a second image through a camera. The first image includes a first depth of a target object, the second image includes a second depth of the target object, and the acquisition time of the second image is later than the acquisition time of the first image. Based on the first depth and the second depth, a third depth of the target object in the first target image is predicted. The acquisition time of the first target image is after the acquisition time of the second image. Based on the third depth, a first focus position of the lens in the camera is determined. Before acquiring the first target image, the lens in the camera is moved to the first focus position by a motor. The first target image is acquired by the camera at the first focus position.

[0005] In a possible implementation, the predicting a third depth of the target object in the first target image based on the first depth and the second depth specifically includes: fitting a depth prediction model based on the first depth and the second depth. The depth prediction model is used to determine the depth of the target object in the image based on the frame number of the image. Based on the depth prediction model and the frame number of the first target image, the third depth of the target object in the first target image is predicted.

[0006] In a possible implementation, determining a first focus position of the lens in the camera based on the third depth specifically includes: determining a first image distance corresponding to the third depth based on the third depth and a focal length of the lens in the camera. Determining the first focus position of the lens in the camera based on the first image distance.

[0007] In a possible implementation, the method further includes: acquiring a third image, the third image including a fourth depth of the target object. The acquisition time of the third image is later than the acquisition time of the second image. Based on the first depth, the second depth and the third depth, predicting a fifth depth of the target object in the second target image. The acquisition time of the second target image is later than the acquisition time of the first target image.

[0008] In a possible implementation, after the first target image is acquired by the camera at the first focus position, the method further includes: determining a second focus position of the lens in the camera based on the fifth depth. Before acquiring the second target image, moving the lens in the camera to the second focus position by a motor. Acquiring the second target image by the camera at the second focus position.

[0009] In a possible implementation, after the first target image is captured by the camera at the first focus position, the method further includes: when the difference between the fifth depth and the third depth is less than the first preset difference, the second target image is captured by the camera at the first focus position. When the difference between the fifth depth and the third depth is greater than the first preset difference, the second focus position of the lens in the camera is determined based on the fifth depth. Before capturing the second target image, the lens in the camera is moved to the second focus position by a motor. The second target image is captured by the camera at the second focus position.

[0010] In a possible implementation, before predicting the third depth of the target object in the first target image based on the first depth and the second depth, the method further includes: displaying a fourth image, receiving an operation acting on a first area on the fourth image, identifying the target object from the first area, detecting that the target object is located in a second area on the first image, and determining the depth of the second area in the first image as the first depth.

[0011] In a second aspect, the present application provides an electronic device, comprising: one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store a computer executable program, and when the one or more processors execute the computer executable program, the electronic device executes a method in any possible implementation of the first aspect.

[0012] In a third aspect, the present application provides a chip system, comprising a processing circuit and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processing circuit, and the processing circuit is used to run the code instructions so that the chip system executes a method as in any possible implementation of the first aspect above.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium, including a computer executable program. When the computer executable program is run on an electronic device, the electronic device executes a method in any possible implementation of the first aspect described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of a specific process of an auto-focusing method provided in an embodiment of the present application;

[0015] Figure 2A-2B A set of user interface schematic diagrams provided for embodiments of the present application;

[0016] Figure 3 A schematic diagram of a predicted depth model fitting provided in an embodiment of the present application;

[0017] Figure 4A A schematic diagram of depth data comparison provided in an embodiment of the present application;

[0018] Figure 4B A schematic diagram of multiple prediction depth models provided in an embodiment of the present application;

[0019] Figure 4C A schematic diagram of a tracking frame provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations including one or more of the listed striking ones. In the embodiments of the present application, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can expressly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0022] With the development of technology in the terminal field and the improvement of living standards, users' demand for electronic devices has increased, and the functional requirements for electronic devices have also become higher and higher. For example, in the process of shooting dynamic scenes, users often need to use electronic devices to clearly shoot the target object in motion through the camera. When the target object moves, the electronic device needs to focus on the target object (for example, autofocus / manual focus) in order to capture a clear image. Among them, focusing refers to the process in which the electronic device moves the lens in the camera through a motor, changes the distance from the imaging plane to the lens in the camera (ie, the image distance), and enables the target object to be clearly imaged on the imaging plane.

[0023] In some embodiments, when an electronic device performs automatic focusing on a moving target object, the electronic device collects and exposes information of the target object through an image sensor and a direct-time offight (TOF) camera to obtain the position and depth of the target object. Then, the automatic focus (AF) module can obtain and calculate the corresponding focus position of the lens in the camera based on the position and depth of the target object. Next, the AF module can drive the lens to move to the above-calculated focus position through a motor to focus on the target object.

[0024] However, due to various delays in the autofocus process (such as the delay of the sensor exposing the image, the delay of processing the depth obtained by the direct-time offight (DTOF) camera, the delay of the AF module calculation, and the delay of the motor movement, etc.), each focusing will have a delay of one or more frames (usually 2 frames), resulting in a certain difference between the camera's focus area and depth (i.e., the distance of the focus area from the camera) and the actual position and depth of the target object (i.e., the distance of the target object from the camera). In the scenario where the target object moves at high speed, the electronic device will probabilistically fail to focus successfully, resulting in the captured image being unclear and the film effect being poor.

[0025] Therefore, the present application provides an autofocus method, which includes: an electronic device can receive an operation 1 to start a camera. In response to the operation 1, the electronic device can collect N frames of images including a target object through the camera. Each frame of the N frames includes the depth of the target object. Then, the electronic device can predict the depth of the target object in the target image after the N frames of images (which can be called the predicted depth) based on the depth included in each frame of the N frames of images. Next, the electronic device can determine the focus position of the lens in the camera based on the predicted depth. Before collecting the target image, the electronic device can push the lens to the focus position through a motor. When the electronic device collects the target image, the lens at the focus position can accurately focus on the target object and obtain a clearly imaged target image.

[0026] Among them, the "clear imaging" described in the embodiment of the present application means that the light emitted from a certain point on the target object can converge on the same point or the adjacent area of ​​the point on the imaging plane after passing through the lens. The type of each frame of N frames of images can be an RGB-D image. The RGB-D image includes the following information of the target object: 1). When the target object is imaged in the image, the ratio of R, G, and B for each pixel. 2). The distance of the target object from the camera.

[0027] In an embodiment of the present application, the depth of each frame of N frames of images can be captured by a TOF camera or a binocular camera, and the embodiment of the present application is not limited thereto.

[0028] The “depth” mentioned in the embodiments of the present application refers to the distance from the location of an object in three-dimensional space to a camera.

[0029] In this implementation, the electronic device can pre-estimate the depth of the target object at a preset time point (i.e., the time point corresponding to the target image acquisition) through the collected N frames of images, and push the lens in the camera to the corresponding focus position through the motor before the preset time point. In this way, in a scene where the target object moves at high speed, when the preset time point is reached, the electronic device can reduce the gap between the focus area and focus depth of the lens and the actual position and depth of the target object, and the electronic device can successfully focus on the target object, thereby capturing a clear target object and improving the quality of the image capture effect.

[0030] Figure 1 The specific process of an automatic focusing method provided in an embodiment of the present application is exemplified.

[0031] like Figure 1 As shown, the specific process of the automatic focusing method may include:

[0032] S101: The electronic device receives an operation 1 to start a camera.

[0033] Among them, the operation 1 of starting the camera can be: a pressing operation on a preset button on the electronic device, a touch operation on a specified icon / control, etc., and this application does not impose any restrictions on this.

[0034] For example, Figure 2A As shown, the electronic device can display a desktop 200. The desktop 200 can display one or more application icons. Among them, the one or more application icons can include a weather application icon, a stock application icon, a calculator application icon, a settings application icon, an email application icon, a video application icon, a calendar application icon, and a gallery application icon, etc. Optionally, the desktop 200 can also display a status bar, a page indicator, and a tray icon area. Among them, the status bar can include one or more signal strength indicators of a mobile communication signal (also known as a cellular signal), a signal strength indicator of a wireless fidelity (Wi-Fi) signal, a battery status indicator, a time indicator, and the like. The page indicator can be used to indicate the positional relationship between the currently displayed page and other pages. The tray icon area includes a plurality of tray icons (such as a dialing application icon, a message application icon, a contact application icon, and a camera application icon 201, etc.), and the tray icon remains displayed when the page is switched. The above page may also include a plurality of application icons and a page indicator, and the page indicator may not be part of the page but exist independently. The above tray icon is also optional, and the embodiment of the present application does not limit this.

[0035] The electronic device may receive a touch operation (also referred to as operation 1, such as a click) on the camera application icon 201. In response to the above operation 1, the electronic device may start the camera to collect images in real time and display a shooting interface.

[0036] S102: The electronic device collects N frames of images including the target object in real time through a camera.

[0037] Wherein, N may be greater than or equal to 2. The type of each image frame in the N frames of images may be an RGB-D image. For the description of the RGB-D image, reference may be made to the above description, which will not be repeated here.

[0038] For example, Figure 2B As shown, in response to a touch operation on the camera application icon 201, the electronic device may display a shooting interface 210. The shooting interface 210 may include a shooting control 211, one or more shooting mode controls (e.g., a night scene mode control 212A, a video mode control 212B, a movie mode control 212C, a professional mode control 212D, and a more mode control 212E), a preview window 213, etc. The preview window 213 may display an image captured by the camera in real time.

[0039] S103: When the electronic device detects that the target object is in motion, the electronic device determines the depth (which may be referred to as predicted depth) of the target object in the target image after the N frame images based on the N frame images including the target object.

[0040] Specifically, the electronic device can obtain the depth of the target object on each frame of the N frames, that is, obtain the distance from the position of the target object in each frame of the image in three-dimensional space to the camera. The electronic device can determine the predicted depth of the target object in the target image based on the depth of the target object on each frame of the image. The specific implementation method will be described in detail in the subsequent embodiments, which will not be repeated here.

[0041] In some embodiments, N is less than or equal to a preset number 1, and the preset number 1 may preferably be 100. When the number of images including the target object captured by the electronic device is greater than the preset number 1, the electronic device may select a preset number 1 of images including the target object based on time from near to far, for determining the depth of the target object in the target image after N frames of images. For example, taking the preset number 1 as 100 as an example, when the electronic device captures 200 frames of images including the target object, the electronic device may select the 101st frame to the 200th frame based on time from near to far, for determining the depth of the target object in the target image after the 200th frame.

[0042] Among them, the interval between the target image and the image with the closest acquisition time among the N frames is related to the delay in the autofocus process under different shooting modes. For example, if the delay in the autofocus process is 2 frames, the interval between the target image and the image with the closest acquisition time among the N frames is 2 frames; if the delay in the autofocus process is 1 frame, the interval between the target image and the image with the closest acquisition time among the N frames is 1 frame. For example, if the electronic device captures the 1st to 5th frames of images including the target object, and the delay in the autofocus process is 2 frames, the target image is the 8th frame, and it is 2 frames away from the 5th frame of image with the closest acquisition time. This application is not limited to this.

[0043] S104: Before acquiring the target image, the electronic device uses a motor to push the lens in the camera to a focus position corresponding to the predicted depth based on the predicted depth of the target object.

[0044] Specifically, if the target object is to be clearly imaged by the camera of an electronic device, the following imaging formula needs to be satisfied:

[0045]

[0046] Among them, u is the object distance, that is, the distance from the position of the target object in three-dimensional space to the camera (that is, the depth of the target object); v is the image distance, that is, the distance from the lens in the camera to the imaging plane; f is the focal length, that is, the distance from the lens focus to the lens.

[0047] When the depth of the target object changes, the image distance also needs to change accordingly so that the target object can be clearly imaged by the camera of the electronic device. Therefore, before acquiring the target image, the electronic device can calculate the corresponding image distance based on the predicted depth of the target object through the imaging formula, and then determine the focus position of the lens in the camera. Then, the electronic device can push the lens in the camera to the focus position through a motor, changing the distance between the lens in the camera and the imaging plane. When acquiring the target image, the lens can accurately focus on the target object, and the target object can be clearly imaged on the imaging plane through the camera of the electronic device.

[0048] S105: The electronic device collects a target image through the lens at the focus position.

[0049] In a possible implementation, when the electronic device captures an image of the target object according to the predicted depth each time, the electronic device may move the lens to a focus position corresponding to each predicted depth.

[0050] For example, the electronic device calculates that target image 1 corresponds to predicted depth 1, and target image 2 corresponds to predicted depth 2, and determines the focus position of the lens as focus position 1 based on predicted depth 1, and determines the focus position of the lens as focus position 2 based on predicted depth 2. Before the electronic device captures target image 1, the electronic device pushes the lens to focus position 1 through a motor, and the electronic device captures target image 1 with the lens at focus position 1. Before the electronic device captures target image 2, the electronic device pushes the lens to focus position 2 through a motor, and the electronic device captures target image 2 with the lens at focus position 2.

[0051] In a possible implementation, when the electronic device pushes the lens to the corresponding focus position to capture an image based on a certain predicted depth, the electronic device can calculate the difference between each subsequent predicted depth and the predicted depth. If the difference is less than or equal to the preset difference 1, the electronic device keeps the focus position of the lens unchanged; if the difference is greater than the preset difference 1, the electronic device changes the focus position of the lens again.

[0052] For example, the electronic device calculates that target image 1 corresponds to predicted depth 1, target image 2 corresponds to predicted depth 2, and target image 3 corresponds to predicted depth 3, and predicted depth 1 corresponds to focus position 1, predicted depth 2 corresponds to focus position 2, and predicted depth 3 corresponds to focus position 3. When the electronic device pushes the lens to focus position 1 to capture target image 1 based on predicted depth 1, if the difference between predicted depth 2 and predicted depth 1 is less than preset difference 1, when the electronic device captures target image 2, the electronic device does not change the focus position of the lens, and still captures target image 2 at focus position 1. When the difference between predicted depth 3 and predicted depth 1 is greater than preset difference 1, the electronic device pushes the lens to focus position 3 before capturing target image 3, and captures target image 3 with the lens at focus position 3.

[0053] Further, yes Figure 1 Step S103 in the illustrated embodiment is described in detail.

[0054] like Figure 3 As shown, if the electronic device collects N frames of images including the target object through a camera, and each frame of the image includes the depth of the target object and the corresponding confidence level, the specific implementation of step S103 may include:

[0055] Stage 1: The electronic device preprocesses the acquired N depths.

[0056] S301: The electronic device determines N depths and corresponding confidence levels of a target object from N frames of images.

[0057] The depth of the target object included in each frame of image indicates the distance of the target object in three-dimensional space from the camera when the camera captures the frame of image. For example, if the depth of the target object in the first frame of image is 1 meter, it means that when the camera captures the first frame of image, the target object is 1 meter away from the camera in three-dimensional space; if the depth of the target object in the second frame of image is 2 meters, it means that when the camera captures the second frame of image, the target object is 2 meters away from the camera in three-dimensional space.

[0058] For example, if the value of N is 8, the electronic device determines 8 depths of the target object from 8 frames of images and their corresponding confidences as shown in the following Table 1:

[0059] Table 1

[0060] Image frame number depth Confidence 1 2 0.2 2 3 0.6 3 4 0.3 4 5 0.8 5 9 0.7 6 6 0.7 7 8 0.8 8 7 0.9

[0061] Among them, it can be seen from Table 1 that the depth of the target object in the first frame image is 2, and the confidence is 0.2; the depth of the target object in the second frame image is 3, and the confidence is 0.6; the depth of the target object in the third frame image is 4, and the confidence is 0.3; the depth of the target object in the fourth frame image is 5, and the confidence is 0.8; the depth of the target object in the fifth frame image is 9, and the confidence is 0.7; the depth of the target object in the sixth frame image is 6, and the confidence is 0.7; the depth of the target object in the seventh frame image is 8, and the confidence is 0.8; the depth of the target object in the eighth frame image is 7, and the confidence is 0.9.

[0062] Table 1 is only used to exemplify the present application and does not impose any specific limitation on the present application.

[0063] S302: The electronic device marks a depth with low confidence.

[0064] Among them, the confidence level less than the preset threshold 1 can be called low confidence. Preferably, the preset threshold 1 can be selected as 0.5, that is, the confidence level less than 0.5 can be called low confidence, and the electronic device can mark the depth with a confidence level less than 0.5. The preset threshold 1 can also be other values ​​besides 0.5, which is not limited by this application.

[0065] Exemplarily, taking Table 1 as an example, the preset threshold 1 is 0.5, the confidence of the depth of the target object in the first frame image is 0.2, and the confidence of the depth of the target object in the third frame image is 0.3. The electronic device can mark the depth 2 corresponding to the first frame image and the depth 4 corresponding to the third frame image as low confidence.

[0066] S303: The electronic device marks an abnormal depth in the effective depth.

[0067] The effective depth refers to the depth with a confidence level higher than a preset threshold value 1.

[0068] Exemplarily, taking Table 1 as an example, the preset threshold 1 is 0.5, and the confidence levels of the depths of the second frame image and the fourth to eighth frame images are 0.6, 0.8, 0.7, 0.7, 0.8 and 0.9 respectively, all greater than 0.5, so the electronic device marks the above depths as valid depths.

[0069] In the embodiment of the present application, the specific implementation method of this step can be as follows:

[0070] A). Arrange the corresponding effective depths according to the image frame number from small to large. For example, taking Table 1 as an example, the corresponding effective depths are arranged from small to large according to the image frame number: 3, 5, 9, 6, 8, 7.

[0071] In this process, the depth with low confidence is not involved, that is, the determination of abnormal depth in the effective depth does not use the depth with low confidence.

[0072] B). The electronic device determines whether the Kth effective depth is an abnormal depth based on the average value and standard deviation of the preset area corresponding to the Kth effective depth.

[0073] Specifically, the electronic device can calculate the average value and standard deviation of each preset area, and based on the average value and standard deviation of each preset area, calculate the deviation estimation value Z of the effective depth corresponding to each preset area. When the absolute value of the deviation estimation value Z is greater than the preset threshold 2, the electronic device marks the effective depth as an abnormal depth; when the absolute value of the deviation estimation value Z is less than or equal to the preset threshold 2, the electronic device marks the effective depth as a normal depth. The formula of the deviation estimation value Z can be:

[0074]

[0075] Wherein, X is the effective depth, μ is the average value of the effective depth corresponding to the preset area, and σ is the standard deviation of the effective depth corresponding to the preset area.

[0076] Exemplarily, taking the example of A) as an example, it is determined whether each effective depth is an abnormal depth:

[0077] Effective depth: 3, 5, 9, 6, 8, 7, then:

[0078] 1). The first effective depth is 3, and its corresponding preset areas are 3, 5, and 9. The average value of the preset areas is 5.7, and the standard deviation is 2.5. The Z value of the first effective depth is -1.08.

[0079] 2). The second effective depth is 5, and its corresponding preset areas are 5, 9, and 6. The average value of the preset area is 6.7 and the standard deviation is 1.7. The Z value of the second effective depth is -1.

[0080] 3). The third effective depth is 9, and its corresponding preset areas are 9, 6, and 8. The average value of the preset areas is 7.7, and the standard deviation is 1.2. The Z value of the third effective depth is 1.08.

[0081] 4). The fourth effective depth is 6, and its corresponding preset areas are 6, 8, and 7. The average value of the preset areas is 7 and the standard deviation is 0.8. The Z value of the fourth effective depth is -1.25.

[0082] 5). The fifth effective depth is 8, and its corresponding preset areas are 6, 8, and 7. The average value of the preset areas is 7, and the standard deviation is 0.8. The Z value of the fifth effective depth is 1.25.

[0083] 6). The sixth effective depth is 7, and its corresponding preset areas are 6, 8, and 7. The average value of the preset areas is 7 and the standard deviation is 0.8. The Z value of the sixth effective depth is 0.

[0084] C). Electronic equipment marks abnormal depth.

[0085] Exemplarily, continuing with the example of B), the preset threshold 2 is 1.2. Since the Z values ​​of the 4th effective depth 6 and the 5th effective depth 8 are greater than 1.2, the electronic device can mark the 4th effective depth and the 5th effective depth as abnormal depths.

[0086] S304: The electronic device replaces the low-confidence depths and abnormal depths to obtain N pre-processed depths.

[0087] In an embodiment of the present application, the electronic device may arrange the N depths in ascending order according to the image frame number. For the low confidence depth and abnormal depth, the electronic device may perform weighted averaging based on one or more of the following data: the previous normal depth 1 of the depth, the distance 1 of the normal depth 1 from the depth, the next normal depth 2 of the depth, and the distance 2 of the normal depth 2 from the depth, replace the depth, and obtain the pre-processed N depths.

[0088] For example, the electronic device needs to replace abnormal depth 1. The electronic device performs weighted average of the normal depth 1 before the abnormal depth 1, the distance 1 between the normal depth 1 and the abnormal depth 1, the normal depth 2 after the abnormal depth 1, and the distance 2 between the normal depth 2 and the abnormal depth 1, and the weighted average can be used to replace the abnormal depth 1. The formula of weighted average can be as follows:

[0089]

[0090] Exemplarily, taking the example of step S303 as an example, the electronic device arranges N depths from small to large according to the image frame number: 2, 3, 4, 5, 9, 6, 8, 7. Among them, the depth 2 of the first frame image and the depth 4 of the third frame image are low confidence depths, and the depth 6 of the sixth frame image and the depth 8 of the seventh frame image are abnormal depths. Among them:

[0091] 1). Replace the depth of the first frame image. The depth of the first frame image has no normal depth before, and the next normal depth is the depth 3 of the second frame image. Therefore, the depth of the first frame image is replaced with 3.

[0092] 2) Replace the depth of the third frame image. The previous normal depth of the depth of the third frame image is the depth 3 of the second frame image, and the distance is 1; the next normal depth of the depth of the third frame image is the depth 5 of the fourth frame image, and the distance is 1. After weighted averaging, the weighted average value is (3×1+5×1) / 2=4, and the depth of the third frame image is replaced with 4.

[0093] 3) Replace the depth of the 6th frame image. The previous normal depth of the 6th frame image is the depth of 9 of the 5th frame image, and the distance is 1; the next normal depth of the 6th frame image is the depth of 7 of the 8th frame image, and the distance is 2. After weighted averaging, the weighted average value is (9×1+7×2) / 3=7.7, and the depth of the 6th frame image is replaced with 7.7.

[0094] 4) Replace the depth of the 7th frame image. The previous normal depth of the 7th frame image is the depth of the 5th frame image, 9, and the distance is 2; the next normal depth of the 7th frame image is the depth of the 8th frame image, 7, and the distance is 1. After weighted averaging, the weighted average value is (9×2+7×1) / 3=8.3, and the depth of the 7th frame image is replaced with 8.3.

[0095] N depths: 2, 3, 4, 5, 9, 6, 8, 7. After preprocessing, the obtained N depths are: 3, 3, 4, 5, 9, 7.7, 8.3, 7.

[0096] In some embodiments, for the low-confidence depth, the electronic device can also use other alternatives, such as calculating the average of the normal depths of the frame images before and after the low-confidence depth, and replacing it with the average. For multiple consecutive (for example, 5, 6, 7, 8) low-confidence depths, the electronic device can discard the multiple low-confidence depths, or replace them through an interpolation algorithm, or output the average of all low-confidence depths, etc.

[0097] Stage 2: The electronic device fits a second-order polynomial to determine the predicted depth based on the preprocessed N depths.

[0098] S305: The electronic device fits a second-order polynomial for determining a predicted depth based on the preprocessed N depths.

[0099] First, the position P of the moving object at time t t =(X t , Y t , Z t ), speed V t =(v xt , v yt , v zt ) and acceleration A t =(a xt , a yt , a zt ) satisfies the following equation of motion:

[0100]

[0101] Among them, P 0 is the initial position of the object at time 0 (i.e. the starting time).

[0102] Therefore, based on the above motion equation, in order to predict the depth of the target object that conforms to the motion equation as accurately as possible, the electronic device can use a second-order polynomial to fit the motion equation. The electronic device can use a polynomial interpolation method to establish the motion equation. The second-order polynomial to be fitted (also called a depth prediction model) can be as follows:

[0103] Depth(t)=a 0 +a 1 t+a 2 t 2

[0104] Among them, Depth(t) is the predicted depth, a 0 is the initial depth, a 1 is the initial longitudinal velocity of the target object, a 2 is the longitudinal acceleration of the target object. 0 、a 1 、a 2 is the parameter to be determined, t is the independent variable, which means the frame number of the image, and Depth(t) is the depth corresponding to the image frame number.

[0105] Since the acceleration of the target object will not remain constant in real three-dimensional space, the electronic device needs to adjust a in real time according to the depth of the historical image frame. 0 、a 1 、a 2 At the same time, due to the low complexity of the calculation, each time the depth is predicted, the electronic device will recalculate a according to the depth of the historical image frame 0 、a1 、a 2 , in order to maintain a high prediction accuracy.

[0106] If the second-order polynomial is fitted using N depths of N frames, the electronic device can obtain a relatively smooth motion equation model, but it may ignore the instantaneous changes and abnormalities of the target object's motion; and if the speed and acceleration of the target object are directly calculated using specific frames (for example, the frames closest to the current moment), although the instantaneous motion state of the target object can be more accurately portrayed, it may be affected by factors such as noise. Therefore, the electronic device can use the weighted least squares method, which can comprehensively consider the motion of the target object in the historical image frames, and can sensitively grasp the instantaneous motion and motion acceleration of the current target object.

[0107] The specific process of solving the second-order polynomial to be fitted can be as follows:

[0108] Depth(t)=a 0 +a 1 t+a 2 t 2

[0109] To solve the second-order polynomial, we need to find a set of values ​​that satisfy the following equation:

[0110]

[0111] In this equation, y i is the actual depth corresponding to the i-th frame image, y′ i is the predicted depth of the i-th frame image calculated by the second-order polynomial, y i -y′ i That is, the error between the actual depth and the predicted depth. In other words, the electronic device needs to find a 0 , a 1 , a 2 The optimal value of Minimum.

[0112] To find a 0 , a 1 , a 2 The optimal value of , the second-order polynomial can be expressed as a system of linear equations:

[0113]

[0114] In this linear system of equations, t 1 Indicates the image frame number corresponding to the first depth, t 2 Indicates the image frame number corresponding to the second depth...t nIndicates the image frame number corresponding to the nth depth; y 1 ,y 2 ……y n Indicates the actual depth of the 1st frame image, the 2nd frame image, ... the nth frame image.

[0115] In the embodiment of the present application, the depths are arranged from small to large according to the image frame number, so the ordinal number of the depth is the image frame number corresponding to the depth. For example, the first depth corresponds to the first frame image, the second depth corresponds to the second frame image, and so on.

[0116] Next, the above linear equations can be expressed as a matrix system:

[0117]

[0118] in, is the coefficient matrix, is the unknown vector, is the vector on the right side of the equal sign.

[0119] Then, for α 0 , a 1 , a 2 Finding partial derivatives, we can get the following partial derivative equations:

[0120]

[0121] Next, based on the weight corresponding to each actual depth (Since the actual depth of each record corresponds to each frame of the image one by one, the weight can also be regarded as the weight corresponding to each frame of the image). The above matrix equations and partial derivative equations are multiplied by the corresponding weights using the weighted least squares method. The following equation group 1 can be obtained:

[0122]

[0123] Where n is the total number of depths (i.e., the total number of image frames), α is the preset attenuation coefficient (preferably 0.7, but can also be 1, 0.5, etc.), t i is the number of image frames corresponding to the i-th depth, y i is the actual depth of the i-th frame image.

[0124] From the weight calculation formula It can be seen that the actual depth of the image frame closer to the current time point has a higher weight. In this way, not only the motion of the target object in the historical image frame can be considered, but also the sudden change of the object's motion speed can be applied.

[0125] Electronic devices can use Cramer's Rule to solve the equation system 1. For a linear equation system with n variables, Cramer's Rule uses the determinant of the coefficient matrix of the equation system and replaces one of the columns with the value of the vector on the right side of the equal sign to obtain the unknown number a. 0 , a 1 , a 2 The solution.

[0126] When determining a 0 , a 1 , a 2 After solving the problem, the electronic device can use a second-order polynomial to calculate the predicted depth of each frame in the historical image frame, and then calculate the weighted average of the prediction errors between the predicted depth of each frame and the actual depth. The electronic device can determine the confidence of the second-order polynomial based on the weighted average of the prediction errors. If the confidence of the second-order polynomial is greater than the preset threshold 3, the electronic device can use the second-order polynomial to calculate the predicted depth of the target image; if the confidence of the second-order polynomial is less than the preset threshold 3, the electronic device can perform autofocus based on phase focusing technology / contrast focusing technology.

[0127] In the embodiment of the present application, The attenuation coefficient α has different values, and the fitted second-order polynomial is also different. Figure 4A As shown in FIG. 1 , when the attenuation coefficient α is 0.7, the fitted second-order polynomial is optimal.

[0128] In the embodiment of the present application, Figure 3 The specific process of step S103 shown is only used to exemplify the present application. In a specific implementation, Figure 4B As shown, in addition to fitting a second-order polynomial using a polynomial interpolation method, the electronic device can also have the following ways to determine the predicted depth: according to the motion equation method, the speed and acceleration of the target object are calculated based on a specified multiple-frame image (for example, the few frames closest to the current time point, etc.), thereby calculating the predicted depth of the target object; or, the electronic device can also determine the predicted depth of the target object based on RGB-D data, Kalman filter and recurrent neural network (RNN); or, the electronic device can also perform three-dimensional reconstruction based on the internal parameters of the camera, calculate the position of the SOT box in the three-dimensional space at the predicted time point, thereby determining the predicted depth of the target object. In other words, how the electronic device determines the predicted depth of the target object in the target image based on N frames of images including the target object is not limited in this application.

[0129] For example, Figure 4BAs shown, after the electronic device obtains the depth and confidence in the N frames of images, it can select a method for determining the predicted depth. The method for predicting the depth can refer to the description in the aforementioned embodiment. After the electronic device determines the method for predicting the depth, it can determine the predicted depth according to the method and return the result of the predicted depth.

[0130] In some embodiments, the electronic device can determine the predicted depths of multiple target objects based on 1) fitting a second-order polynomial method, and / or 2) a motion equation method, and / or 3) a method for calculating predicted depth based on RGB-D data, a Kalman filter, and an RNN, and / or 4) a method for calculating predicted depth based on internal parameters of the camera (the four methods are described above for details), so that the electronic device can select the predicted depth for autofocus. For example, the electronic device can select the minimum value of multiple predicted depths for autofocus. The electronic device can select the predicted depth for autofocus from the above-mentioned multiple predicted depths by other methods.

[0131] In some embodiments, the electronic device may receive an operation acting on area 1 on image A, and the electronic device may identify the target object from area 1 and mark it with a single object tracking (SOT) frame. For multiple frames of images such as image B, image C, etc. after the acquisition time of image A, the electronic device may determine the position of the SOT frame in each image through a kernelized correlation filter (KCF) algorithm (e.g., a DS-KCF algorithm). Among them, the SOT frame can be used to represent the position of the target object in each image. Then, the electronic device may determine the position of the SOT frame in the target image based on a motion estimation and motion compensation (MEMC) algorithm. The electronic device may calculate the depth of the target object in the target image based on the depth corresponding to the position of the SOT frame of each image in the N-frame image (i.e., the depth of the target object on each image), according to the method of the aforementioned embodiment. Among them, the DS-KCF algorithm uses data in an RGB-D image.

[0132] In some embodiments, the electronic device may receive an operation acting on area 1 on image A, and the electronic device may identify the target object from area 1 and mark it with SOT frame 1. Then, the electronic device may configure multiple SOT frames (e.g., SOT frame 2, SOT frame 3, etc.) around the SOT frame 1. Then, the electronic device may determine the position of each SOT frame in each image by a kernelized correlation filter (KCF) algorithm (e.g., DS-KCF algorithm). Then, the electronic device may determine the position of each SOT frame in the target image based on the MEMC algorithm. The electronic device may calculate the depth corresponding to each SOT frame position on the target image by the method of the aforementioned embodiment. Wherein, when the electronic device calculates the depth corresponding to the SOT frame on each target image by the aforementioned embodiment, the input is the depth corresponding to the SOT frame position in the N frame image. In this way, the electronic device can obtain multiple predicted depths. Then, the electronic device may determine the predicted depth actually used for autofocus and the corresponding SOT frame from multiple predicted depths according to the preset condition 1. Wherein, the preset condition 1 may be the predicted depth with the smallest value, or the median value of multiple predicted depths. In the time domain, the electronic device can also eliminate abnormal predicted depths based on filtering algorithms.

[0133] For example, Figure 4CAs shown, for example, the electronic device may receive an operation acting on area 1 on image A, and the electronic device may identify the target object from area 1 and mark it with SOT frame 1. The electronic device may configure SOT frames 2 to 9 around SOT frame 1. The electronic device may determine the positions of SOT frames 1 to 9 in each image in N frames of images by a kernelized correlation filter (KCF) algorithm (e.g., DS-KCF algorithm). Then, the electronic device may determine the positions of SOT frames 1 to 9 in the target image based on the MEMC algorithm. The electronic device may calculate the depth corresponding to the position of SOT frame 1 in the target image by using the depth corresponding to the position of SOT frame 1 on each image as input through the aforementioned embodiment; the electronic device may calculate the depth corresponding to the position of SOT frame 2 in the target image by using the depth corresponding to the position of SOT frame 2 on each image as input through the aforementioned embodiment; the electronic device may calculate the depth corresponding to the position of SOT frame 3 in the target image by using the depth corresponding to the position of SOT frame 3 on each image as input through the aforementioned embodiment. The depth corresponding to the position of SOT frame 4 to SOT frame 9 in the target image is analogous. In this way, the electronic device can calculate 9 possible predicted depths of the target object in the target image. Then, the electronic device can determine the predicted depth actually used for autofocus and the corresponding SOT frame from the 9 possible predicted depths according to preset condition 1. Among them, the preset condition 1 can be the predicted depth with the smallest value, or the median value of the 9 predicted depths.

[0134] For example, taking the first image and the second image captured by the electronic device through the camera as an example, the implementation method is specifically described:

[0135] An electronic device may capture a first image and a second image through a camera. The first image includes a first depth of a target object, the second image includes a second depth of the target object, and the capture time of the second image is later than the capture time of the first image. The electronic device predicts a third depth of the target object in the first target image based on the first depth and the second depth. The capture time of the first target image is after the capture time of the second image. The electronic device determines a first focus position of the lens in the camera based on the third depth. Before capturing the first target image, the electronic device moves the lens in the camera to the first focus position through a motor. The electronic device captures the first target image at the first focus position through the camera.

[0136] In a possible implementation, the electronic device may fit a depth prediction model based on the first depth and the second depth. The depth prediction model is used to determine the depth of the target object in the image based on the frame number of the image. The electronic device predicts the third depth of the target object in the first target image based on the depth prediction model and the frame number of the first target image.

[0137] In a possible implementation, the electronic device may determine a first image distance corresponding to the third depth based on the third depth and the focal length of the lens in the camera. The electronic device may determine the first focus position of the lens in the camera based on the first image distance.

[0138] In a possible implementation, in addition to the first image and the second image, the electronic device may also capture a third image, which includes a fourth depth of the target object. The acquisition time of the third image is later than the acquisition time of the second image. Then, the electronic device may predict the fifth depth of the target object in the second target image based on the first depth, the second depth and the third depth. The acquisition time of the second target image is later than the acquisition time of the first target image.

[0139] In a possible implementation, after the electronic device acquires the first target image through the camera at the first focus position, it can also determine the second focus position of the lens in the camera based on the fifth depth. Before acquiring the second target image, the electronic device moves the lens in the camera to the second focus position through a motor. The electronic device acquires the second target image through the camera at the second focus position.

[0140] In a possible implementation, after the electronic device captures the first target image through the camera at the first focus position, when the difference between the fifth depth and the third depth is less than the first preset difference, the electronic device captures the second target image through the camera at the first focus position. When the difference between the fifth depth and the third depth is greater than the first preset difference, the electronic device determines the second focus position of the lens in the camera based on the fifth depth. Before capturing the second target image, the electronic device moves the lens in the camera to the second focus position through a motor. The electronic device captures the second target image through the camera at the second focus position.

[0141] In a possible implementation, the electronic device may further display a fourth image before predicting the third depth of the target object in the first target image based on the first depth and the second depth. The electronic device receives an operation acting on a first area on the fourth image. The electronic device identifies the target object from the first area. The electronic device detects that the target object is located in a second area on the first image. The electronic device determines the depth of the second area in the first image as the first depth.

[0142] Figure 5 The hardware structure of an electronic device provided by an embodiment of the present application is exemplified.

[0143] In the embodiment of the present application, the electronic device 100 may be the electronic device mentioned above.

[0144] like Figure 5 As shown, the electronic device 100 may include a processor 501, a memory 502, a wireless communication module 503 (optional), a display screen 504, a camera 505, an audio module 506 (optional) and a microphone 507 (optional), and the processor 501, the memory 502, the wireless communication module 503 (optional), the display screen 504, the camera 505, the audio module 506 (optional) and the microphone 507 (optional) may be connected via a bus.

[0145] It is understandable that the structure shown in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also include Figure 5 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 5 The components shown may be implemented in hardware, software or a combination of software and hardware.

[0146] The processor 501 may include one or more processor units, for example, the processor 501 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. The controller can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0147] The processor 501 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 501 is a cache memory. The memory may store instructions or data that the processor 501 has just used or circulated. If the processor 501 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 501, and thus improves the efficiency of the system.

[0148] In some embodiments, the processor 501 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, etc.

[0149] The memory 502 is coupled to the processor 501 and is used to store various software programs and / or multiple groups of instructions. In a specific implementation, the memory 502 may include a volatile memory, such as a random access memory (RAM); it may also include a non-volatile memory, such as a ROM, a flash memory, a hard disk drive (HDD) or a solid state drive (SSD); the memory 502 may also include a combination of the above-mentioned types of memories. The memory 502 may also store some program codes so that the processor 501 can call the program codes stored in the memory 502 to implement the implementation method of the embodiment of the present application in the electronic device 100. The memory 502 may store an operating system, such as an embedded operating system such as uCOS, VxWorks, RTLinux, etc.

[0150] The wireless communication module 503 can provide wireless communication solutions applied to the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 503 can be one or more devices integrating at least one communication processing module. The wireless communication module 503 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 501. The wireless communication module 503 can also receive signals to be sent from the processor 501, modulate and amplify them, and convert them into electromagnetic waves for radiation through the antenna. In some embodiments, the electronic device 100 can also communicate with the processor 501 through the Bluetooth module ( Figure 5 Not shown), WLAN module ( Figure 5The WLAN module may provide a solution including one or more Bluetooth communication in basic rate / enhanced data rate (BR / EDR) or Bluetooth low energy (BLE), and a solution including one or more WLAN communication in Wi-Fi direct, Wi-Fi LAN or Wi-Fi softAP.

[0151] The display screen 504 can be used to display images, videos, etc. The display screen 504 may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 504, where N is a positive integer greater than 1.

[0152] The camera 505 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 505, where N is a positive integer greater than 1.

[0153] The audio module 506 can be used to convert digital audio information into analog audio signal output, and can also be used to convert analog audio input into digital audio signal. The audio module 506 can also be used to encode and decode audio signals. In some embodiments, the audio module 506 can also be arranged in the processor 501, or some functional modules of the audio module 506 can be arranged in the processor 501.

[0154] Microphone 507, which can also be called "microphone" or "microphone", can be used to collect sound signals in the environment surrounding the electronic device, convert the sound signals into electrical signals, and then subject the electrical signals to a series of processes, such as analog-to-digital conversion, to obtain an audio signal in digital form that can be processed by the processor 501 of the electronic device. When making a call or sending a voice message, the user can speak by approaching the microphone 507 with his mouth to input the sound signal into the microphone 507. The electronic device 100 can be provided with at least one microphone 507. In other embodiments, the electronic device 100 can be provided with two microphones 507, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 507 to realize the collection of sound signals, noise reduction, identification of sound sources, and realization of directional recording function, etc.

[0155] The electronic device 100 may further include a sensor module ( Figure 5 The sensor module may include a plurality of sensor devices, for example, a touch sensor ( Figure 5 The touch sensor may also be referred to as a "touch control device". The touch sensor may be provided on the display screen 504, and the touch sensor and the display screen 504 form a touch screen, also referred to as a "touch control screen". The touch sensor may be used to detect a touch operation on or near the touch sensor.

[0156] It should be noted that Figure 5 The electronic device 100 shown in the figure is only used to exemplify the hardware structure of the electronic device provided by the present application, and does not constitute a specific limitation to the present application.

[0157] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0158] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0159] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An automatic focusing method, characterized in that: include: Capturing a first image and a second image through a camera; wherein the first image includes a first depth of the target object, the second image includes a second depth of the target object, and the acquisition time of the second image is later than the acquisition time of the first image; Based on the first depth and the second depth, predicting a third depth of the target object in a first target image; wherein the acquisition time of the first target image is after the acquisition time of the second image; Based on the third depth, determining a first focus position of a lens in the camera; Before acquiring the first target image, moving the lens in the camera to the first focusing position by a motor; The first target image is captured by the camera at the first focus position.

2. The method according to claim 1, characterized in that The predicting, based on the first depth and the second depth, a third depth of the target object in the first target image specifically includes: Based on the first depth and the second depth, fitting a depth prediction model; wherein the depth prediction model is used to determine the depth of the target object in the image based on the frame number of the image; The third depth of the target object in the first target image is predicted based on the depth prediction model and the frame number of the first target image.

3. The method according to claim 1 or 2, characterized in that: The determining, based on the third depth, a first focus position of a lens in the camera specifically includes: Determining a first image distance corresponding to the third depth based on the third depth and a focal length of a lens in the camera; Based on the first image distance, the first focus position of the lens in the camera is determined.

4. The method according to claim 1, characterized in that: The method further comprises: A third image is acquired, wherein the third image includes a fourth depth of the target object; wherein the acquisition time of the third image is later than the acquisition time of the second image; Based on the first depth, the second depth and the third depth, a fifth depth of the target object in a second target image is predicted; wherein the acquisition time of the second target image is after the acquisition time of the first target image.

5. The method according to claim 4, characterized in that After acquiring the first target image at the first focus position by the camera, the method further includes: Based on the fifth depth, determining a second focus position of the lens in the camera; Before acquiring the second target image, moving the lens in the camera to the second focusing position by a motor; The second target image is captured by the camera at the second focus position.

6. The method according to claim 4, characterized in that After acquiring the first target image at the first focus position by the camera, the method further includes: When the difference between the fifth depth and the third depth is less than a first preset difference, capturing the second target image at the first focus position by the camera; When the difference between the fifth depth and the third depth is greater than a first preset difference, determining a second focus position of the lens in the camera based on the fifth depth; Before acquiring the second target image, moving the lens in the camera to the second focusing position by a motor; The second target image is captured by the camera at the second focus position.

7. The method according to claim 1, characterized in that Before predicting a third depth of the target object in the first target image based on the first depth and the second depth, the method further includes: displaying a fourth image; receiving an operation acting on a first area on the fourth image; identifying the target object from the first area; detecting that the target object is located in a second area on the first image; The depth of the second area in the first image is determined as the first depth.

8. An electronic device, characterized in that: include: One or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer executable programs. When the one or more processors execute the computer executable programs, the electronic device executes the method as described in any one of claims 1-7.

9. A chip system, characterized in that: It comprises a processing circuit and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processing circuit, and the processing circuit is used to run the code instructions so that the chip system executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The method comprises a computer executable program, and when the computer executable program is run on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

Images