Moving object focusing method and device

Abstract

The invention provides a focusing method and device for a moving object, which can improve the focusing processing efficiency of terminal equipment and improve the definition of the moving object in a preview picture of a camera, thereby improving the user experience. The method is applied to terminal equipment and comprises the steps that at a first moment, a first frame image is displayed on a preview picture, and the first frame image comprises a moving object located at a first position; at a second moment, a second frame image is displayed on the preview picture, the second frame image comprises the moving object at the second position, and the second moment is after the first moment; and at a third moment, displaying a third frame of image on the preview picture, the third frame of image being acquired after focusing the lens of the terminal device by using the predicted lens defocusing amount, the lens defocus amount is obtained based on any one of a first phase corresponding to the first frame image and a second phase corresponding to the second frame image, the first position and the second position, and the third moment is after the second moment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method and device for focusing a moving object. Background Art

[0002] In people's daily lives, it has become a common phenomenon to use the shooting functions of terminal devices such as mobile phones and smart watches to record life anytime and anywhere. The shooting function of terminal devices has added a lot of fun to people's lives.

[0003] Currently, when using the camera of a terminal device to shoot, the user needs to wait for a moment after aiming the camera at the object to be shot, so that the object in the picture can be clearly shown. At this time, the shooting button can be pressed to shoot a clear image. This moment of waiting time is the time required for the camera to focus. The same is true when shooting videos. The user needs to wait for a moment after aiming at the object to be shot, and the picture will be clear. If you want every picture in the video to be clear, the speed of the object in the picture should not move too fast to give the camera enough time to focus.

[0004] Therefore, for shooting moving objects, the focusing efficiency of the above focusing method is not high, and a high-definition picture may not be obtained, which affects the user experience. Summary of the invention

[0005] The present application provides a method and device for focusing on a moving object, which can improve the focusing processing efficiency of a terminal device, improve the clarity of the moving object in a preview image of a camera, and thus improve the user experience.

[0006] In a first aspect, the present application provides a method for focusing a moving object, which is applied to a terminal device, wherein a camera of the terminal device is in an on state and a preview screen of the camera includes a moving object, the method comprising: at a first moment, displaying a first frame image on the preview screen, the first frame image including the moving object at a first position; at a second moment, displaying a second frame image on the preview screen, the second frame image including the moving object at a second position, and the second moment being after the first moment; at a third moment, displaying a third frame image on the preview screen, the third frame image being acquired after focusing the lens of the terminal device using a predicted lens defocus amount, the lens defocus amount being obtained based on any one of a first phase corresponding to the first frame image and a second phase corresponding to the second frame image, as well as the first position and the second position, and the third moment being after the second moment.

[0007] In an embodiment of the present application, after displaying the first frame image and the second frame image, the terminal device can predict the lens defocus amount when capturing the third frame image based on any one of the first phase corresponding to the first frame image and the second phase corresponding to the second frame image acquired during the capture of the first frame image and the second frame image, as well as the first position and the second position, and push the lens to the quasi-focus position before the third frame image is captured, so that the third frame image is captured based on the quasi-focus position of the lens, which is beneficial for the user to see the clarity of the third frame image at the third moment. The method provided in the present application can improve the focus processing efficiency of the terminal device and improve the clarity of moving objects in the preview screen of the camera by predicting the lens defocus amount and pushing the lens to the quasi-focus position in advance, thereby improving the user experience.

[0008] Optionally, the terminal device may include a front camera and a rear camera, the number of which may be one or more. The lens of the terminal device may refer to the lens in the front camera or the lens in the rear camera, which is not limited in this application.

[0009] It is worth noting that the first phase of the first frame image includes multiple values ​​detected by multiple phase detection pixels arranged on the image sensor when capturing the first frame image, and each phase detection pixel obtains at least two phases in different directions (for the up and down and / or left and right of a single pixel plane), that is, each phase detection pixel obtains at least one pair of phases. The first phase is usually stored in a phase detection buffer (PD buffer) for subsequent calculation of the lens defocus amount. The detection method and data format of the second phase are similar to those of the first phase and will not be repeated.

[0010] Optionally, the terminal device may calculate the phase by scheduling a phase detection library (PDLIB) algorithm or a phase detection auto focus (PDAF) algorithm, which is not limited in the present application.

[0011] It should be understood that if the camera preview screen displayed by the terminal device is to become clear as quickly as possible, the first frame image, the second frame image and the third frame image can be three consecutive frame images, and the first frame image can be the first image displayed in the preview screen after the camera is started, and before each frame image from the third frame image onwards (including the third frame image) is captured, the lens defocus amount when the next frame image data is captured can be predicted based on the image data of the previous frame, and the motor can be driven to push the lens by the predicted defocus amount before the next frame image is captured. In other words, each frame image displayed by the terminal device from the third frame image onwards (including the third frame image) is likely to be a clearer image of the moving object.

[0012] It should also be understood that the first frame image, the second frame image and the third frame image can be three discontinuous frames of images. The first frame image and the second frame image are at least two frames of images that must be collected to ensure the display effect of the third frame image. It is not necessary to collect only two frames of images or to collect two consecutive frames of images. In some scenes with high accuracy requirements, data of multiple frames of images can be collected to predict the defocus amount of the lens when collecting the third frame of image, and this application does not limit this.

[0013] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: predicting a third position of the moving object based on the first position, the second position, and the time interval between the first moment and the second moment; predicting the lens defocus amount based on any one of the first phase and the second phase and the third position; and driving a motor to adjust the position of the lens of the terminal device according to the lens defocus amount to complete focusing.

[0014] It should be understood that the first position, the second position and the third position refer to different spatial positions of the moving object on its motion trajectory.

[0015] In combination with the first aspect, in certain implementations of the first aspect, predicting the third position of the moving object based on the first position, the second position, and the time interval between the first moment and the second moment includes: determining the first speed of the moving object at the second position and the second speed of the moving object at the second position based on the first position, the second position, and the time interval between the first moment and the second moment; predicting the third speed of the moving object when it is at the third position based on the first position and the first speed, the second position and the second speed; and calculating the third position based on the third speed and any one of the first position and the second position using a preset motion equation, wherein the preset motion equation is a relationship equation among the position, speed, and motion time of the moving object.

[0016] It should be understood that the first position and the second position can be represented by three-dimensional coordinates in the camera coordinate system, the first position of the moving object can be represented by multiple coordinates, and the multiple coordinates correspond to multiple points on the moving object. The second position can also be represented by multiple coordinates, and the multiple coordinates of the second position should correspond to the multiple coordinates of the first position one by one. The first position, second position, third position, etc. of the moving object can be represented by the coordinates of a point on the moving object, or can be represented by the center coordinates of the moving object calculated by multiple coordinates of each position, and this application does not limit this.

[0017] Optionally, the third position of the moving object can also be identified through a trained neural network model based on the first position, the second position and the time interval between the first moment and the second moment. It should be understood that the trained neural network model is obtained by training a preset neural network model with a large amount of object motion trajectory data, and the motion trajectory of the moving object can be predicted based on partial position and time data, thereby obtaining the third position on the trajectory.

[0018] Optionally, the preset neural network model can be a convolutional neural network (CNN), a feedforward (BP) neural network, or any other neural network that can be trained to predict the trajectory of a moving object, and this application does not make any specific limitations on this.

[0019] In combination with the first aspect, in certain implementations of the first aspect, predicting the lens defocus amount based on any one of the first phase and the second phase and the third position includes: determining a fourth position of the moving object in the image coordinate system based on the third position and an intrinsic parameter matrix and an extrinsic parameter matrix of the camera, and using an area including the fourth position as a first focus area of ​​the third frame image; determining the product obtained by multiplying the first phase difference or the second phase difference by a defocus transformation coefficient as the lens defocus amount, the first phase difference being a phase difference formed by phase detection pixels corresponding to the first focus area in the first phase, and the second phase difference being a phase difference formed by phase detection pixels corresponding to the first focus area in the second phase.

[0020] It should be understood that the fourth position may also include multiple coordinates, the size of the first focus area relative to the third frame image is a preset fixed value, and the area including the fourth position may refer to the center coordinates including the multiple coordinates, which is not limited in this application.

[0021] It should be understood that the three-dimensional coordinates of the moving object in the camera coordinate system can be converted into two-dimensional coordinates in the image coordinate system corresponding to the camera of the terminal device based on the camera's intrinsic parameter matrix and extrinsic parameter matrix. Therefore, the focus area that can be referenced when collecting the third frame image can be pre-determined according to the fourth position of the moving object in the image coordinate system, and the area including the fourth position is used as the first focus area for reference when focusing the third frame image. In addition, since the position of the phase detection pixels set in the image sensor in its pixel array is fixed, and the coordinate system formed by its pixel array has a corresponding relationship with the image coordinate system (the image coordinate system and the pixel coordinate system are in the same plane, and the origin of the pixel coordinate system is in the upper left corner of the image), the phase detection pixels included in the first focus area including the fourth position can be determined according to the fourth position of the moving object in the image coordinate system, and then one or more pairs of phase detection pixels in the first focus area are selected from the multiple values ​​of the first phase, and the first phase difference is calculated by a frequency domain algorithm or a space domain algorithm. Similarly, the second phase difference can also be calculated, which will not be repeated.

[0022] In the embodiment of the present application, the first focus area when capturing the third frame of the image is determined by predicting the third position of the moving object when capturing the third frame of the image, and then the phase difference corresponding to the first focus area is selected from the first phase or the second phase to calculate the lens defocus amount. Compared with the calculation result obtained by directly using the first phase corresponding to the first frame of the image or the second phase corresponding to the second frame of the image, the calculation result obtained in this way ensures that the lens of the terminal device is in a favorable position for imaging the moving object, which is conducive to making the moving object in the first focus area in the third frame of the image clearer, so as to achieve the purpose of the user tracking the moving object.

[0023] In combination with the first aspect, in certain implementations of the first aspect, before driving the motor to adjust the position of the lens of the terminal device according to the lens defocus amount, it includes: judging the relative position relationship between the imaging plane and the focal plane of the camera; determining a first distance between the imaging plane and the lens, and a second distance between the focal plane and the lens; driving the motor to adjust the position of the lens of the terminal device according to the lens defocus amount to complete focusing, including: when the first distance is less than the second distance, driving the motor to control the lens to move the lens defocus amount in a direction away from the imaging plane to complete focusing.

[0024] In combination with the first aspect, in some implementations of the first aspect, the method further includes: when the first distance is greater than the second distance, driving a motor to control the lens to move the lens defocus amount in a direction close to the imaging plane to complete focusing.

[0025] In the embodiment of the present application, before the motor drives the lens to move, the movement direction is first determined based on the first distance between the imaging plane and the lens and the second distance between the focal plane and the lens, which is conducive to moving the lens to the target position more quickly and accurately.

[0026] In combination with the first aspect, in some implementations of the first aspect, the moving object is in a second focus area of ​​the first frame image.

[0027] It should be understood that the purpose of the user's shooting is to focus on a moving object. When the first frame image is used as the first image captured in the current scene, the moving object is placed in the second focus area of ​​the first frame image, which is beneficial for the terminal device to quickly determine the moving object to be focused on through the second focus area, and then determine the first position of the moving object.

[0028] In combination with the first aspect, in some implementations of the first aspect, the second focus area is a default focus area.

[0029] In combination with the first aspect, in certain implementations of the first aspect, the second focus area is an area selected by the terminal device in response to a click operation of the user.

[0030] In an embodiment of the present application, the moving object is in the second focus area of ​​the first frame image, which is conducive to the terminal device quickly finding the moving object when obtaining the first position, and ignoring other objects in the current picture, only obtaining the position of the moving object, saving the computing power of the terminal device.

[0031] In a second aspect, the present application provides a focusing device for a moving object, the device comprising a module for implementing the method described in the first aspect.

[0032] In a third aspect, the present application provides a terminal device, including a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a receiver and transmit signals through a transmitter to execute the method in any possible implementation of the first aspect.

[0033] Optionally, there are one or more processors and one or more memories.

[0034] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0035] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips respectively. The embodiments of the present application do not limit the type of memory and the setting method of the memory and the processor.

[0036] It should be understood that the relevant data interaction process, such as sending the connection data request information, can be a process of outputting the indication information from the processor, and receiving the capability information can be a process of receiving the input capability information from the processor. Specifically, the processed output data can be output to the transmitter, and the input data received by the processor can come from the receiver. Among them, the transmitter and the receiver can be collectively referred to as a transceiver.

[0037] The terminal device in the third aspect mentioned above can be a chip, and the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0038] In a fourth aspect, the present application provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute a method in any possible implementation of the first aspect.

[0039] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program (also referred to as code or instructions), which, when executed on a computer, enables the computer to execute a method in any possible implementation of the first aspect described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0041] Figure 2 A software structure diagram of a terminal device provided in an embodiment of the present application;

[0042] Figure 3 A scene diagram applicable to the embodiments of the present application;

[0043] Figure 4 Preview screens of the terminal device at different times provided in the embodiment of the present application;

[0044] Figure 5 A schematic flow chart of a method for focusing a moving object provided in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the imaging principle provided in an embodiment of the present application;

[0046] Figure 7A schematic flow chart of another method for focusing a moving object provided in an embodiment of the present application;

[0047] Figure 8 A schematic block diagram of a focusing device for a moving object provided in an embodiment of the present application;

[0048] Fig. 9 A schematic block diagram of another focusing device for a moving object provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0050] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0051] It should be noted that in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0052] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0053] At present, when users take pictures or videos through terminal devices, they need to wait for a while when the camera is aimed at the object to be photographed, and the object to be photographed in the preview screen displayed by the terminal device can be clearly displayed. This is because the auto focus (AF) algorithm of the existing terminal devices is a negative feedback mode, that is, the terminal device will start the AF algorithm process to focus the lens only when it detects that the lens is out of focus through the currently collected image, which means that the user needs to wait for the terminal device to complete the lens focusing process before it can see a clear image. Simply put, if the image of the previous frame is out of focus, when the relative position of the terminal device and the object to be photographed does not change, the next frame of image can be collected after the lens focus is completed through the previous frame of image, and the next frame of image may be clear.

[0054] If the user's shooting scene keeps changing or the object is moving, after the terminal device detects that the lens is out of focus and performs focusing processing, the relative position of the terminal device and the object being photographed has changed. The lens position after focusing processing based on the previous frame of image is also out of focus for the next frame of image. Therefore, when the terminal device shoots a moving object, the lens may continue to be out of focus, making it difficult for the user to see a clear picture, resulting in a poor experience.

[0055] In view of this, the present application proposes a focusing method for a moving object, which predicts the position of the moving object in the third frame image by acquiring the position of the moving object in the first frame image and the second frame image, and calculates the lens defocus amount through the first phase of the first frame image or the second phase of the second frame image, and pushes the lens to the quasi-focus position before the third frame image starts to be captured, so that the lens completes the capture of the third frame image at the predicted quasi-focus position, and from the third frame image onwards, the lens of each subsequent frame image is pushed to the quasi-focus position in advance through predictive calculation before image capture, thereby improving the focus processing efficiency and helping to improve the clarity of the moving object in the preview picture seen by the user.

[0056] In order to better understand the embodiments of the present application, the terminal device involved in the embodiments of the present application is introduced below. Figure 1 This is a schematic diagram of the structure of the terminal device 100 provided in the embodiment of the present application. It should be understood that Figure 1 The structure shown is only exemplary and does not constitute a specific limitation on the structure of the terminal device.

[0057] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0058] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0059] In the embodiment of the present application, the terminal device 100 can implement a shooting function through the camera 193 and display it through the display screen 194.

[0060] In some possible implementations, the camera 193 includes a lens, a motor, an image sensor, an image processor, and the like. Furthermore, the image sensor may also include a microlens array, a filter, a pixel array, an optoelectronic signal converter, and other structures.

[0061] When a user takes a picture through the camera application, light is transmitted to the image sensor through the lens, and the light signal is converted into an electrical signal. The image sensor transmits the electrical signal to the image processor for processing. The image processor converts the electrical signal into a digital image signal and transmits it to the display screen 194 for image display.

[0062] Optionally, the image sensor may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor, which is not specifically limited in the present application.

[0063] Figure 2 A software structure block diagram of the terminal device 100 provided in an embodiment of the present application.

[0064] The software system of the terminal device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 100.

[0065] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: application layer, application framework layer, system runtime layer, hardware abstraction layer, and kernel layer.

[0066] The application layer may include a series of application packages.

[0067] In the embodiment of the present application, the application layer may include at least application programs such as a camera and a gallery. The terminal device 100 may respond to the user's operation on the camera application to implement functions such as taking pictures and recording videos, and the pictures and videos taken by the user may be viewed through the gallery.

[0068] The application framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions.

[0069] like Figure 2 As shown, the application framework layer may include a content provider, a resource manager, and the like.

[0070] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, etc.

[0071] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0072] The system runtime layer includes C / C++ program libraries and Android Runtime.

[0073] The C / C++ library may include multiple functional modules, such as media libraries, 3D graphics processing engine (such as OpenGL ES), 2D graphics processing engine (such as SGL), etc.

[0074] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0075] The 3D graphics processing engine is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing.

[0076] A two-dimensional graphics processing engine is a drawing engine for two-dimensional drawing.

[0077] In some possible implementations, the user may edit the captured images or videos through a media library, a three-dimensional graphics processing engine, a two-dimensional graphics processing engine, and the like.

[0078] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0079] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0080] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0081] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. In order to protect the intellectual property rights of hardware manufacturers, it hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. From the perspective of software and hardware testing, the testing of software and hardware can be completed based on the hardware abstraction layer, making it possible to conduct software and hardware testing in parallel. In layman's terms, the actions of controlling the hardware are placed in the hardware abstraction layer.

[0082] In the embodiment of the present application, the hardware abstraction layer at least includes the interface code of the camera, and the program code for performing phase detection (PD) and calculation and the program code for performing lens defocus calculation.

[0083] The Linux kernel layer is the layer between hardware and software. The Linux kernel layer contains at least camera driver, storage driver, inter-process communication driver, display driver, and sensor driver.

[0084] In an embodiment of the present application, when the touch sensor 180K receives a touch operation of the user on the camera application, the interface of the application framework layer is called to start the camera application, and then the camera driver is started by calling the Linux kernel layer, and a static image or video is captured through the camera 193. The image or video is processed and converted into a digital image signal, and the display driver is called to display it on the display screen 194 of the terminal device 100.

[0085] In a possible implementation, the terminal device stores the image or video captured by the camera 193 in response to the user clicking the capture button of the camera application interface.

[0086] Next, combine Figures 3 to 7 A focusing method for a moving object provided in an embodiment of the present application is described in detail.

[0087] It should be noted that the focusing mentioned in this application is achieved by adjusting the distance between the lens (convex lens or convex lens group) and the image sensor in the camera of the terminal device so that the image sensor continues to remain on the focal plane formed after the reflected light of the moving object is focused by the lens, which is conducive to making the image of the moving object seen by the user on the display screen of the terminal device a clear image.

[0088] The focusing method of a moving object provided in the present application can be applied to Figure 1 The terminal device 100 shown in FIG. 1 may have a software structure as follows: Figure 2 As shown, the specific forms and usage scenarios of the terminal devices shown in the embodiments shown in this application are only examples and should not constitute any limitation on the implementation of the method provided in this application.

[0089] Figure 3 A possible scenario 300 applicable to the present application is shown, in which a user calls the camera of a terminal device 100 through a camera application to shoot a moving object 301 and the surrounding environment. A preview screen of the camera is displayed on the display screen of the terminal device 100, and the screen includes the moving object 301. The rectangular selection box in the preview screen is the focus area 302 of the current shooting screen. The focus area 302 is the reference area of ​​the terminal device when focusing the lens. After the terminal device 100 performs the focusing process, the object in the focus area in the preview screen seen by the user is the clearest. The user can place the target object to be shot in the focus area so that the terminal device is not disturbed by other surrounding scenes and focuses on the object in the focus area as a reference. By way of example, in an embodiment of the present application, the target object shot by the user can be as follows Figure 3 A moving object 301 is shown.

[0090] exist Figure 3 In the scene shown, the moving object 301 is in a continuous moving state. If the lens of the terminal device is in a focused position when the first frame of the image is captured, the first frame of the image in the preview screen of the camera displayed by the terminal device at the first moment seen by the user may be a clear image of the moving object. However, if the lens of the terminal device is out of focus when the first frame of the image is captured, the first frame of the image in the preview screen of the camera displayed by the terminal device at the first moment seen by the user is a blurred image of the moving object (e.g., Figure 4 When capturing the second frame of image, the spatial position of the moving object 301 moves from the first position to the second position. When capturing the second frame of image, its lens may still be out of focus. Therefore, the second frame of image in the preview screen of the camera displayed by the terminal device at the second moment seen by the user is still a blurred image of the moving object (as shown in FIG. Figure 4 (as shown in b in the figure).

[0091] It should be understood that the first frame image and the second frame image may be two continuous frames of images or two discontinuous frames of images, and the present application does not limit this.

[0092] It should also be understood that the preview screen of the camera seen by the user is the result of the imaging light reflected by the objects in the scene being collected by the camera, converted into an image signal and presented on the display screen of the terminal device. In this process, the terminal device can record the phase formed by the reflected light of the photographed object in the phase detection pixel, and can also obtain a depth map based on the time difference between the light emitted by the laser or infrared device configured in the camera and the reflected light, and then obtain the position of the moving object. Therefore, the terminal device can obtain the first position of the moving object and the first phase corresponding to the first frame of the image in the process of collecting the first frame of the image, and can obtain the second position of the moving object and the second phase corresponding to the second frame of the image in the process of collecting the second frame of the image.

[0093] It is worth noting that the first phase of the first frame image described in the embodiment of the present application includes multiple values ​​detected by multiple phase detection pixels arranged on the image sensor when collecting the first frame image, and each phase detection pixel obtains at least two phases in different directions (for the up and down and / or left and right of a single pixel plane), that is, each phase detection pixel obtains at least one pair of phases. The first phase is usually stored in a phase buffer register (phase detection buffer, PDbuffer) for subsequent calculation of the lens defocus amount. The detection method and data format of the second phase are similar to those of the first phase and will not be repeated.

[0094] Optionally, the terminal device may calculate the phase by scheduling a phase detection library (PDLIB) algorithm or a phase detection auto focus (PDAF) algorithm, which is not limited in the present application.

[0095] Before starting to capture the third frame of image, the terminal device predicts the defocus amount of the lens when capturing the third frame of image based on the first position, the second position, the first phase corresponding to the first frame of image, and the second phase corresponding to the second frame of image, and controls the motor to push the lens to the quasi-focus position corresponding to the lens defocus amount in advance. Then the terminal device captures the third frame of image and displays the third frame of image at the third moment. Figure 4 As shown in c), the image is acquired by focusing the lens of the terminal device based on the predicted lens defocus amount. After the third frame image is acquired, no further focusing processing is required, which improves the focusing efficiency and helps the user see a clear third frame image.

[0096] It should be understood that if the camera preview screen displayed by the terminal device is to become clear as quickly as possible, the first frame image, the second frame image and the third frame image can be three consecutive frame images, and the first frame image can be the first image displayed in the preview screen after the camera is started, and before each frame image from the third frame image onwards (including the third frame image) is captured, the lens defocus amount when the next frame image data is captured can be predicted based on the image data of the previous frame, and the motor can be driven to push the lens by the predicted defocus amount before the next frame image is captured. In other words, each frame image displayed by the terminal device from the third frame image onwards (including the third frame image) is a clear image of the moving object, which realizes real-time focus tracking of the moving object.

[0097] It should also be understood that the first frame image, the second frame image and the third frame image can be three discontinuous frames of images. The first frame image and the second frame image are at least two frames of images that must be collected to ensure the display effect of the third frame image. It is not necessary to collect only two frames of images or to collect two consecutive frames of images. In some scenes with high accuracy requirements, data of multiple frames of images can be collected to predict the defocus amount of the lens when collecting the third frame of image, and this application does not limit this.

[0098] In an embodiment of the present application, after displaying the first frame image and the second frame image, the terminal device can predict the lens defocus amount when capturing the third frame image based on any one of the first phase corresponding to the first frame image and the second phase corresponding to the second frame image acquired during the capture of the first frame image and the second frame image, as well as the first position and the second position, and push the lens to the quasi-focus position before the third frame image is captured, so that the third frame image is captured based on the quasi-focus position of the lens, which is beneficial to improving the clarity of the third frame image seen by the user. The method provided in the present application improves the focusing efficiency of the terminal device by predicting the lens defocus amount and pushing the lens to the quasi-focus position in advance, and at the same time helps to improve the clarity of the preview image seen by the user, thereby improving the user experience.

[0099] Figure 5 Schematic flow chart of a method 500 for focusing a moving object provided in an embodiment of the present application. Figure 1 The terminal device 100 shown in FIG. 1 may have a software structure as follows: Figure 2 As shown, the method 500 includes the following steps:

[0100] S501. Predict a third position of the moving object according to the first position, the second position, and the time interval between the first moment and the second moment.

[0101] S502: predict the lens defocus amount according to any one of the first phase and the second phase and the third position.

[0102] S503: driving a motor to adjust the position of the lens of the terminal device according to the lens defocus amount to complete focusing.

[0103] It should be understood that the first position, the second position and the third position refer to different spatial positions of the moving object on its motion trajectory.

[0104] A possible implementation method of the above S501 includes: determining a first speed of the moving object at the second position and a second speed of the moving object at the second position according to the first position, the second position, and the time interval between the first moment and the second moment; predicting a third speed of the moving object when it is at the third position based on the first position and the first speed, the second position and the second speed; and calculating the third position based on any one of the first position and the second position and the third speed using a preset motion equation, where the preset motion equation is a relationship equation between the position, speed, and motion time of the moving object.

[0105] It should be understood that the first position and the second position can be represented by three-dimensional coordinates in the camera coordinate system. The first position of the moving object can be represented by multiple coordinates, and the multiple coordinates correspond to multiple points on the moving object. The second position can also be represented by multiple coordinates, and the multiple coordinates of the second position should correspond one-to-one with the multiple coordinates of the first position.

[0106] It should also be understood that when exposing the pixel array in the image sensor, it always starts from the upper left corner of the image and is exposed from left to right and from top to bottom. That is, when the exposure reaches the right edge from the upper left corner of the image, it quickly reaches the left side of the next row and starts exposing the pixels in the next row. The time interval from the right side of the previous row to the left side of the next row is called horizontal blanking (h blank). A complete image is called a frame. After the exposure of a frame of image is completed, it is necessary to return from the lower right corner of the image to the upper left corner of the image to start scanning a new frame. This time interval is called vertical blanking, also known as field blanking (v blank). The process of exposing the pixel array is the process of acquiring images (it can also be said to be acquiring image data). When the frame rate is constant, the time interval from the start of capturing a frame (SOF) to the end of capturing a frame (EOF) is fixed, and the time of field blanking (v blank) is also fixed, so when capturing the first frame and the second frame, the time interval between the start frame and the end frame is fixed, and is equal to the time interval between displaying the first frame and the second frame (that is, the time interval between the first moment and the second moment). Therefore, the displacement of the moving object can be obtained based on the first position and the second position that have been obtained, and the average speed can be obtained based on the time interval based on the two frames of image acquisition. The first speed and the second speed can be obtained by finding the limit, and then the acceleration of the moving object in the time interval is obtained, and the speed of the moving object at the third position after the time interval is predicted based on the acceleration, and then multiple coordinates of the third position are obtained based on the preset motion equation.

[0107] Exemplarily, taking the three-dimensional coordinate system OXYZ and a point P on a moving object as an example, the speed calculation of the point P at various positions is explained. When the moving object is at the first position at time t1, the first coordinate P1 of the point P can be expressed as P1(x1, y1, z1). At time t2, the moving object has moved from the first position to the second position. The coordinate P2 of the point P on the moving object when it is at the second position can be expressed as P2(x2, y2, z2). The time interval Δt for the moving object to move from the first position to the second position is t2-t1. By calculating the displacement of the point P on the moving object in various directions in the OXYZ coordinate system within the time Δt, the speed of the moving object point P in various directions is calculated, and the speed of the point P at time t1 is obtained by finding the limit. and the speed at time t2 And through the speed at time t1 and the speed at time t2 Calculate the acceleration of the moving object at any time within the Δt period, and calculate the moving speed at time t3 based on the acceleration A plurality of coordinates representing the third position of the moving object at time t3 are calculated by using a preset motion equation, wherein the preset motion equation is a relationship equation between the position, speed and moving time of the moving object.

[0108] In a possible implementation, the expression of the preset motion equation is as follows:

[0109]

[0110] in, is the coordinate of point P on the moving object at time t1.

[0111] Optionally, the first position, second position, third position, etc. of the moving object can be represented by the coordinates of a point on the moving object, or can be represented by the central coordinates of the moving object calculated from multiple coordinates of each position, and this application is not limited to this.

[0112] Possible implementations of the above S501 also include: based on the first position, the second position, and the time interval between the first moment and the second moment, identifying the third position of the moving object through a trained neural network model. It should be understood that the trained neural network model is obtained by training a preset neural network model with a large amount of object motion trajectory data, and the motion trajectory of the moving object can be predicted based on partial position and time data, thereby obtaining the third position on the trajectory.

[0113] Optionally, the preset neural network model can be a convolutional neural network (CNN), a feedforward (BP) neural network, or any other neural network that can be trained to predict the trajectory of a moving object, and this application does not make any specific limitations on this.

[0114] Optionally, the speed at time t1 has been obtained above and the speed at time t2 The movement speed at time t3 can also be predicted based on the trained neural network model. This is to reduce the computing power burden of the terminal device, and this application does not impose any restrictions on this.

[0115] A possible implementation method of the above S502 includes: determining the fourth position of the moving object in the image coordinate system according to the third position and the intrinsic parameter matrix and the extrinsic parameter matrix of the camera, and taking the area including the fourth position as the first focus area of ​​the third frame image; multiplying the first phase difference formed by the phase detection pixels corresponding to the first focus area in the first phase, or the second phase difference formed by the phase detection pixels corresponding to the first focus area in the second phase, by the defocus transformation coefficient to calculate the lens defocus amount.

[0116] It should be understood that the three-dimensional coordinates of the moving object in the camera coordinate system can be converted into two-dimensional coordinates in the image coordinate system corresponding to the camera of the terminal device based on the intrinsic parameter matrix and the extrinsic parameter matrix of the camera. Therefore, the focus area that can be referenced when collecting the third frame image can be predetermined according to the fourth position of the moving object in the image coordinate system, and the area including the fourth position is used as the first focus area referenced when focusing the third frame image. Since the position of the phase detection pixels set in the image sensor in its pixel array is fixed, and the coordinate system formed by its pixel array (referred to as the pixel coordinate system) can be converted to each other with the image coordinate system, the phase detection pixels included in the first focus area including the fourth position can be determined according to the fourth position of the moving object in the image coordinate system, and then one or more pairs of phases of at least one phase detection pixel in the first focus area are selected from multiple values ​​of the first phase. Based on the one or more pairs of phases, the first phase difference is calculated by a frequency domain algorithm or a space domain algorithm. The first phase difference is a value. Similarly, the second phase difference can also be calculated, which will not be repeated.

[0117] Then, the lens defocus amount when the third frame image is captured is calculated according to the relationship between the first phase difference (or the second phase difference) and the lens defocus amount and the defocus conversion coefficient (DDC): lens defocus amount = phase difference * DDC. Wherein, DDC represents the coefficient of the linear relationship between the drive current of the motor controlling the lens and the lens movement position.

[0118] It should be understood that the time it takes for the motor to drive the lens from the out-of-focus position to move the lens defocus amount and stabilize at the in-focus position is the motor stabilization time. In the embodiment of the present application, the time interval from the start of the second frame image acquisition to the start of the third frame image acquisition includes the motor stabilization time.

[0119] In a possible implementation, since the second moment is after the first moment and the third moment is after the second moment, it can be considered that the second phase obtained at the second moment is closer to the phase when the third frame of image is collected, and therefore, the second phase difference can be selected to perform the prediction calculation of the lens defocus amount when the third frame of image is collected. Moreover, in the prediction calculation of subsequent frames, the phase of the previous frame closest to the frame to be predicted can be selected to calculate the lens defocus amount.

[0120] In an embodiment of the present application, the first focus area when capturing the third frame of the image is determined by predicting the third position of the moving object when capturing the third frame of the image, and then the phase difference formed with the phase detection pixel in the first focus area is selected from the first phase or the second phase to calculate the lens defocus amount. Compared with the calculation result obtained by directly using the first phase corresponding to the first frame of the image or the second phase corresponding to the second frame of the image, the calculation result obtained in this way ensures that the lens of the terminal device is in a favorable position for imaging the moving object, which is conducive to making the moving object in the first focus area in the third frame of the image clearer, so as to achieve the purpose of the user tracking the moving object.

[0121] As an optional embodiment, after the defocus amount of the lens has been predicted and before executing the above S503, the relative position relationship between the imaging plane and the focal plane may be determined first. Figure 6 As shown in the figure, the M2 plane is the plane where the imaging plane of the image sensor is located when the terminal device captures the object W and successfully focuses, which is called the focal plane. When the imaging plane is located at M2, the imaging light reflected from the object W to the lens L in different directions converges on M2, that is, the reflected light of the object W is imaged at the same position on the image sensor, and the image of the image sensor is clear at this time. Considering that before executing the above S503, the imaging plane of the image sensor may be Figure 6 As shown in M1 or M3, it is necessary to determine a first distance between the imaging plane and the lens, and a second distance between the focal plane and the lens, and when the first distance is less than the second distance, drive the motor to control the lens to move the lens away from the imaging plane by the lens defocus amount to complete the focusing; when the first distance is greater than the second distance, drive the motor to control the lens to move the lens toward the imaging plane by the lens defocus amount to complete the focusing.

[0122] In the embodiment of the present application, before the motor drives the lens to move, the movement direction is first determined based on the first distance between the imaging plane and the lens and the second distance between the focal plane and the lens, which is conducive to moving the lens to the target position more quickly and accurately.

[0123] As an optional embodiment, when the terminal device obtains the first position of the moving object in the first frame image, the moving object is in the second focus area of ​​the first frame image.

[0124] It should be understood that in the embodiment of the present application, the purpose of the user's shooting is to focus on a moving object. When the first frame image is used as the first image captured in the current scene, the moving object is placed in the second focus area of ​​the first frame image, which is beneficial for the terminal device to quickly determine the moving object that needs to be focused through the second focus area, and then determine the first position of the moving object.

[0125] In a possible implementation, the second focus area is a default focus area of ​​the camera, and the default focus area is located in the center area of ​​the preview image.

[0126] In another possible implementation, the second focus area is an area selected by the terminal device in response to a click operation of the user.

[0127] It should be understood that the user clicks on the preview image to place the target moving object in the second focus area.

[0128] In an embodiment of the present application, the moving object is in the second focus area of ​​the first frame image, which is conducive to the terminal device quickly finding the moving object when obtaining the first position, and ignoring other objects in the current picture, only obtaining the position of the moving object, saving the computing power of the terminal device.

[0129] As an optional embodiment, when the terminal device obtains the second position of the moving object in the second frame image, it can identify the moving object in the above-mentioned second focus area based on an image recognition algorithm, and obtain the second position of the moving object through depth camera algorithm, binocular ranging algorithm (requires at least two cameras), infrared or laser ranging and other methods.

[0130] In an embodiment of the present application, after the first frame image is captured and it is known that the object in the focus area of ​​the first frame image is the moving object to be focused, when the second frame image is captured, the object is identified based on the image recognition algorithm, and only the position of the object can be obtained, thereby reducing the amount of calculation of the terminal device and facilitating rapid determination of the second position of the moving object.

[0131] Figure 7 The present invention provides a schematic flow chart of a method 700 for focusing a moving object according to an embodiment of the present invention. The method 700 comprises the following steps:

[0132] S701 , capturing a first frame image, recording a first phase corresponding to the first frame image, acquiring a first position, and displaying the first frame image at a first moment.

[0133] S702 , capturing a second frame of image, recording a second phase corresponding to the second frame of image, acquiring a second position, and displaying the second frame of image at a second moment.

[0134] It should be understood that during the acquisition process of the first frame image and the second frame image, the first phase corresponding to the first frame image and the second phase corresponding to the second frame image are recorded respectively, and during the acquisition process of the first frame image and the second frame image, the first position and the second position of the moving object are acquired based on the depth camera algorithm, binocular ranging algorithm, infrared or laser ranging algorithm and other algorithms.

[0135] S703 : predicting the lens defocus amount when acquiring the third frame of image based on any one of the first phase and the second phase and the first position and the second position.

[0136] S704 , driving the motor to adjust the position of the lens according to the predicted lens defocus amount to complete focusing.

[0137] In a possible implementation, the calculations of S703 and S702 may be completed during field blanking or vertical blanking (vblank) time to ensure that the focus is completed at the beginning of the third frame.

[0138] S705 , capturing a third frame of image, and displaying the third frame of image at a third moment.

[0139] It should be understood that the third frame image is acquired by capturing the lens position after focusing based on the predicted lens defocus amount, which is beneficial for making the moving object in the third frame image seen by the user clear.

[0140] It should also be understood that after 705, the terminal device can repeat the calculations corresponding to S703 and S702 based on the third phase corresponding to the third frame image recorded when the third frame image is captured and the third position obtained, so that the moving objects in subsequent images can achieve the effect of the above-mentioned third frame image.

[0141] Combination of the above Figures 1 to 7 , describes in detail the focusing method of a moving object provided by an embodiment of the present application, and will now be combined with Figure 8 and Fig. 9 , a focusing device for a moving object provided in an embodiment of the present application is described in detail.

[0142] Figure 8A moving object focusing device 800 is provided in an embodiment of the present application. The device 800 includes: a display module 801 and a processing module 802. The display module 801 is used to: at a first moment, display a first frame image on a preview screen, the first frame image includes a moving object at a first position; at a second moment, display a second frame image on a preview screen, the second frame image includes a moving object at a second position, and the second moment is after the first moment; at a third moment, display a third frame image on a preview screen, the third frame image is acquired after focusing the lens of the terminal device using the predicted lens defocus amount, the lens defocus amount is obtained based on any one of the first phase corresponding to the first frame image and the second phase corresponding to the second frame image, as well as the first position and the second position, and the third moment is after the second moment.

[0143] Optionally, the processing module 802 is used to: predict the third position of the moving object based on the first position, the second position, and the time interval between the first moment and the second moment; predict the lens defocus amount based on any one of the first phase and the second phase and the third position; and drive the motor to adjust the position of the lens of the terminal device according to the lens defocus amount to complete focusing.

[0144] Optionally, the processing module 802 is specifically used to: determine a first speed of the moving object at the first position and a second speed of the moving object at the second position based on the first position, the second position, and the time interval between the first moment and the second moment; predict a third speed of the moving object when it is at a third position based on the first position and the first speed, the second position and the second speed; calculate the third position based on any one of the first position and the second position and the third speed using a preset motion equation, where the preset motion equation is a relationship equation between the position, speed, and motion time of the moving object.

[0145] Optionally, the processing module 802 is also specifically used to: determine the fourth position of the moving object in the image coordinate system according to the third position and the intrinsic parameter matrix and the extrinsic parameter matrix of the camera, and use the area including the fourth position as the first focus area of ​​the third frame image; multiply the first phase difference or the second phase difference by the defocus transformation coefficient to calculate the lens defocus amount, the first phase difference being the phase difference formed by the phase detection pixels corresponding to the first focus area in the first phase, and the second phase difference being the phase difference formed by the phase detection pixels corresponding to the first focus area in the second phase.

[0146] Optionally, the processing module 802 is also specifically used to: determine the relative position relationship between the imaging plane and the focal plane of the camera; determine a first distance between the imaging plane and the lens, and a second distance between the focal plane and the lens; and, when the first distance is less than the second distance, drive the motor to control the lens to move the lens defocus amount in the direction away from the imaging plane to complete focusing.

[0147] Optionally, the processing module 802 is further specifically configured to: when the first distance is greater than the second distance, drive the motor to control the lens to move the lens defocus amount in a direction close to the imaging plane to complete focusing.

[0148] Optionally, the moving object is in a second focus area of ​​the first frame image.

[0149] Optionally, the second focus area is a default focus area.

[0150] Optionally, the second focus area is an area selected by the terminal device in response to a click operation of the user.

[0151] Fig. 9 A schematic block diagram of a moving object focusing device 900 provided in an embodiment of the present application. The device 900 includes a processor 901, a communication interface 902, and a memory 903. The processor 901, the communication interface 902, and the memory 903 communicate with each other through an internal connection path, the memory 903 is used to store instructions, and the processor 901 is used to execute the instructions stored in the memory 903. The communication interface 902 can be used to send signals to other devices (such as the processor 901 or the touch screen of the terminal device), and can also be used to receive signals from other devices (such as the memory 903). Exemplarily, the communication interface 902 reads the instructions stored in the memory 903 and sends the instructions to the processor 901.

[0152] It should be understood that the device 900 can be specifically a terminal device in the above embodiment, and can be used to execute the various steps and / or processes corresponding to the terminal device in the above method embodiment. Optionally, the memory 903 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 901 may be used to execute instructions stored in the memory, and when the processor 901 executes instructions stored in the memory, the processor 901 is used to execute the various steps and / or processes of the above method embodiment.

[0153] The present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the method corresponding to the terminal device in the above embodiment.

[0154] The present application also provides a computer program product, which includes a computer program (also referred to as code or instruction). When the computer program runs on a computer, the computer can execute the method corresponding to the terminal device shown in the above embodiment.

[0155] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0157] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0158] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0160] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0161] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, which should be included in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Claims

1. A method for focusing a moving object, characterized in that: Applied to a terminal device, the camera of the terminal device is in an on state and the preview image of the camera includes a moving object, the method includes: At a first moment, displaying a first frame of image on the preview screen, wherein the first frame of image includes the moving object at a first position; At a second moment, displaying a second frame of image on the preview screen, the second frame of image including the moving object at a second position, the second moment being after the first moment; At a third moment, a third frame image is displayed on the preview screen, and the third frame image is acquired after focusing the lens of the terminal device using the predicted lens defocus amount, and the lens defocus amount is acquired based on any one of the first phase corresponding to the first frame image and the second phase corresponding to the second frame image, as well as the first position and the second position, and the third moment is after the second moment.

2. The method according to claim 1, characterized in that The method further comprises: predicting a third position of the moving object according to the first position, the second position, and a time interval between the first moment and the second moment; predicting the lens defocus amount according to any one of the first phase and the second phase and the third position; The motor is driven according to the lens defocus amount to adjust the position of the lens of the terminal device to complete the focusing.

3. The method according to claim 2, characterized in that The predicting the third position of the moving object according to the first position, the second position, and the time interval between the first moment and the second moment includes: Determine a first speed of the moving object at the first position and a second speed of the moving object at the second position according to the first position, the second position, and a time interval between the first moment and the second moment; predicting a third speed of the moving object at the third position based on the first position and the first speed, the second position and the second speed; Based on any one of the first position and the second position and the third speed, the third position is calculated using a preset motion equation, where the preset motion equation is a relationship equation between the position, speed, and motion time of the moving object.

4. The method according to claim 2 or 3, characterized in that: The predicting the lens defocus amount according to any one of the first phase and the second phase and the third position comprises: Determine a fourth position of the moving object in the image coordinate system according to the third position and an intrinsic parameter matrix and an extrinsic parameter matrix of the camera, and use an area including the fourth position as a first focus area of ​​the third frame image; The product obtained by multiplying the first phase difference or the second phase difference by the defocus transformation coefficient is determined as the lens defocus amount, the first phase difference is the phase difference formed by the phase detection pixels corresponding to the first focus area in the first phase, and the second phase difference is the phase difference formed by the phase detection pixels corresponding to the first focus area in the second phase.

5. The method according to any one of claims 2 to 4, characterized in that Before the step of driving a motor to adjust the position of the lens of the terminal device according to the lens defocus amount, the method includes: Determining the relative position relationship between the imaging plane and the focal plane of the camera; Determine a first distance between the imaging plane and the lens, and a second distance between the focal plane and the lens; The step of driving a motor to adjust the position of the lens of the terminal device according to the lens defocus amount to complete focusing includes: When the first distance is smaller than the second distance, the driving motor controls the lens to move the lens defocus amount in a direction away from the imaging plane to complete focusing.

6. The method according to claim 5, characterized in that The method further comprises: When the first distance is greater than the second distance, the driving motor controls the lens to move the lens defocus amount in a direction close to the imaging plane to complete focusing.

7. The method according to any one of claims 1 to 6, characterized in that The moving object is in a second focus area of ​​the first frame image.

8. The method according to claim 7, characterized in that The second focus area is a default focus area.

9. The method according to claim 7, characterized in that: The second focus area is an area selected by the terminal device in response to a click operation by the user.

10. A focusing device for a moving object, characterized in that: The method comprises means for implementing the method as claimed in any one of claims 1 to 9.

11. A terminal device, characterized in that: include: A processor, the processor is coupled to a memory, the memory is used to store a computer program, when the processor calls the computer program, the terminal device executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 9.

Images