Focus control method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a focus control method and device, electronic equipment and storage medium. The focus control method comprises: obtaining a first focus region of interest of a first reference frame and a second focus region of interest of a second reference frame; generating a prediction region of a current frame according to the first focus region of interest and the second focus region of interest; determining a phase difference value (PD value) of the current frame according to the prediction region; and focusing in the current frame according to the PD value. According to the focus control method of the present disclosure, the problem of focus blur caused by frame delay can be overcome, and the focus effect can be optimized.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of cameras, and particularly relates to a focus control method and device, electronic equipment and storage medium. BACKGROUND

[0002] The current camera generally adopts continuous focusing when shooting moving objects, and generally searches the focusing area by single-point focusing or multi-point focusing when focusing. For example, the prior art improves the focusing effect by predicting the focusing position of the focused lens for a specific frame, specifically by a processor executing a first focusing position of a focused lens in a specific frame of a main exposure in a continuous shooting period, and referencing the first focusing position to predict a second focusing position of the focusing lens relative to a specific object in a plurality of frames in front of the specific frame in the continuous shooting period, and moving the focusing lens towards the second focusing position. This focusing technology mainly detects the focusing area, and there is a problem of focusing blur caused by frame delay. SUMMARY

[0003] The present disclosure provides a focus control method and device, electronic equipment and storage medium, which mainly aims to overcome the problem of focusing blur caused by frame delay and optimize the focus effect.

[0004] According to a first aspect of the present disclosure, a focus control method is provided, comprising:

[0005] obtaining a first focus region of interest of a first reference frame, and obtaining a second focus region of interest of a second reference frame;

[0006] generating a prediction region of a current frame according to the first focus region of interest and the second focus region of interest;

[0007] determining a phase difference value (PD value) of the current frame according to the prediction region;

[0008] focusing in the current frame according to the PD value.

[0009] In an embodiment of the present disclosure, an update region of the current frame is generated according to the first focus region of interest, wherein the PD value of the current frame is determined according to the prediction region and the update region.

[0010] In an embodiment of the present disclosure, the generating of the prediction region of the current frame according to the first focus region of interest and the second focus region of interest comprises inputting the first focus region of interest and the second focus region of interest into a self-optimization algorithm model to generate the prediction region of the current frame.

[0011] In an embodiment of the present disclosure, the generating the update region of the current frame according to the first focus interest region comprises: obtaining a third focus interest region of a third reference frame, wherein the third reference frame is a previous frame of the first reference frame; generating a change reference amount according to the first focus interest region of the first reference frame and the third focus interest region of the third reference frame; and generating the update region according to the first focus interest region and the change reference amount.

[0012] In an embodiment of the present disclosure, the change reference amount comprises a first change reference amount and a second change reference amount, wherein the first change reference amount is:

[0013] ratioX = (endX1 - startX0) / (endX1 - startX1)

[0014] The second change reference amount is:

[0015] ratioY = (endY0 - startY1) / (endY1 - startY1)

[0016] In the formula, startX0 is a starting horizontal coordinate of the third focus interest region, startX1 is a starting horizontal coordinate of the first focus interest region, endX1 is an ending horizontal coordinate of the first focus interest region, endY1 is an ending vertical coordinate of the first focus interest region, endY0 is an ending vertical coordinate of the third focus interest region, and startY1 is a starting vertical coordinate of the first focus interest region.

[0017] In an embodiment of the present disclosure, the determining the PD value of the current frame according to the prediction region and the update region comprises: obtaining an overlapping region between the prediction region and the update region, calculating a PD value of the overlapping region, and taking the PD value as the PD value of the current frame.

[0018] In an embodiment of the present disclosure, the above-mentioned focus control method comprises that the current frame is an Nth image frame, the first reference frame is an (N-2)th image frame, and the second reference frame is an (N-1)th image frame.

[0019] According to the second aspect embodiment of the present disclosure, a focus control device is further provided, comprising:

[0020] The obtaining module is configured to obtain a first focus interest region of a first reference frame and a second focus interest region of a second reference frame.

[0021] The prediction module is configured to generate a prediction region of a current frame according to the first focus interest region and the second focus interest region.

[0022] a calculation module configured to determine a phase difference value (PD value) of the current frame according to the prediction region;

[0023] a focusing module configured to focus in the current frame according to the PD value.

[0024] In an embodiment of the present disclosure, the device comprises an update region generation module configured to generate an update region of the current frame according to the first focus region of interest; and the calculation module is specifically configured to determine a PD value of the current frame according to the prediction region and the update region.

[0025] In an embodiment of the present disclosure, the prediction module is specifically configured to input the first focus region of interest and the second focus region of interest into a self-recurrence optimization algorithm model to generate the prediction region of the current frame.

[0026] In an embodiment of the present disclosure, the update region generation module comprises a third reference frame acquisition unit configured to acquire a third focus region of interest of a third reference frame, wherein the third reference frame is a previous frame of the first reference frame; a change reference amount calculation unit configured to generate a change reference amount according to a first focus region of interest of the first reference frame and a third focus region of interest of the third reference frame; and a region generation unit configured to generate the update region according to the first focus region of interest and the change reference amount.

[0027] In an embodiment of the present disclosure, the change reference amount comprises a first change reference amount and a second change reference amount,

[0028] the first change reference amount is:

[0029] ratioX = (endX1 - startX0) / (endX1 - startX1)

[0030] the second change reference amount is:

[0031] ratioY = (endY0 - startY1) / (endY1 - startY1)

[0032] wherein startX0 is a starting horizontal coordinate of the third focus region of interest, startX1 is a starting horizontal coordinate of the first focus region of interest, endX1 is an ending horizontal coordinate of the first focus region of interest, endY1 is an ending vertical coordinate of the first focus region of interest, endY0 is an ending vertical coordinate of the third focus region of interest, and startY1 is a starting vertical coordinate of the first focus region of interest.

[0033] In one embodiment of the present disclosure, the calculation module is specifically configured to: acquire an overlapping region between the prediction region and the update region, calculate a PD value of the overlapping region, and take the PD value as the PD value of the current frame.

[0034] In one embodiment of the present disclosure, the control device for focusing includes that the current frame is an Nth image frame, the first reference frame is an N-2th image frame, and the second reference frame is an N-1th image frame.

[0035] According to a third aspect of the present disclosure, an electronic device is also provided, which includes at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method for focusing according to the first aspect of the present disclosure.

[0036] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to enable the computer to perform the control method for focusing according to the first aspect of the present disclosure.

[0037] According to a fifth aspect of the present disclosure, a computer program product is also provided, which includes a computer program, and the computer program, when executed by a processor, implements the control method for focusing according to the first aspect of the present disclosure.

[0038] In one or more embodiments of the present disclosure, a first focus region of interest of the first reference frame is acquired, and a second focus region of interest of the second reference frame is acquired; a prediction region of the current frame is generated according to the first focus region of interest and the second focus region of interest; a phase difference value PD value of the current frame is determined according to the prediction region; and focusing is performed in the current frame according to the PD value. Therefore, the prediction region of the current frame is obtained through the first focus region of interest and the second focus region of interest corresponding to the first reference frame and the second reference frame, the phase difference value PD value of the prediction region is determined, and then the focusing is realized. Thus, the AF focusing effect can be optimized by reducing frame delay, the out-of-focus problem can be reduced, and the user's photographing experience can be improved.

[0039] Additional aspects and advantages of the present disclosure will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0040] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0041] Figure 1A background diagram of a control method of focusing provided by an embodiment of the present disclosure is shown.

[0042] Figure 2 A flow diagram of a first control method of focusing provided by an embodiment of the present disclosure is shown.

[0043] Figure 3 A flow diagram of a second control method of focusing provided by an embodiment of the present disclosure is shown.

[0044] Figure 4 A flow diagram of a detection method of a target detection network model provided by an embodiment of the present disclosure is shown.

[0045] Figure 5 A flow diagram of an update region generation method provided by an embodiment of the present disclosure is shown.

[0046] Figure 6 A shape diagram of an update region provided by an embodiment of the present disclosure is shown.

[0047] Figure 7 A structural block diagram of a first control device of focusing provided by an embodiment of the present disclosure is shown.

[0048] Figure 8 A structural block diagram of a second control device of focusing provided by an embodiment of the present disclosure is shown.

[0049] Figure 9 A structural block diagram of a third control device of focusing provided by an embodiment of the present disclosure is shown.

[0050] Figure 10 A block diagram of an electronic device for implementing the control method of focusing provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0051] The exemplary embodiments will be described in detail herein below with reference to the drawings. The following description is related to the drawings when the drawings are referred to, and the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0052] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0053] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified. It should also be understood that the term "and / or" used in the present disclosure means and includes any or all possible combinations of one or more associated listed items.

[0054] The embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0055] With the development of science and technology and the improvement of people's living standards, people's demand for electronic products increases, and the functional requirements for electronic products are also increasingly high. For example, when a user needs to take an image, the user relies on a camera and hopes that the camera can clearly capture the object in a static and dynamic state.

[0056] Figure 1 A background diagram of a control method for focusing provided by an embodiment of the present disclosure is shown. As shown in Figure 1 When a user needs to take an image, the user enters the camera interface of the terminal, can switch to the required camera by clicking the conversion key of the camera interface of the terminal, click the shooting key to take a picture, and the picture taken is saved in the gallery. When shooting, the terminal can automatically focus, but the picture taken is not always clear, especially when the object being photographed is a dynamic object, resulting in poor user experience.

[0057] According to some embodiments, in the process of shooting a fast-moving object, the camera needs to automatically determine which area or focus point to use for automatic focusing. In order to obtain the focus of interest (ROI) area and the phase difference (PD) value of the ROI area, the camera needs to use the data of the first two frames to calculate the PD value of the ROI area of the current frame, wherein the first frame obtains the ROI, the second frame configures the AF ROI, and the third frame obtains the PD value of the ROI area. However, this processing method will have a delay of 2 frames in each focusing process, which will lead to a probabilistic out-of-focus problem, especially for high-speed moving objects, the focus blur problem will be more serious. Therefore, reducing the frame delay in the focus process is of great significance to improve the focus effect.

[0058] The present disclosure will be described in detail below with reference to specific embodiments.

[0059] In the first embodiment, as shown in Figure 2 , Figure 2 a flowchart of a first focus control method provided by the embodiments of the present disclosure is shown. The method can be implemented by relying on a computer program and can be run on a control device for focus processing. The computer program can be integrated in an application or run as an independent tool application.

[0060] The control device can be a terminal with a camera, which includes but is not limited to a wearable electronic device, a handheld electronic device, a personal computer, a tablet computer, a vehicle-mounted electronic device, a smart phone, a computing electronic device, or other processing electronic devices connected to a wireless modem, etc. The terminal can be called by different names in different networks, such as a user electronic device, an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile electronic device, a user terminal, a terminal, a wireless communication electronic device, a user agent or a user device, a cellular phone, a cordless phone, a personal digital assistant (PDA), a terminal in a 5th Generation Mobile Communication Technology (5G) network, a 4th Generation Mobile Communication Technology (4G) network, a 3rd-Generation (3G) network, or a future evolution network, etc. The camera can have video shooting and still image capture functions. When the camera captures an image through a lens, the image is processed and converted into a digital signal recognizable by the terminal by the photosensitive component circuit and control component in the camera, and then input into the processor of the terminal through the parallel port for image restoration.

[0061] Specifically, the control method of focusing includes:

[0062] S101, obtaining a first focus region of interest of a first reference frame, and obtaining a second focus region of interest of a second reference frame.

[0063] In some embodiments, the first reference frame and the second reference frame are images obtained after the terminal controls the camera to shoot and the camera inputs the image into the terminal for image restoration, i.e., images obtained by the camera module.

[0064] In some embodiments, the first reference frame and the second reference frame refer to images on the imaging field of view of the camera, and do not refer to images displayed on the display interface of the terminal. For example, when the shooting position changes, the first reference frame and the second reference frame will change accordingly. When the shooting angle changes, the first reference frame and the second reference frame will also change accordingly. When the camera model changes, the first reference frame and the second reference frame will also change accordingly.

[0065] As can be easily understood, the focus region of interest (ROI) is a selected image region from an image, which is the focus of subsequent image analysis. The region is circled for further processing.

[0066] Optionally, the first focus region of interest is a region circled from the first reference frame, and the second focus region of interest is a region circled from the second reference frame, for example, when the first reference frame or the second reference frame changes, the corresponding first focus region of interest or the second focus region of interest also changes accordingly. Wherein, the first focus region of interest can be represented as ROI1, and the second focus region of interest can be represented as ROI2.

[0067] S102, generating a prediction region of the current frame according to the first focus region of interest and the second focus region of interest.

[0068] In some embodiments, the first reference frame is a previous frame of the second reference frame, and the second reference frame is a previous frame of the current frame. The current frame is an image obtained after the terminal inputs the image captured by the camera to the terminal for image restoration, that is, an image obtained by the camera module. The current frame refers to the image on the imaging field of view of the camera, and does not refer to the image displayed on the display interface of the terminal.

[0069] In some embodiments, the prediction region refers to the focus region of interest of the current frame predicted by the first focus region of interest and the second focus region of interest. Wherein, the prediction region can be represented as ROI 预测 .

[0070] S103, determining a phase difference value PD value of the current frame according to the prediction region.

[0071] It is easy to understand that the phase difference (Phase Difference, PD) value is the phase difference of two images. Specifically, when the terminal images the object, through the special pixel points in the chip itself, a "left eye imaging image" and a "right eye imaging image" can be formed similar to the function of the human eye, and the PD value is obtained by comparing the phase difference of the two images.

[0072] S104, focusing in the current frame according to the PD value.

[0073] In some embodiments, the minimum PD value is considered as the point of clear focus, and after the terminal calculates the PD value, the terminal can generate a focus control instruction based on the query result by querying the table, and then realize fast focusing.

[0074] Please refer to Figure 3 , Figure 3 a flowchart of a second focus control method provided by the embodiments of the present disclosure is shown.

[0075] S201, obtaining a first focus region of interest of a first reference frame, and obtaining a second focus region of interest of a second reference frame.

[0076] The specific process is as above, which is not repeated here.

[0077] It is easy to understand that for the ROI region, in machine vision and image processing, the region is generally outlined in the form of a box, a circle, an ellipse, an irregular polygon, etc. from the processed image, and various operators and functions are commonly used on machine vision software such as Halcon, OpenCV, Matlab, etc. to obtain the region. After enabling the ROI function, the ROI region will be encoded with high quality and lossless, while the regions outside the ROI will be compressed with standard definition video, even without transmitting the video of the regions, to save network bandwidth occupation and video storage space, reduce processing time, and increase precision.

[0078] In some embodiments, in the process of shooting the target object (especially a moving object), the position and speed information of the target object in the reference frames (e.g., the first reference frame and the second reference frame) are obtained in real time by using a target detection technology, and then an AF (Automatic Focus) module is used to frame the focus area of the foreground target object (i.e., the moving target) in the first two reference frames based on the detection results, so as to calculate the ROI1 area of the first reference frame and the ROI2 area of the second reference frame.

[0079] It is easy to understand that the target detection technology is implemented by a target detection network model. Target detection is to find all objects of interest in an image, which includes object positioning and object classification, and determines the class and position of the object. A commonly used target detection network model is a deep learning-based target detection algorithm. The deep learning-based target detection algorithm can be roughly divided into two categories: the first category is One-Stage target detection algorithm, which does not need a Region Proposal stage and can directly output the class probability and position coordinate value of the object through one stage. Typical algorithms include YOLO, SSD, and CornerNet. The second category is Two-Stage target detection algorithm, which divides the detection problem into two stages. The first stage generates candidate regions (Region Proposals) containing the approximate position information of the target, and the second stage classifies and refines the position of the candidate regions. Typical representatives of this algorithm include R-CNN, Fast R-CNN, Faster R-CNN, etc. The main performance indicators of the target detection network model are detection accuracy and speed, among which the accuracy mainly considers the positioning and classification accuracy of the object.

[0080] In some embodiments, as shown in FIG. 2A, Figure 4 Figure 4 ​A flowchart of a detection method of a target detection network model provided by an embodiment of the present disclosure is shown. The detection method of the target detection network includes collecting a preview frame image through a camera; then pre-processing the image, detecting the position and area of the moving target of the image through the target detection network; then determining whether it is a first reference frame, if yes, recording the position and related parameters of the current frame, and waiting for the next frame; if not, entering step S202 to predict the position and area of the next frame and output the position and parameters of the target in the next frame according to the position and speed information of the previous two frames.

[0081] S202, generating a predicted area of the current frame according to the first focus region of interest and the second focus region of interest.

[0082] The specific process is as above, which will not be repeated here.

[0083] In some embodiments, if the current frame is the Nth image frame, the first reference frame is the (N-2)th image frame, and the second reference frame is the (N-1)th image frame.

[0084] According to some embodiments, the first focus region of interest and the second focus region of interest are input into an autoregressive optimization algorithm model to generate the predicted area ROI of the current frame. 预测 .

[0085] It is easy to understand that the autoregressive optimization algorithm model can use the position and speed information of the previous frame to predict the position and speed information of the next frame, and the system model and the observation model are as follows:

[0086] X k =F k-1 X k-1 +G k-1 W k-1

[0087] wherein F k-1 represents a linear state transition matrix, G k-1 represents an input matrix, W k-1 represents a system input, X k-1 represents the motion target information in the previous frame, and X k represents the motion target information in the next frame. The model obtains the predicted motion state variable of the motion target by obtaining the speed and coordinates of the motion target in the previous frame, wherein X k =[x(k),y(k),s(k),r(k),v x (k),v y (k)]. Wherein x(k) and y(k) represent the position of the motion target in the x-axis and y-axis in the next frame, respectively, v x (k),v y(k) represents the velocity of the moving target on the x-axis and y-axis in the next frame, respectively; s(k) represents the area of ​​the detection box in the next frame; and r(k) represents the ratio of the width to the height of the detection box in the next frame. x (k) and v y (k) is represented as follows:

[0088]

[0089]

[0090] x(k-1) and y(k-1) represent the positions of the moving target on the x-axis and y-axis in the previous frame, respectively, and Δt represents the time interval between the previous frame and the next frame. k-1 =F k F k It can be represented as:

[0091]

[0092] In some embodiments, the autoregressive optimization algorithm model employs, for example, a Kalman filter, which can accurately predict the target's position and velocity.

[0093] S203, Generate the updated region of the current frame based on the first focused region of interest.

[0094] In some embodiments, such as Figure 5 and Figure 6 As shown, Figure 5 This diagram illustrates a flowchart of the update region generation method provided in an embodiment of this disclosure. Figure 6 A schematic diagram of the shape of the updated region provided in an embodiment of this disclosure is shown. The AF module can generate the updated region of the current frame based on the first region of interest in focus by: obtaining the third region of interest in focus of a third reference frame, wherein the third reference frame is the frame preceding the first reference frame (step S2031); generating a change reference amount based on the first region of interest in focus of ...

[0095] In some embodiments, the third reference frame in step S2031 is the frame preceding the first reference frame, i.e., the third reference frame is the (N-3)th image frame. The third reference frame is the image obtained after image reconstruction from the camera input to the terminal when the terminal controls the camera to take a picture; that is, the image acquired by the camera module. The third reference frame refers to the image in the camera's imaging field of view, not the image displayed on the terminal's display interface.

[0096] In some embodiments, the updated region refers to a focusing region of interest of the current frame calculated by using the first focusing region of interest of the first reference frame and the third focusing region of interest of the third reference frame. The updated region is able to completely cover the third focusing region of interest and the first focusing region of interest, and the updated region is less than or equal to the full-frame region. The third focusing region of interest can be denoted as ROI0, and the updated region can be denoted as ROI 更新 .

[0097] In some embodiments, considering that the moving object is in motion, the position change information of the ROI region between two frames is able to reflect the motion speed of the moving object, and therefore the magnification ratio calculated by the change range of the third focusing region of interest and the first focusing region of interest is taken as the change reference in step S2032.

[0098] The change reference includes a first change reference and a second change reference, wherein the first change reference is a change reference in the x-axis (horizontal axis) direction, and the second change reference is a change reference in the y-axis (vertical axis) direction.

[0099] In some embodiments, the first change reference ratioX is:

[0100] ratioX = (endX1 - startX0) / (endX1 - startX1)

[0101] The second change reference ratioY is:

[0102] ratioY = (endY0 - startY1) / (endY1 - startY1)

[0103] In the formula, startX0 is the starting horizontal coordinate of the third focusing region of interest ROI0, startX1 is the starting horizontal coordinate of the first focusing region of interest ROI1, endX1 is the ending horizontal coordinate of the first focusing region of interest ROI1, endY1 is the ending vertical coordinate of the first focusing region of interest ROI1, endY0 is the ending vertical coordinate of the third focusing region of interest ROI0, and startY1 is the starting vertical coordinate of the first focusing region of interest ROI1. At this time, the first focusing region of interest ROI1 is enlarged by using the change reference in step S2033 to obtain the updated region ROI 更新 . As shown in FIG. 2B, the updated region ROI Figure 6 is obtained by using the third focusing region of interest ROI0 and the first focusing region of interest ROI1. The updated region ROI 更新 is able to completely cover the third focusing region of interest ROI0 and the first focusing region of interest ROI1. 更新

[0104] ​In some embodiments, the first change reference amount can be:

[0105] ratioX = (endX1 - startX0) / (endX0 - startX0)

[0106] The second change reference amount can also be:

[0107] ratioY = (endY0 - startY1) / (endY0 - startY0)

[0108] In the formula, endX0 is the end horizontal coordinate of the third focus region of interest ROI0, and startY0 is the start vertical coordinate of the third focus region of interest ROI0. At this time, the third focus region of interest ROI0 is enlarged by using the change reference amount to obtain an updated region ROI 更新 .

[0109] In some embodiments, the region in the second reference frame can also be obtained based on the third focus region of interest ROI0 and the first focus region of interest ROI1 and the change reference amount, and the updated region ROI 更新 .

[0110] S204, determining the PD value of the current frame according to the prediction region and the updated region.

[0111] The specific process is as described above, and will not be repeated here.

[0112] In some embodiments, the AF module can obtain the overlapping region between the prediction region ROI 预测 and the updated region ROI 更新 , calculate the PD value of the overlapping region, and use it as the PD value of the current frame. Since the PD value is calculated using ROI1 or ROI2, it is easy to delay two frames or one frame, and the effect is not good, so in the embodiments of the present disclosure, when calculating the output PD value of the current frame, ROI 更新 is used, and it is considered that when the speed of the object being tracked is very fast, there may be only part of ROI 预测 in ROI 更新 . The PD value in the overlapping region of the updated region ROI 更新 and the prediction region ROI 预测 is selected for calculation, and the PD value in the overlapping region is used as the final result of the PD value of the current frame, so as to facilitate subsequent control of the camera focus. In this way, the problem of frame delay can be solved, and good continuous focusing effect can be achieved when tracking a moving object.

[0113] S205, focusing in the current frame according to the PD value.

[0114] The specific process is as described above, and will not be repeated here.

[0115] In some embodiments, the camera can be a digital camera or an analog camera. The terminal can control the camera to collect the first image in a digital manner or an analog manner. For example, when the camera is an analog camera, the terminal sends a shooting instruction to a driving control unit inside the camera through the display end, at the same time, a detector such as a gyroscope inside the camera sends a detection signal to the driving control unit. The lens sends a negative feedback signal to the driving control unit. The driving control unit generates a driving signal according to the shooting instruction, the detection signal and the negative feedback signal. The driving motor controls the movement of the lens according to the driving signal. The sensor converts the light signal collected by the lens into an electrical signal, and after analog-digital conversion, the digital electrical signal is sent to the terminal. The terminal collects the digital electrical signal through the image acquisition unit and converts the digital electrical signal into an image signal. The display control unit acquires the image signal to control the image signal to be displayed on the display end. At this time, the phase difference between the two is calculated in step S205, the ideal position to be reached by the driving motor is calculated by querying the table, and the driving motor is directly controlled to achieve fast focusing, eliminating the problem of frame delay. When shooting a moving object, good continuous focusing effect can be achieved.

[0116] In the embodiments of the present disclosure, a first focusing region of interest of the first reference frame is obtained, and a second focusing region of interest of the second reference frame is obtained; a prediction region of the current frame is generated according to the first focusing region of interest and the second focusing region of interest; a phase difference value PD value of the current frame is determined according to the prediction region; and focusing is performed in the current frame according to the PD value. Therefore, the prediction region of the current frame is obtained through the corresponding focusing regions of interest of the first reference frame and the second reference frame, the phase difference value PD value of the prediction region is determined, and then focusing is realized. In this way, the AF focusing effect is optimized by reducing the frame delay through AI detection (i.e., a target detection network model) and the prediction of the focusing region of interest of the current frame, especially the focusing effect when shooting a moving object is improved, the out-of-focus problem is reduced, and the problem of motion tracking blur caused by frame delay is avoided, improving the user's shooting experience. The control method of the present disclosure is suitable for the field of mobile phone cameras, especially for the field of mobile phone cameras with motion tracking function, and can also be extended to the field of cameras other than mobile phones, such as micro single, single-lens reflex and digital cameras with motion tracking function.

[0117] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution comply with the relevant legal regulations and do not violate public order and good customs.

[0118] The following is an embodiment of the device of the present disclosure, which can be used to perform the method embodiment of the present disclosure. For details not disclosed in the device embodiment of the present disclosure, please refer to the method embodiment of the present disclosure.

[0119] Please refer to Figure 7 , which shows the structure block diagram of the first focus control device provided by the embodiment of the present disclosure. The focus control device can be realized by software, hardware or a combination of the two to become all or part of the device. The focus control device 10 includes an acquisition module 11, a prediction module 12, a calculation module 13 and a focusing module 14, wherein:

[0120] The acquisition module 11 is configured to acquire a first focus region of interest of a first reference frame and a second focus region of interest of a second reference frame;

[0121] The prediction module 12 is configured to generate a predicted region of a current frame according to the first focus region of interest and the second focus region of interest;

[0122] The calculation module 13 is configured to determine a phase difference value PD value of the current frame according to the predicted region;

[0123] The focusing module 14 is configured to focus in the current frame according to the PD value.

[0124] Optionally, Figure 8 The structure block diagram of the second focus control device provided by the embodiment of the present disclosure is shown. As Figure 8 indicated, the focus control device 10 further includes an update region generation module 15. The update region generation module 15 is configured to generate an update region of the current frame according to the first focus region of interest; wherein the calculation module 13 is specifically configured to determine the PD value of the current frame according to the predicted region and the update region.

[0125] Optionally, the acquisition module 11 can include a target detection network model.

[0126] Optionally, the prediction module 12 can be specifically configured to input the first focus region of interest and the second focus region of interest into a self-optimization algorithm model to generate the predicted region of the current frame.

[0127] Optionally, Figure 9 The structure block diagram of the third focus control device provided by the embodiment of the present disclosure is shown. As Figure 9 indicated, the update region generation module 15 includes a third reference frame acquisition unit 151, a change reference quantity calculation unit 152 and a region generation unit 153.

[0128] The third reference frame acquisition unit 151 is configured to acquire a third focus region of interest of a third reference frame, wherein the third reference frame is a previous frame of the first reference frame;

[0129] The change reference quantity calculation unit 152 is configured to generate a change reference quantity according to the first focus region of interest and the third focus region of interest of the third reference frame;

[0130] The region generation unit 153 is configured to generate an update region according to the first focus region of interest and the change reference quantity.

[0131] Optionally, the change reference quantity comprises a first change reference quantity and a second change reference quantity,

[0132] The first change reference quantity is:

[0133] ratioX=(endX1-startX0) / (endX1-startX1)

[0134] The second change reference quantity is:

[0135] ratioY=(endY0-startY1) / (endY1-startY1)

[0136] In the formula, startX0 is the starting horizontal coordinate of the third focus region of interest, startX1 is the starting horizontal coordinate of the first focus region of interest, endX1 is the ending horizontal coordinate of the first focus region of interest, endY1 is the ending vertical coordinate of the first focus region of interest, endY0 is the ending vertical coordinate of the third focus region of interest, and startY1 is the starting vertical coordinate of the first focus region of interest.

[0137] Optionally, the calculation module 13 can be specifically configured to acquire an overlapping region between the prediction region and the update region, calculate a PD value of the overlapping region, and take the PD value as the PD value of the current frame.

[0138] Optionally, the current frame is an Nth image frame, the first reference frame is an (N-2)th image frame, and the second reference frame is an (N-1)th image frame.

[0139] It is to be noted that the control device for focusing provided in the above embodiment is only used as an example to illustrate the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the electronic device is divided into different functional modules to complete all or part of the above described functions. In addition, the control device for focusing and the control method for focusing provided in the above embodiment belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be described here.

[0140] The above sequence numbers of the embodiments of the present disclosure are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0141] In the embodiments of the present disclosure, the acquisition module 11 acquires a first focus region of interest of the first reference frame and acquires a second focus region of interest of the second reference frame; the prediction module 12 generates a prediction region of the current frame according to the first focus region of interest and the second focus region of interest; the calculation module 13 determines a phase difference value PD value of the current frame according to the prediction region; and the focus module 14 focuses in the current frame according to the PD value. Therefore, the prediction region of the current frame is obtained through the focus regions of interest corresponding to the first reference frame and the second reference frame, the phase difference value PD value of the prediction region is determined, and then the focusing is realized. Thus, the AF focusing effect is optimized by reducing the frame delay through AI detection (i.e., a target detection network model) and the prediction of the focus region of interest of the current frame, the out-of-focus problem can be reduced, and the motion focusing blur problem caused by the frame delay can be avoided, especially the focusing effect when shooting a moving object is improved, and the user's shooting experience is improved. The control method of the present disclosure is suitable for the field of mobile phone cameras, especially for the field of mobile phone cameras with motion focusing function, and can also be extended to the field of non-mobile phone cameras with camera, such as micro single, single-lens reflex, and digital cameras with motion focusing function.

[0142] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0143] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0144] Figure 10 is a block diagram of an electronic device 20 for implementing the control method of focusing according to the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable electronic devices, and other similar computing devices. The components shown in the present disclosure, the connection and relationship of the components, and the functions of the components are only as examples, and are not intended to limit the implementation of the present disclosure described and / or required in the present disclosure.

[0145] As Figure 10As shown, the electronic device 20 includes a computing unit 21 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 22 or a computer program loaded into a random access memory (RAM) 23 from a storage unit 28. Various programs and data required for the operation of the electronic device 20 can also be stored in the RAM 23. The computing unit 21, the ROM 22, and the RAM 23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0146] Various components in the electronic device 20 are connected to the I / O interface 25, including an input unit 26, such as a keyboard, a mouse, and the like; an output unit 27, such as various types of displays, a speaker, and the like; a storage unit 28, such as a magnetic disk, an optical disk, and the like, which is communicatively connected to the computing unit 21; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 29 allows the electronic device 20 to exchange information / data with other electronic devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0147] The computing unit 21 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 21 performs various methods and processes described above, such as the control method of focusing. For example, in some embodiments, the control method of focusing can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 28. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the computing unit 21, one or more steps of the control method of focusing described above can be performed. Alternatively, in other embodiments, the computing unit 21 can be configured to perform the control method of focusing by any other appropriate means, such as by means of firmware.

[0148] Various implementations of the systems and techniques described above in the disclosure can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic electronic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0149] Program code to implement methods of the disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0150] In the disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or electronic device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or electronic device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage electronic device, a magnetic storage electronic device, or any suitable combination of the foregoing.

[0151] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0152] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0153] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server, or VPS for short) services. The server can also be a server of a distributed system, or a server combined with a blockchain.

[0154] It should be understood that various forms of flow shown above can be used with reordering, additions, or removals of steps. For example, steps recited in the present disclosure can be executed in parallel, in series, or in different orders, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited in this regard.

[0155] The above detailed description does not limit the scope of the disclosure. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed description. Any modification, equivalent replacement and improvement etc. made within the spirit and principle of the disclosure shall be included in the scope of the disclosure.

Claims

1. A focusing control method, characterized in that, include: Obtain the first region of interest in focus of the first reference frame, and obtain the second region of interest in focus of focus of the second reference frame; The predicted region of the current frame is generated based on the first and second regions of interest in focus; wherein the current frame is the Nth image frame, the first reference frame is the (N-2)th image frame, and the second reference frame is the (N-1)th image frame; Obtain the third region of interest in the third reference frame, generate a change reference value based on the first region of interest and the third region of interest, and generate the update region of the current frame based on the first region of interest and the change reference value. The phase difference (PD) value of the current frame is determined based on the predicted region and the updated region. Focus is performed in the current frame based on the PD value.

2. The method as described in claim 1, characterized in that, The step of generating the prediction region for the current frame based on the first and second regions of interest includes: The first and second regions of interest are input into an autoregressive optimization algorithm model to generate the predicted region for the current frame.

3. The method as described in claim 1, characterized in that, The change reference values ​​include: a first change reference value and a second change reference value. The first reference value for change is: ratioX=(endX1-startX0) / (endX1-startX1) The second reference value for change is: ratioY=(endY0-startY1) / (endY1-startY1) In the formula, startX0 is the starting x-coordinate of the third region of interest, startX1 is the starting x-coordinate of the first region of interest, endX1 is the ending x-coordinate of the first region of interest, endY1 is the ending y-coordinate of the first region of interest, endY0 is the ending y-coordinate of the third region of interest, and startY1 is the starting y-coordinate of the first region of interest.

4. The method as described in claim 1, characterized in that, Determining the phase difference (PD) value of the current frame based on the predicted region and the updated region includes: Obtain the overlapping region between the predicted region and the updated region, calculate the PD value of the overlapping region, and use it as the phase difference value (PD value) of the current frame.

5. A focusing control device, characterized in that, include: The acquisition module is used to acquire the first region of interest in focus of the first reference frame and the second region of interest in focus of the second reference frame; The prediction module is used to generate a prediction region for the current frame based on the first and second focus regions of interest; wherein the current frame is the Nth image frame, the first reference frame is the (N-2)th image frame, and the second reference frame is the (N-1)th image frame. An update region generation module is used to obtain the third focus region of interest of the third reference frame, generate a change reference value based on the first focus region of interest and the third focus region of interest, and generate the update region of the current frame based on the first focus region of interest and the change reference value. The calculation module is used to determine the phase difference (PD) value of the current frame based on the predicted region and the updated region. A focusing module is used to focus in the current frame based on the PD value.

6. The apparatus according to claim 5, characterized in that, The prediction module is specifically used for: The first and second regions of interest are input into an autoregressive optimization algorithm model to generate the predicted region for the current frame.

7. The apparatus according to claim 5, characterized in that, The change reference values ​​include: a first change reference value and a second change reference value. The first reference value for change is: ratioX=(endX1-startX0) / (endX1-startX1) The second reference value for change is: ratioY=(endY0-startY1) / (endY1-startY1) In the formula, startX0 is the starting x-coordinate of the third region of interest, startX1 is the starting x-coordinate of the first region of interest, endX1 is the ending x-coordinate of the first region of interest, endY1 is the ending y-coordinate of the first region of interest, endY0 is the ending y-coordinate of the third region of interest, and startY1 is the starting y-coordinate of the first region of interest.

8. The apparatus according to claim 5, characterized in that, The computing module is specifically used for: Obtain the overlapping region between the predicted region and the updated region, calculate the PD value of the overlapping region, and use it as the phase difference value (PD value) of the current frame.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the focusing control method according to any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the focusing control method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the focusing control method according to any one of claims 1-4.

Images