Focusing method, device and equipment and readable storage medium

Abstract

The invention relates to a focusing method, device and equipment and a readable storage medium, and the method comprises the steps: obtaining corresponding phase difference images through a plurality of groups of phase difference detection pixel units of an image sensor, recognizing the phase difference between the phase difference images through a neural network, determining a focusing offset based on the phase difference, and carrying out the focusing adjustment. The influence of external interference on phase difference estimation is reduced, so that the accuracy of image phase difference recognition is improved, better focusing precision is provided, quick response of focusing adjustment is realized, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a focusing method, device, equipment and readable storage medium. Background Art

[0002] Focusing is the process of adjusting the lens or settings of the image acquisition device so that the photographed subject or area of ​​interest can obtain a clear and sharp focus on the image. The accuracy of focusing affects the quality and clarity of the image.

[0003] At present, related technologies usually use PDAF (Phase Detection Auto Focus) technology to perform focus adjustment, obtain the phase difference estimation value by calculating the focus difference between two images in a small aperture, and use SAD (Sum of Absolute Differences) to evaluate the calculation accuracy of the phase difference estimation value, so as to adjust and optimize the focus position according to the SAD result.

[0004] However, PDAF technology has poor anti-interference ability. In environments with poor lighting conditions, such as dark light or overexposure, the accuracy and stability of the phase difference estimation value obtained and the process of using SAD to evaluate the calculation accuracy are poor, affecting the focusing effect. Summary of the invention

[0005] In view of this, in order to solve the above technical problems, the present disclosure provides a focusing method, device, equipment and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a focusing method is provided, the method comprising:

[0007] Acquire a first phase difference image and a second phase difference image according to phase difference information collected by each phase difference detection pixel unit in an image sensor of an image acquisition device;

[0008] Determine, according to the focus area position information, a first area in the first phase difference image and a second area in the second phase difference image, wherein positions of the first area and the second area in the images to which they belong match;

[0009] Inputting images corresponding to the first region and the second region into a pre-trained classification network, and determining the image phase difference between the first region and the second region according to a predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to a set phase difference value;

[0010] A focus offset is determined according to the image phase difference, and focus adjustment is performed on the image acquisition device according to the focus offset.

[0011] Optionally, determining the first area in the first phase difference image and the second area in the second phase difference image according to the focus area position information includes:

[0012] Determining a first focus area of ​​the first phase difference image and a second focus area of ​​the second phase difference image according to the focus area position information;

[0013] An area having a set ratio to the first focus area is determined from the first focus area as a first area, and an area matching the position of the first area is determined from the second focus area as a second area.

[0014] Optionally, inputting the images corresponding to the first area and the second area into a pre-trained classification network includes:

[0015] Segmenting the images corresponding to the first region and the second region to obtain N groups of matching image blocks, each group of matching image blocks including two image blocks, which respectively correspond to image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region; N is greater than or equal to 2;

[0016] The N groups of matching image blocks are input into the classification network in groups.

[0017] Optionally, in response to the first phase difference image and the second phase difference image being single-channel images, inputting the images corresponding to the first region and the second region into a pre-trained classification network includes:

[0018] The image corresponding to the first region and the image corresponding to the second region are stacked into a dual-channel image, and the dual-channel image is sent to the classification network.

[0019] Optionally, determining the image phase difference between the first area and the second area according to the predicted phase difference classification result output by the classification network includes:

[0020] After obtaining all predicted phase difference classification results for the images corresponding to the first area and the second area, determining a first target phase difference classification corresponding to a maximum probability value from the classification results; the maximum probability value is greater than or equal to a first confidence threshold;

[0021] If a first target phase difference classification is determined, the phase difference value corresponding to the first target phase difference classification is determined as the image phase difference;

[0022] If at least two of the first target phase difference categories are determined, the image phase difference is determined according to the phase difference values ​​corresponding to the first target phase difference categories.

[0023] Optionally, for dividing the images corresponding to the first region and the second region into N groups of matching image blocks, the predicted phase difference classification result output by the classification network includes a predicted phase difference classification result for each group of matching image blocks;

[0024] The step of determining the image phase difference between the first region and the second region according to the predicted phase difference classification result output by the classification network comprises:

[0025] For the predicted phase difference classification results of each group of matching image blocks, determine a second target phase difference classification corresponding to a probability value greater than a second confidence threshold from the classification results;

[0026] The image phase difference is determined according to the phase difference value corresponding to each second target phase difference category.

[0027] Optionally, determining a focus offset according to the image phase difference includes:

[0028] According to a preset mapping relationship between an image phase difference and a focus offset in the image sensor, the image phase difference between the first area and the second area is converted into a focus offset.

[0029] According to a second aspect of an embodiment of the present disclosure, a focusing device is provided, the device comprising:

[0030] A phase difference image acquisition module, used to acquire a first phase difference image and a second phase difference image according to phase difference information acquired by each phase difference detection pixel unit in an image sensor of an image acquisition device;

[0031] an area determination module, configured to determine, according to the focus area position information, a first area in the first phase difference image and a second area in the second phase difference image, wherein the positions of the first area and the second area in the images to which they belong match;

[0032] A phase difference determination module, used to input the images corresponding to the first area and the second area into a pre-trained classification network, and determine the image phase difference between the first area and the second area according to a predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to a set phase difference value;

[0033] An adjustment module is used to determine a focus offset according to the image phase difference, and to adjust the focus of the image acquisition device according to the focus offset.

[0034] Optionally, the area determination module is specifically used to:

[0035] Determining a first focus area of ​​the first phase difference image and a second focus area of ​​the second phase difference image according to the focus area position information;

[0036] An area having a set ratio to the first focus area is determined from the first focus area as a first area, and an area matching the position of the first area is determined from the second focus area as a second area.

[0037] Optionally, when the phase difference determination module is used to input the images corresponding to the first area and the second area into a pre-trained classification network, it includes:

[0038] Segmenting the images corresponding to the first region and the second region to obtain N groups of matching image blocks, each group of matching image blocks including two image blocks, which respectively correspond to image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region; N is greater than or equal to 2;

[0039] The N groups of matching image blocks are input into the classification network in groups.

[0040] Optionally, in response to the first phase difference image and the second phase difference image being single-channel images, the phase difference determination module, when used to input the images corresponding to the first region and the second region into a pre-trained classification network, includes:

[0041] The image corresponding to the first region and the image corresponding to the second region are stacked into a dual-channel image, and the dual-channel image is sent to the classification network.

[0042] Optionally, the phase difference determination module is specifically used to:

[0043] After obtaining all predicted phase difference classification results for the images corresponding to the first area and the second area, determining a first target phase difference classification corresponding to a maximum probability value from the classification results; the maximum probability value is greater than or equal to a first confidence threshold;

[0044] If a first target phase difference classification is determined, the phase difference value corresponding to the first target phase difference classification is determined as the image phase difference;

[0045] If at least two of the first target phase difference categories are determined, the image phase difference is determined according to the phase difference values ​​corresponding to the first target phase difference categories.

[0046] Optionally, for dividing the images corresponding to the first region and the second region into N groups of matching image blocks, the predicted phase difference classification result output by the classification network includes the predicted phase difference classification result for each group of matching image blocks; and the phase difference determination module is specifically used to:

[0047] For the predicted phase difference classification results of each group of matching image blocks, determine a second target phase difference classification corresponding to a probability value greater than a second confidence threshold from the classification results;

[0048] The image phase difference is determined according to the phase difference value corresponding to each second target phase difference category.

[0049] Optionally, when the adjustment module is used to determine the focus offset according to the image phase difference, it includes:

[0050] According to a preset mapping relationship between an image phase difference and a focus offset in the image sensor, the image phase difference between the first area and the second area is converted into a focus offset.

[0051] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: a processor and a memory; the memory is used to store a computer program; and the processor is used to call the computer program to implement the above-mentioned focusing method.

[0052] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned focusing method is implemented.

[0053] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0054] The focusing method provided by the embodiment of the present disclosure obtains corresponding phase difference images through multiple groups of phase difference detection pixel units of the image sensor, and uses a neural network to extract the features of each phase difference image to determine the phase difference value between the phase difference images, so as to determine the focus offset based on the phase difference to perform focus adjustment, thereby providing better focus accuracy, achieving rapid response of focus adjustment, and improving user experience.

[0055] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present disclosure. In addition, any embodiment of the present disclosure does not need to achieve all the above effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0057] Figure 1 is a flow chart of a focusing method according to an exemplary embodiment;

[0058] Figure 2 is a schematic diagram of a process of determining an image input to a classification network according to an exemplary embodiment;

[0059] Figure 3 is a flow chart showing a method of inputting images corresponding to two regions into a classification network according to an exemplary embodiment;

[0060] Figure 4 is a schematic diagram of a process of determining an image phase difference according to a classification result of a classification network according to an exemplary embodiment;

[0061] Figure 5 is another flowchart of determining image phase difference after dividing an input image into N groups of matching image blocks according to an exemplary embodiment;

[0062] Figure 6 is a flow chart of a focusing method applied to a camera focusing of a mobile phone according to an exemplary embodiment;

[0063] Figure 7 is a schematic structural diagram of a focusing device according to an exemplary embodiment;

[0064] Figure 8 The present invention is a block diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0066] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0067] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first classification threshold may also be referred to as the second classification threshold, and similarly, the second classification threshold may also be referred to as the first classification threshold. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0068] At present, PDAF technology has become the mainstream focusing method in the field of image acquisition due to its fast focusing speed. The PDAF focusing principle is mainly to place two different types of pixels on the pixel array of the photosensitive element sensor, which are called left pixels (or L pixels) and right pixels (or R pixels). These pixels correspond to the microlenses that enter different positions of light. During the focusing process, the camera captures the image after being focused by the microlens, and then calculates the phase difference between the left pixel and the right pixel, calculates the phase difference value, and converts it into the corresponding focus offset through the distance mapping model, thereby moving the lens to adjust the focus.

[0069] In the process of phase difference calculation, the SAD values ​​of the left and right pixels are usually calculated, and the best focus position is determined in combination with the confidence of the phase difference value. The confidence of the phase difference value is usually calculated by the image sensor based on the collected phase information. For harsh lighting environments such as dark light or overexposure, the SAD value and the confidence of the phase difference value calculated by the above method will be greatly affected, resulting in inaccurate or even abnormal calculated phase difference values; at the same time, the phase difference calculation in the PDAF focus solution is easily affected by noise, which can easily lead to inaccurate phase difference calculation.

[0070] Therefore, in order to solve the above technical problems, the present disclosure proposes a focusing method, which obtains the corresponding phase difference images after the image acquisition is completed, and uses a neural network to extract the features of each phase difference image to determine the phase difference value between the phase difference images, so that the focal length difference can be determined according to the phase difference value to achieve focus adjustment. This method can be applied to image acquisition devices such as cameras, camcorders, mobile phones, and tablets that require lens focus adjustment.

[0071] Figure 1 is a flow chart of a focusing method according to an exemplary embodiment. Figure 1 As shown, the method disclosed herein may include the following steps:

[0072] S101, acquiring a first phase difference image and a second phase difference image according to phase difference information collected by each phase difference detection pixel unit in an image sensor of an image acquisition device;

[0073] The phase difference detection pixel unit pair can be composed of two independent pixels at a specific position in the pixel array of the image sensor or two groups of sub-pixels within the same pixel, and is used to measure the phase difference information generated when light passes through different optical paths to reach each phase difference detection pixel unit. The phase difference detection pixel unit pair depends on the pixel array layout and design of the image sensor, and each phase difference detection pixel unit corresponds to at least one photosensor.

[0074] For example, on the horizontal center line of the pixel array of the image sensor, pixels are symmetrically distributed on the left and right sides of the center line, and two pixels opposite to each other on the left and right sides can constitute the phase difference detection pixel unit pair, which is usually used for phase difference detection in the horizontal direction; similarly, pixels can be symmetrically distributed on both sides in the vertical center line or diagonal direction of the pixel array, and two pixels opposite to each other on the two sides can constitute the phase difference detection pixel unit pair, which is used for phase difference detection in the vertical and oblique directions.

[0075] For another example, for each pixel on the pixel array of the image sensor, two adjacent pixels may constitute the phase difference detection pixel unit pair, such as a super PD type pixel arrangement; or, the same pixel may include multiple sub-pixels, and two or two groups of sub-pixels symmetrically arranged about the center line of the same pixel may constitute the phase difference detection pixel unit pair, such as a full-pixel dual-core focusing dual PD type pixel arrangement, and the two or two groups of sub-pixels may be divided into two directions, left and right, and top and bottom, of the center line of the same pixel.

[0076] The first phase difference image and the second phase difference image are images captured simultaneously from the same scene, have the same exposure time, and are usually in the form of original RAW images, i.e., original data that has not been color analyzed and processed. The size, dimensions, format, etc. of the two phase difference images are consistent, and each corresponds to a different pixel unit in the phase difference detection pixel unit pair. For example, the pixel array of the camera's image sensor adopts a full-pixel dual-core focus type pixel arrangement, and each pixel includes two symmetrically positioned left sub-pixel units and right sub-pixel units. The first phase difference image reflects the phase difference information collected by the left sub-pixel unit, and the second phase difference image reflects the phase difference information collected by the right sub-pixel unit.

[0077] The first phase difference image and the second phase difference image can be presented as a grayscale image, and the grayscale value of each pixel corresponds to the phase difference at that position; wherein the brightness of the grayscale value depends on the size of the phase difference, and the larger the phase difference, the brighter the grayscale value is generally, and the smaller the phase difference or the closer it is to zero, the darker the grayscale value is. Generating a phase difference image based on phase difference information is generally to map the phase difference value to a specific grayscale value or other pixel data to obtain it, and the specific implementation method adopts the processing means in the relevant technology, which is not elaborated in this disclosure.

[0078] S102, determining a first area in the first phase difference image and a second area in the second phase difference image according to the focus area position information, wherein positions of the first area and the second area in the images to which they belong match;

[0079] The focus area position information refers to the position information of the image area that is expected to be in sharp focus during the image acquisition process. After determining the focus area of ​​the entire image, the position information of the focus area in the entire image can be obtained. The focus area can be determined in the following ways:

[0080] According to a preset focus area selection ratio, an area indicated by the ratio is determined from the collected complete image as the focus area. For example, the focus area selection ratio is to select an area that occupies 1 / 3 of the complete image and is symmetrical along the center point of the complete image; or, through a set focus detection algorithm, a detected area containing a portrait or an object is determined as the focus area, for example, a rectangular area where a portrait detected by a portrait detection algorithm is located is determined as the focus area; or, when receiving focus indication information from a user, the focus area is determined according to the focus indication information, such as when a user manually touches the screen to select a focus area.

[0081] Because the phase difference is the difference formed by the light passing through the lens to the different phase difference detection pixel units on the image sensor, the pixels in the focus area will produce obvious phase difference, while the pixel phase difference in the non-focus area is relatively small or close to zero. Therefore, the focus adjustment process needs to rely on the phase difference in the focus area to determine the optimal focus position; and, since the first phase difference image and the second phase difference image have the same scene, resolution and size, the matching areas in the two images have the same image features and details, and their pixels can correspond one to one, and the two have different phase information, the focus position can be accurately determined by comparing the matching areas in the two images.

[0082] Based on the above principle, in this embodiment, the first area and the second area are located in the focus area, and the position of the first area in the first phase difference image and the position of the second area in the second phase difference image are matched. The selection of the first area and the second area can be determined from the focus areas of the two phase difference images according to the set area selection ratio.

[0083] S103, inputting the images corresponding to the first area and the second area into a pre-trained classification network, and determining the image phase difference between the first area and the second area according to the predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to the set phase difference value;

[0084] The classification network is used to extract features from the two input images, and classify them according to the extracted features, and predict the probability that the phase difference between the two images belongs to different preset phase difference classifications.

[0085] Among them, multiple phase difference classifications are pre-set in the output layer of the classification network, and each classification corresponds to a phase difference value. For example, the classification network pre-sets 21 categories, which correspond to the phase difference value range [-10, 10], respectively, wherein the positive or negative phase difference value indicates the focus adjustment direction, and the value indicates the pixel distance of moving the focus from the current position along the adjustment direction. Usually, if the phase difference is a positive number, the focus adjustment is made in the direction away from the sensor, and vice versa, the focus adjustment is made in the direction close to the sensor. Based on the fact that the unit of the phase difference is a complete pixel, category 1 can be set to correspond to -10, category 2 to -9, ..., and category 21 to 10.

[0086] After setting the phase difference classification, the above classification network also needs to perform network training in advance, which mainly includes the following processes: collecting a training data set containing phase difference classification annotations, the data set contains multiple groups of pictures and corresponding label classifications, and preprocessing the training data set; using the training data set to iteratively train the classification, passing batches of training samples into the network for forward propagation, calculating the loss function, and calculating the gradient through the back propagation algorithm, and updating the weights and parameters of the network; when the set iterative training stop conditions are reached, such as reaching the maximum number of iterations or the loss function converges, stop training and save the network weights and parameters obtained through training for subsequent prediction and application.

[0087] In the embodiments of the present disclosure, the classification network may adopt various existing network models that support the extraction of image features, such as MoblieNet series networks, VGG series networks, Unet series networks, etc., and the present disclosure does not limit this.

[0088] After determining the first region and the second region, the images corresponding to the two regions can be directly input into the classification network for classification, and a set of probability values ​​output by the classification network can be obtained, which correspond to the set phase difference classifications respectively. The number of the probability values ​​in the set is the same as the number of the preset phase difference classifications, and the sum of the probability values ​​is 1. According to the set of probability values ​​and the preset confidence threshold, the image phase difference corresponding to the two regions can be determined. Alternatively, the images corresponding to the two regions can be divided into multiple batches of image block pairs and then input to obtain the classification results for each image block pair. The image phase difference between the first region and the second region can be obtained by fusing the predicted phase difference values ​​of each image block pair, such as taking the average value, weighted average value, or adaptive fusion according to the size and importance of the image block pair.

[0089] S104: Determine a focus offset according to the image phase difference, and adjust the focus of the image acquisition device according to the focus offset.

[0090] Focus offset refers to the distance difference between the actual focus point and the desired focus point in a camera or optical device, and is used for focus adjustment to improve the focus effect and quality of the image. Since the phase difference can reflect the change in the focus position of the light, the focus offset can be determined based on the image phase difference between the first area and the second area, so as to adjust the focus of the motor or lens in the image acquisition device according to the focus offset to achieve clear focus of the image.

[0091] After determining the image phase difference between the first area and the second area, the image phase difference between the first area and the second area can be converted into a corresponding focus offset according to a pre-established mapping relationship between the image phase difference and the focus offset for the device to be focused. Based on differences in configuration information of image sensors in different devices, optical system design, and other aspects, the mapping relationship between the image phase difference and the focus offset in different devices is different. For example, for a camera that uses phase difference focusing, the image phase difference can be directly used as the focus offset.

[0092] The mapping relationship between the image phase difference and the focus offset can be determined in the following manner:

[0093] Use an interferometer or phase measurement device to obtain the interference pattern of the focus area. These devices usually include components such as laser light source, beam splitter, lens and camera; adjust the focus position of the lens to make the interference pattern achieve the best clarity, and record the corresponding lens position as a reference point; change the phase difference of the interference pattern by fine-tuning the lens position or changing the focal length, and use an interferometer or phase measurement device to capture interference patterns with different phase differences; analyze the interference pattern and calculate the phase difference value at different positions; compare the phase difference value with the phase difference of the reference point to obtain the phase difference change amount of the focus area; and construct a mapping relationship between the phase difference change amount and the focus offset amount based on the phase difference change amount and the focus offset amount.

[0094] In the disclosed embodiments, based on the fact that the neural network has a certain degree of robustness against interferences such as noise, illumination changes and image distortion, two phase difference images corresponding to different phase difference pixel units are obtained, and the phase difference between the two phase difference images is identified using the neural network. The focus offset is determined based on the phase difference to perform focus adjustment, thereby reducing the impact of external interference on phase difference estimation, thereby improving the accuracy of image phase difference recognition, providing better focus accuracy, achieving rapid response to focus adjustment, and improving user experience.

[0095] In addition, since the first phase difference image and the second phase difference image obtained by the present invention are obtained based on information collected by the phase difference detection pixel unit, the focusing method provided by the present invention is applicable to the focusing of various image sensors, and the area used to determine the image phase difference is a local area of ​​the focusing area of ​​the two phase difference images, so that the focusing scheme is applicable to focus adjustment under various focus types.

[0096] In some embodiments, Figure 2 As shown, the step S102 of determining the first area in the first phase difference image and the second area in the second phase difference image according to the focus area position information can be implemented in the following manner:

[0097] S201, determining a first focus area of ​​the first phase difference image and a second focus area of ​​the second phase difference image according to the focus area position information;

[0098] The focus area position information is information predetermined by the image sensor according to the current image to be collected and stored locally, and can be directly obtained locally from the image sensor.

[0099] Based on the focus area position information, the area indicated by the position information can be directly determined in the first phase difference image and the second phase difference image, and the area is determined as the focus area; or, further, after determining the area indicated by the position information, the area range is adjusted according to the pixel or phase difference transformation of the area, and the adjusted area is determined as the focus area. It can be understood that the focus area positions determined by the two phase difference images match. When the two phase difference images are of the same size and dimension, the positions of the determined focus areas in their respective images can be the same.

[0100] S202: Determine, from the first focus area, an area that is in a set ratio to the focus area as a first area, and determine, from the second focus area, an area that matches the position of the first area as a second area.

[0101] That is, after determining the focus areas of the two phase difference images, for the first phase difference image, a local area is selected from the first focus area of ​​the image as the first area, and the first area and the focus area meet the set ratio. For example, if the set ratio is 1:1, the first focus area of ​​the first phase difference image is determined as the first area; for another example, if the set ratio is 1:3, a local area that occupies 1 / 3 of the entire focus area is selected from the first focus area and determined as the first area.

[0102] For the second phase difference image, the position of the local area selected from the second focus area of ​​the image in the second phase difference image matches the position of the first area in the first phase difference image. In the case where the focus areas of the two phase difference images are the same, for the second phase difference image, a local area that is in the set ratio with the second focus area can also be selected from the second focus area of ​​the image and determined as the second area.

[0103] In the embodiment of the present disclosure, based on the fact that the first phase difference image and the second phase difference image are usually large, the amount of computation required to directly input the images into the classification network is large, and the phase difference data quality of each phase difference image in the non-focus area is poor. Therefore, a local area of ​​the focus area of ​​the entire phase difference image or the entire focus area is selected as the input of the classification network, so that the predicted phase difference result output by the classification network is more accurate, thereby improving the computational efficiency and prediction quality of the classification network.

[0104] In some embodiments, based on the fact that the first phase difference image and the second phase difference image are single-channel images, the step S103 of inputting the images corresponding to the first region and the second region into the pre-trained classification network can also be implemented in the following manner:

[0105] The image corresponding to the first region and the image corresponding to the second region are stacked into a dual-channel image, and the dual-channel image is sent to the classification network.

[0106] Generally, the first phase difference image and the second phase difference image determined in the above steps are single-channel images, that is, each pixel includes grayscale or brightness information, and the grayscale value of the pixel represents the intensity, brightness or degree of change of light. Based on this, the images corresponding to the first area and the second area are also single-channel images.

[0107] In the case where the classification network is a twin network, the first single-channel image and the second single-channel image can be respectively sent to the same convolutional network. Since the same convolutional network shares weight parameters, the output prediction result is more accurate.

[0108] When the first phase difference image and the second phase difference image are not single-channel images, the two phase difference images can be converted into grayscale images, and then the two converted grayscale images are superimposed into a dual-channel image and input into the classification network, which can reduce interference information in the image.

[0109] In the disclosed embodiment, by superimposing the single-channel images corresponding to the two regions into a dual-channel image and inputting it into the classification network, the network parameters can be shared and the similarity between the two images can be better utilized, which helps the classification network obtain more texture or structural information and improves the classification accuracy.

[0110] In some embodiments, Figure 3 As shown, the inputting of the images corresponding to the first region and the second region into the pre-trained classification network in the aforementioned step S103 can be achieved by the following steps:

[0111] S301, segmenting the images corresponding to the first region and the second region to obtain N groups of matching image blocks, each group of matching image blocks including two image blocks, which respectively correspond to image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region; N is greater than or equal to 2;

[0112] That is, the image corresponding to the first region is divided into N first image blocks, and according to the rule of image block position matching, the image corresponding to the second region is also divided into N second image blocks, and the image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region constitute a group of matching image blocks. For example, if N is set to 4, the image corresponding to the first region is divided into 4 image blocks such as A1, B1, C1, and D1, and the image corresponding to the second region is divided into 4 image blocks such as A2, B2, C2, and D2, respectively, to obtain 4 groups of matching image blocks such as A1 and A2, B1 and B2, C1 and C2, and D1 and D2, and the two image blocks in each group of matching image blocks come from images corresponding to different regions, and the positions in their respective images match.

[0113] S302, inputting the N groups of matching image blocks into the classification network in groups.

[0114] Based on the phase difference relationship between the images corresponding to the first region and the second region, when cut into multiple image blocks, each image block contains local image information, and the phase difference relationship also exists between the image blocks with matching positions. Therefore, the predicted classification results of each group of matching image blocks can be used to estimate the phase difference between the images corresponding to the first region and the second region.

[0115] After N groups of matching image blocks are input into the classification network in groups, the classification network will perform feature extraction on each group of input matching image blocks to output a predicted probability distribution of a preset phase difference classification, thereby determining the image phase difference between the images corresponding to the first area and the second area based on the classification results of the N groups of matching image blocks.

[0116] In the embodiment of the present disclosure, by dividing the images corresponding to the first region and the second region into N groups of matching image blocks and inputting them into the classification network respectively, based on the fact that each image block includes the part of the image corresponding to the region, the memory usage is reduced, the processing efficiency of the classification network is improved, and when the classification results are subsequently obtained, the probability distribution results of each group of matching image blocks can be comprehensively considered, thereby improving the overall prediction accuracy, thereby enhancing the accuracy of the phase difference between the determined images corresponding to the first region and the second region.

[0117] In some embodiments, Figure 4 As shown, the image phase difference between the first region and the second region is determined according to the predicted phase difference classification result output by the classification network as described in the aforementioned step S103, which can be achieved by the following steps:

[0118] S401, after obtaining all predicted phase difference classification results for images corresponding to the first area and the second area, determining a first target phase difference classification corresponding to a maximum probability value from the classification results; the maximum probability value is greater than or equal to a first confidence threshold;

[0119] The first confidence threshold is used to determine whether the predicted phase difference classification result is reliable. A confidence threshold can be set for each preset phase difference classification, or it can be set so that all preset phase difference classifications share the same confidence threshold. The confidence threshold can be set based on the experience of those skilled in the art, or can be determined based on the classification network training results; and the confidence threshold can be self-learned and updated based on each predicted phase difference classification result.

[0120] For the input method of inputting the images corresponding to the first area and the second area into the classification network for classification without segmentation processing, the classification network outputs a set of predicted phase difference classification results, which means that all predicted phase difference classification results have been obtained, wherein the set of predicted phase difference classification results includes probability values ​​of a preset number of phase difference classifications, and each probability value corresponds to a preset phase difference classification; and for the method of segmenting the images corresponding to the first area and the second area into N groups of matching image blocks and then inputting them into the classification network, after the classification network outputs N groups of predicted phase difference classification results, it means that all predicted phase difference classification results have been obtained.

[0121] For the one or N groups of predicted phase difference classification results obtained above, determine the highest probability value among all groups of probability values, and when the highest probability value is greater than or equal to the first confidence threshold, determine the classification corresponding to the highest probability value as the first target phase difference classification.

[0122] If there is no highest probability value greater than or equal to the first confidence threshold, the aforementioned steps S102 and S103 may be re-executed, and the first area and the second area may be re-selected for calculation, or the traditional PDAF technology may be used for focus adjustment.

[0123] S402, if a first target phase difference category is determined, determining the phase difference value corresponding to the first target phase difference category as the image phase difference;

[0124] S403: If at least two first target phase difference categories are determined, then if at least two first target phase difference categories are determined, the image phase difference is determined according to the phase difference values ​​corresponding to the respective first target phase difference categories.

[0125] That is, when the first target phase difference classification is determined in the manner of step S401, a first target phase difference classification may be determined. For example, if only one set of predicted phase difference classification results is obtained, or there is only one highest probability value in the N sets of predicted phase difference classification results, then the phase difference value corresponding to the first target phase difference classification may be directly determined as the image phase difference.

[0126] When multiple highest probability values ​​are determined from N groups of predicted phase difference classification results, that is, when multiple first target phase difference classifications are determined, the mean of the phase difference values ​​corresponding to the multiple first target phase difference classifications can be calculated. Based on the image phase difference taking a complete pixel as a unit, the rounded result of the mean can be determined as the image phase difference by rounding up or rounding down.

[0127] In the disclosed embodiment, by determining the highest probability value in the results output by the classification network and using a confidence threshold to filter out inaccurate estimates caused by noise or other uncertain factors, the estimation of the phase difference is optimized, the amount of data processed and the computational cost are reduced, and the overall accuracy of the phase difference estimation is improved.

[0128] In some embodiments, for Figure 3 The method shown in FIG. 1 is to divide the image corresponding to the first region and the second region into N groups of matching image blocks and input them into a classification network, wherein the predicted phase difference classification results output by the classification network include N groups of predicted phase difference classification results, and each group of predicted phase difference classification results corresponds to a group of matching image blocks; Figure 5 As shown, the above step S103, according to the predicted phase difference classification result output by the classification network, determines the image phase difference between the first region and the second region, which can also be achieved in the following way:

[0129] S501, for N groups of predicted phase difference classification results, determine from the classification results a second target phase difference classification corresponding to a probability value greater than a second confidence threshold;

[0130] The second confidence threshold has the same function as the first confidence threshold, and is used to determine whether the predicted phase difference classification result is reliable. The setting of the second confidence threshold can rely on the experience of those skilled in the art, or determine the appropriate classification threshold based on the accuracy-recall curve during the classification network training process. Based on the focus method disclosed in the present invention, which pays more attention to accuracy, a higher classification threshold can be selected.

[0131] Traverse the N groups of predicted phase difference classification results, obtain the classification corresponding to each probability value greater than the second confidence threshold, and determine the classification as the second target phase difference classification.

[0132] S502: Determine the image phase difference according to the phase difference value corresponding to each second target phase difference category.

[0133] If a second target phase difference classification is obtained, the phase difference value corresponding to the target phase difference classification may be determined as the image phase difference between the first region and the second region.

[0134] If at least two second target phase difference classifications are obtained, the phase difference values ​​corresponding to the second target phase difference classifications can be fused to obtain the final output image phase difference. For example, the integer result of the average value of the corresponding phase difference values ​​can be calculated, and the rounded result is determined as the final output image phase difference. Alternatively, when the image is divided into N matching image blocks, the weights of each group of matching image blocks can be set, so that according to the weights of the matching image blocks corresponding to each second target phase difference classification, the corresponding phase difference values ​​are weighted averaged, and the rounded result of the weighted average is determined as the image phase difference.

[0135] In the disclosed embodiment, the predicted phase difference classification results of N groups of matching image blocks are screened by using a confidence threshold, the phase difference value corresponding to the phase difference classification with high confidence is determined, and the image phase difference corresponding to the first area and the second area is determined by fusion processing based on the phase difference value, thereby reducing the matching error rate in the matching image blocks, reducing data interference such as noise and abnormal points in multiple groups of classification results, and obtaining a stable image phase difference.

[0136] In order to enable those skilled in the art to more clearly understand the focusing method provided by the present disclosure, the method of the present disclosure will be further explained by taking the camera focusing of a mobile phone as an example.

[0137] The focusing method provided by the embodiment of the present disclosure involves a classification network, which needs to be pre-trained. After selecting the classification network architecture, a suitable loss function is selected to measure the difference between the network output and the true label classification. Common loss functions such as cross entropy loss and mean square error can be selected; Next, the classification network is trained using the collected training data set, which contains multiple groups of input images and phase difference label classifications between each group of input images. The weights and parameters of the network are updated according to the iterative process, and the training is stopped when the specified iteration stop condition is reached. Furthermore, the trained classification network can be evaluated using a validation set or a test set to calculate indicators such as classification accuracy, precision, and recall rate. The confidence threshold can also be determined, and the hyperparameters can be adjusted according to the evaluation results to train the classification network again to obtain better classification performance. After continuous training and optimization, a trained classification network is obtained, which is applied to the focusing method of the present disclosure.

[0138] In this embodiment, the camera focus of the mobile phone adopts the phase difference focus method, and the determined phase difference can be directly used as the focus offset to adjust the focus. Figure 6 As shown, the focusing method provided in this embodiment may include the following steps:

[0139] S601, after the camera exposure is completed, obtaining a left phase difference image and a right phase difference image;

[0140] The phase detection pixel array of the image sensor configured for the camera of the mobile phone is provided with a plurality of groups of left-side phase difference pixel units and right-side phase difference pixel units symmetrically distributed in the horizontal direction, which are respectively used to use the phase difference information caused by light entering the sensor from different optical paths.

[0141] When the camera exposure is completed, the image sensor receives enough light, and the light is exposed on the sensor for an appropriate time. The captured image has been fully exposed and can be used for subsequent processing or storage. Therefore, after the camera exposure is complete, the left phase difference image can be constructed based on the phase difference data collected by each left phase difference pixel unit; the right phase difference image can be constructed based on the phase difference data collected by each right phase difference pixel unit.

[0142] S602, determining a first region image from the left phase difference image and a second region image from the right phase difference image according to the position information of the focus region; the positions of the first region image and the second region image in their respective images are matched;

[0143] The first area image refers to a sub-area in the focus area of ​​the left phase difference image, and the second area image refers to a sub-area in the focus area of ​​the right phase difference image. The sub-area is usually a square or rectangular image and contains local information of the image.

[0144] like Figure 6 As shown, based on the fact that the left phase difference image and the right phase difference image come from the same shooting scene and have the same exposure time, the local information included in the exemplary first area image and the second area image and the pixels within the area match, and there is a difference in phase difference information.

[0145] When no indication information is received that the user has touched the photo-taking interface, the position information of the focus area can be determined based on the configured focus detection algorithm and the current shooting scene; if indication information is received that the user has touched the photo-taking interface, the position information of the area touched by the user is determined as the position information of the focus area.

[0146] S603, dividing the first region image and the second region image into N groups of matching image blocks, and inputting them into a classification network in groups, wherein two image blocks in each group of matching image blocks are from different region images and their positions match;

[0147] S604, obtaining N groups of predicted phase difference classification results corresponding to the N groups of matching image blocks, each group of predicted phase difference classification results including a predicted probability distribution that the phase difference between two image blocks in the group belongs to different phase difference classifications;

[0148] S605, determining a phase difference classification corresponding to a highest probability from the N groups of predicted phase difference classification results, determining it as a target classification, and determining an image phase difference between the first region and the second region according to the target classification;

[0149] If one target category is determined, the phase difference value corresponding to the target category is determined as the image phase difference; if two or more target categories are determined, the average of the phase difference values ​​corresponding to the multiple target categories is calculated, and the average value is rounded off and determined as the image phase difference.

[0150] S606: Determine the value of the image phase difference as a focus offset, and move the camera motor according to the focus offset to perform focus adjustment.

[0151] It is understandable that the steps described in the above embodiments do not necessarily require a specific execution order or execution time. In some embodiments, they can be executed in different orders or times and still achieve the desired effect. For example, in the step of dividing the two regional images into N groups of image blocks described in step S603 above, the two regional image blocks can be first input into the classification network, and the image processing layer in the classification network divides them into N groups of image blocks. For another example, the process for determining the focus offset in the above S604 to S606 can be executed by the classification network, and a new output layer can be added after the output layer of the classification network outputs the predicted phase difference classification result, which is used to process the classification result and finally output the focus offset.

[0152] In the disclosed embodiments, based on the advantages of neural networks such as high prediction accuracy, timely response, and strong adaptability, multiple phase difference classifications are set in the classification network and the phase difference classifications correspond to the phase difference values, and the trained classification network is used to predict the phase difference between the left phase difference image and the right phase difference image. The phase difference estimation of the focus adjustment is converted into a multi-classification prediction problem, which is suitable for phase difference prediction in different environments, lighting conditions and object scenes, meets the application requirements of actual camera focus, and thus improves the focus accuracy.

[0153] Corresponding to the embodiment of the aforementioned focusing method, Figure 7 is a schematic structural diagram of a focusing device according to an exemplary embodiment. Figure 7 As shown, the device includes a phase difference image acquisition module 701, a region determination module 702, a phase difference determination module 703, and an adjustment module 704.

[0154] The phase difference image acquisition module 701 is used to acquire a first phase difference image and a second phase difference image according to the phase difference information collected by the phase difference detection pixel unit in the image sensor;

[0155] An area determination module 702 is used to determine a first area in the first phase difference image and a second area in the second phase difference image according to the focus area position information, wherein the positions of the first area and the second area in the images to which they belong match;

[0156] The phase difference determination module 703 is used to input the images corresponding to the first area and the second area into a pre-trained classification network, and determine the image phase difference between the first area and the second area according to the predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to the set phase difference value;

[0157] The adjustment module 704 is used to determine a focus offset according to the image phase difference, and perform focus adjustment on the image acquisition device according to the focus offset.

[0158] In some embodiments, the region determination module is specifically used to:

[0159] Determining a first focus area of ​​the first phase difference image and a second focus area of ​​the second phase difference image according to the focus area position information;

[0160] An area having a set ratio to the first focus area is determined from the first focus area as a first area, and an area matching the position of the first area is determined from the second focus area as a second area.

[0161] In some embodiments, when the phase difference determination module is used to input the images corresponding to the first area and the second area into a pre-trained classification network, it includes:

[0162] Segmenting the images corresponding to the first region and the second region to obtain N groups of matching image blocks, each group of matching image blocks including two image blocks, which respectively correspond to image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region; N is greater than or equal to 2;

[0163] The N groups of matching image blocks are input into the classification network in groups.

[0164] In some embodiments, in response to the first phase difference image and the second phase difference image being single-channel images, the phase difference determination module, when used to input the images corresponding to the first region and the second region into a pre-trained classification network, includes:

[0165] The image corresponding to the first region and the image corresponding to the second region are stacked into a dual-channel image, and the dual-channel image is sent to the classification network.

[0166] Optionally, the phase difference determination module is specifically used to:

[0167] After obtaining all predicted phase difference classification results for the images corresponding to the first area and the second area, determining a first target phase difference classification corresponding to a maximum probability value from the classification results; the maximum probability value is greater than or equal to a first confidence threshold;

[0168] If a first target phase difference classification is determined, the phase difference value corresponding to the first target phase difference classification is determined as the image phase difference;

[0169] If at least two of the first target phase difference categories are determined, the image phase difference is determined according to the phase difference values ​​corresponding to the first target phase difference categories.

[0170] Optionally, for dividing the images corresponding to the first region and the second region into N groups of matching image blocks, the predicted phase difference classification result output by the classification network includes the predicted phase difference classification result for each group of matching image blocks; and the phase difference determination module is specifically used to:

[0171] For the predicted phase difference classification results of each group of matching image blocks, determine a second target phase difference classification corresponding to a probability value greater than a second confidence threshold from the classification results;

[0172] The image phase difference is determined according to the phase difference value corresponding to each second target phase difference category.

[0173] Optionally, when the adjustment module is used to determine the focus offset according to the image phase difference, it includes:

[0174] According to a preset mapping relationship between an image phase difference and a focus offset in the image sensor, the image phase difference between the first area and the second area is converted into a focus offset.

[0175] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0176] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art may understand and implement it without creative work.

[0177] The embodiment of the present disclosure also provides a terminal device. The terminal device includes: a memory and a processor. The memory stores processor executable instructions, and the processor is configured to execute the executable instructions in the memory to implement the steps of the above-mentioned focusing method. In the embodiment of the present disclosure, the terminal device can be selected as an image acquisition device that needs to adjust the focus, such as a mobile phone camera, a SLR camera, a video camera, etc.

[0178] Figure 8 is a block diagram of a terminal device provided according to an exemplary embodiment. Figure 8 As shown, the terminal device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, a communication component 816, and an image acquisition component.

[0179] The processing component 802 generally controls the overall operation of the terminal device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions. In addition, the processing component 802 may include one or more units to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia unit to facilitate interaction between the multimedia component 808 and the processing component 802.

[0180] The memory 804 is configured to store various types of data to support operations on the terminal device 800. Examples of such data include instructions for any application or method operating on the terminal device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0181] The power supply component 806 provides power to various components of the terminal device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 800.

[0182] The multimedia component 808 includes a screen that provides an output interface between the terminal device 800 and the target object. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the target object. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0183] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0184] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface unit, such as a keyboard, a click wheel, a button, etc.

[0185] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device 800. For example, the sensor assembly 814 can detect the open / closed state of the terminal device 800, the relative positioning of components, such as the display screen and keypad of the terminal device 800, and the sensor assembly 814 can also detect the position change of the terminal device 800 or a component, the presence or absence of contact between the target object and the terminal device 800, the orientation or acceleration / deceleration of the terminal device 800, and the temperature change of the terminal device 800. For another example, the sensor assembly 814 also includes a light sensor, which is arranged below the OLED display screen.

[0186] The communication component 816 is configured to facilitate the communication between the terminal device 800 and other devices in a wired or wireless manner. The terminal device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) unit to facilitate short-range communication. For example, the NFC unit can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0187] In an exemplary embodiment, the terminal device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0188] In an exemplary embodiment, the present disclosure also provides a readable storage medium, which stores executable instructions. The above executable instructions can be executed by a processor of a terminal device to implement the steps of the focusing method provided above. Among them, the readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0189] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the above claims.

[0190] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A focusing method, characterized in that: The method comprises: Acquire a first phase difference image and a second phase difference image according to phase difference information collected by each phase difference detection pixel unit in an image sensor of an image acquisition device; Determine, according to the focus area position information, a first area in the first phase difference image and a second area in the second phase difference image, wherein positions of the first area and the second area in the images to which they belong match; Inputting images corresponding to the first region and the second region into a pre-trained classification network, and determining the image phase difference between the first region and the second region according to a predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to a set phase difference value; A focus offset is determined according to the image phase difference, and focus adjustment is performed on the image acquisition device according to the focus offset.

2. The method according to claim 1, characterized in that: The determining, according to the focus area position information, a first area in the first phase difference image and a second area in the second phase difference image comprises: Determining a first focus area of ​​the first phase difference image and a second focus area of ​​the second phase difference image according to the focus area position information; An area having a set ratio to the first focus area is determined from the first focus area as a first area, and an area matching the position of the first area is determined from the second focus area as a second area.

3. The method according to claim 1, characterized in that: In response to the first phase difference image and the second phase difference image being single-channel images, inputting the images corresponding to the first region and the second region into a pre-trained classification network includes: The image corresponding to the first region and the image corresponding to the second region are stacked into a dual-channel image, and the dual-channel image is sent to the classification network.

4. The method according to claim 1, characterized in that The step of inputting the images corresponding to the first region and the second region into a pre-trained classification network comprises: Segmenting the images corresponding to the first region and the second region to obtain N groups of matching image blocks, each group of matching image blocks including two image blocks, which respectively correspond to image blocks with matching positions in the image corresponding to the first region and the image corresponding to the second region; N is greater than or equal to 2; The N groups of matching image blocks are input into the classification network in groups.

5. The method according to claim 1 or 4, characterized in that: The step of determining the image phase difference between the first region and the second region according to the predicted phase difference classification result output by the classification network comprises: After obtaining all predicted phase difference classification results for the images corresponding to the first area and the second area, determining a first target phase difference classification corresponding to a maximum probability value from the classification results; the maximum probability value is greater than or equal to a first confidence threshold; If a first target phase difference classification is determined, the phase difference value corresponding to the first target phase difference classification is determined as the image phase difference; If at least two of the first target phase difference categories are determined, the image phase difference is determined according to the phase difference values ​​corresponding to the first target phase difference categories.

6. The method according to claim 4, characterized in that The predicted phase difference classification result output by the classification network includes the predicted phase difference classification result for each group of matching image blocks; The step of determining the image phase difference between the first region and the second region according to the predicted phase difference classification result output by the classification network comprises: For the predicted phase difference classification results of each group of matching image blocks, determine a second target phase difference classification corresponding to a probability value greater than a second confidence threshold from the classification results; The image phase difference is determined according to the phase difference value corresponding to each second target phase difference category.

7. The method according to claim 1, characterized in that The determining of the focus offset according to the image phase difference comprises: According to a preset mapping relationship between an image phase difference and a focus offset in the image sensor, the image phase difference between the first area and the second area is converted into a focus offset.

8. A focusing device, characterized in that: The device comprises: A phase difference image acquisition module, used to acquire a first phase difference image and a second phase difference image according to phase difference information acquired by each phase difference detection pixel unit in an image sensor of an image acquisition device; an area determination module, configured to determine, according to the focus area position information, a first area in the first phase difference image and a second area in the second phase difference image, wherein the positions of the first area and the second area in the images to which they belong match; A phase difference determination module, used to input the images corresponding to the first area and the second area into a pre-trained classification network, and determine the image phase difference between the first area and the second area according to a predicted phase difference classification result output by the classification network; the predicted phase difference classification result includes a probability value that the phase difference between the input images belongs to a plurality of preset phase difference classifications; the preset phase difference classification corresponds to a set phase difference value; An adjustment module is used to determine a focus offset according to the image phase difference, and to adjust the focus of the image acquisition device according to the focus offset.

9. An electronic device, characterized in that: include: Processor, memory; The memory is used to store computer programs; The processor is used to call the computer program to implement the focusing method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the focusing method according to any one of claims 1 to 7 is implemented.

Images