Focusing method and device, electronic equipment and storage medium

Abstract

The invention relates to a focusing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first image and a second image; wherein the first image is an image generated after shielding a first area of a photosensitive element; the second image is an image generated after shielding a second area except the first area of the photosensitive element; calculating the similarity between the first image and the second image by using a preset focusing model, and obtaining the phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network; and controlling the focusing distance during image acquisition according to the phase difference. Through the scheme, the definition of imaging based on the focusing distance can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of imaging technology, and in particular to a focusing method and device, an electronic device, and a storage medium. Background Art

[0002] Nowadays, for electronic devices such as mobile phones, cameras, tablets, and wearable devices, functions such as taking photos and recording videos have become the core requirements of users when purchasing electronic devices, among which the clarity of the image has attracted much attention. At present, phase detection auto focus (PDAF) technology has become the mainstream autofocus method due to its fast focusing speed. Phase focus technology is widely used in the front end of image processing modules in electronic devices. The phase difference is determined by the grayscale difference between pixels, and then the focus is guided based on the phase difference to improve the clarity of the image. However, the accuracy of determining the phase difference by the grayscale difference between pixels is low, and it is difficult to meet the requirements of clear imaging. Summary of the invention

[0003] The present invention provides a focusing method and device, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a focusing method, comprising:

[0005] Acquire a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​the photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element;

[0006] Calculating the similarity between the first image and the second image using a preset focusing model, and obtaining a phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network;

[0007] The focusing distance during image acquisition is controlled according to the phase difference.

[0008] In some embodiments, the method further comprises:

[0009] determining a first block in the first image;

[0010] Determine a second block including the coordinate position of the first block in the second image; wherein the size of the second block is larger than the size of the first block;

[0011] The calculating the similarity between the first image and the second image by using a preset focus model, and obtaining the phase difference between the first image and the second image according to the similarity, includes:

[0012] Processing the first block and the second block using the preset focus model to determine the similarity between each sub-block in the second block and the first block; wherein the size of the sub-block in the second block is consistent with the size of the first block;

[0013] The phase difference between the first image and the second image is determined according to the similarity between the first block and each sub-block in the second block.

[0014] In some embodiments, the preset focus model includes a feature extraction layer, a feature processing layer, and a prediction layer; the using the preset focus model to process the first block and the second block to determine the similarity between each sub-block in the second block and the first block includes:

[0015] Using the feature extraction layer to extract features from the first block and the second block respectively, to obtain a first feature corresponding to the first block and a second feature corresponding to the second block;

[0016] Using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculate the similarity between the first feature and each sub-feature;

[0017] The determining the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes:

[0018] The prediction layer is used to determine a phase difference between the first image and the second image according to a similarity between the first feature and each sub-feature.

[0019] In some embodiments, determining the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature by using the prediction layer includes:

[0020] Through the prediction layer, the phase difference between the first image and the second image is determined according to the position difference between the position of the sub-block corresponding to the sub-feature whose similarity meets the preset similarity threshold in the second image and the position of the first block in the first image.

[0021] In some embodiments, the feature processing layer includes a feature selection layer, an upsampling layer, and a feature calculation layer;

[0022] The using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculating the similarity between the first feature and each sub-feature, includes:

[0023] using the feature selection layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature;

[0024] Using the upsampling layer, upsampling the first feature and each sub-feature to obtain the upsampled first feature and each upsampled sub-feature;

[0025] The feature calculation layer is used to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

[0026] In some embodiments, the method further comprises:

[0027] Determining a confidence level of a phase difference between the first image and the second image according to a similarity between each sub-block in the first block and the second block;

[0028] The controlling the focus distance during image acquisition according to the phase difference comprises:

[0029] When the confidence level is greater than a preset confidence threshold, the focusing distance during image acquisition is controlled according to the phase difference.

[0030] In some embodiments, determining the confidence of the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes:

[0031] Obtaining a similarity curve according to the similarities between each of the sub-blocks and the first block;

[0032] Determining, according to the similarity curve, a slope of a curve of a similarity corresponding to a phase difference between the first image and the second image in the similarity curve;

[0033] The confidence level of the phase difference is determined according to the slope of the curve.

[0034] In some embodiments, the method further comprises:

[0035] Acquire a first image sample and a second image sample collected at each lens position during the lens movement process; wherein the first image sample is a plurality of images generated after a first area of ​​the photosensitive element is shielded; and the second image sample is a plurality of images generated after a second area outside the first area of ​​the photosensitive element is shielded;

[0036] Generate training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determine the label values ​​corresponding to the training image samples;

[0037] The neural network is trained using the training image samples and the label values ​​corresponding to the training image samples to obtain the trained preset focusing model.

[0038] In some embodiments, generating training image samples based on first image samples and second image samples corresponding to each lens position during the lens movement process, and determining label values ​​corresponding to the training image samples, includes:

[0039] Based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, synthesize to obtain a synthesized image sample corresponding to each lens position;

[0040] Determining the clarity of the synthetic image samples corresponding to each of the lens positions;

[0041] Determine the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample;

[0042] A training image sample is generated based on the first image sample and the second image sample corresponding to each lens position, and a label value corresponding to the training image sample associated with each lens position is generated based on the phase difference corresponding to the synthetic image sample associated with each lens position.

[0043] In some embodiments, the phase difference corresponding to each synthetic image sample includes a phase difference corresponding to a synthetic block sample in each synthetic image sample; and determining the clarity of the synthetic image samples corresponding to each lens position includes:

[0044] For each synthetic image sample, divide it into a plurality of synthetic block samples;

[0045] Determining the sharpness of each synthesized block sample included in each synthesized image sample;

[0046] The determining of the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample comprises:

[0047] According to the clarity of the synthetic block samples at the same coordinate position in each synthetic image sample, determine the target synthetic block sample whose clarity corresponding to the coordinate position meets the preset clarity threshold;

[0048] For each synthetic block sample in each synthetic image sample, the phase difference between the synthetic block sample and the associated target synthetic block sample is determined according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where a target synthetic block sample associated with the synthetic block sample is located.

[0049] In some embodiments, determining the phase difference between the synthetic block sample and the associated target synthetic block sample according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located comprises:

[0050] determining a position difference between the first lens position and the second lens position;

[0051] According to the position difference and a preset mapping relationship between the position difference and the phase difference, a phase difference between the synthesized block sample and an associated target synthesized block sample is determined.

[0052] In some embodiments, the training image samples include a multi-block sample group; the generating of the training image samples based on the first image samples and the second image samples corresponding to each lens position includes:

[0053] Divide the first image sample corresponding to each lens position into a plurality of first block samples;

[0054] For each first block sample at each lens position, determining a second block sample including a coordinate position of the first block sample in a second image sample corresponding to the lens position; wherein a size of the second block sample is larger than a size of the first block sample;

[0055] For each first block sample, the first block sample and the second block sample corresponding to the first block sample are taken as a block sample group to obtain the training image sample including multiple block sample groups.

[0056] In some embodiments, the phase difference corresponding to the synthetic image sample includes the phase difference corresponding to the synthetic block sample; the training image sample includes a multi-block sample group; and the step of generating a label value corresponding to the training image sample associated with the lens position based on the phase difference corresponding to the synthetic image sample associated with each lens position includes:

[0057] For the synthetic image samples and the training image samples corresponding to the same lens position, the phase difference corresponding to the synthetic block samples in the synthetic image samples is used as the label value of the block sample group in the training image samples that has the same coordinate position as the synthetic block samples.

[0058] According to a second aspect of an embodiment of the present disclosure, there is provided a focusing device, comprising:

[0059] A first acquisition module, used to acquire a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​the photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element;

[0060] An obtaining module, configured to calculate the similarity between the first image and the second image using a preset focusing model, and obtain a phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network;

[0061] A focusing module is used to control the focusing distance during image acquisition according to the phase difference.

[0062] In some embodiments, the apparatus further comprises:

[0063] A first determining module, configured to determine a first block in the first image;

[0064] A second determining module is used to determine a second block including the coordinate position of the first block in the second image; wherein the size of the second block is larger than the size of the first block;

[0065] The obtaining module is used to process the first block and the second block using the preset focusing model to determine the similarity between each sub-block in the second block and the first block; wherein the size of the sub-blocks in the second block is consistent with the size of the first block; and determine the phase difference between the first image and the second image based on the similarity between each sub-block in the first block and the second block.

[0066] In some embodiments, the preset focusing model includes a feature extraction layer, a feature processing layer, and a prediction layer; the obtaining module is used to use the feature extraction layer to extract features from the first block and the second block respectively, to obtain a first feature corresponding to the first block, and a second feature corresponding to the second block; the feature processing layer is used to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature, and calculate the similarity between the first feature and each sub-feature; the prediction layer is used to determine the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature.

[0067] In some embodiments, the obtaining module is used to determine the phase difference between the first image and the second image through the prediction layer based on the position difference between the position of the sub-block corresponding to the sub-feature whose similarity satisfies a preset similarity threshold in the second image and the position of the first block in the first image.

[0068] In some embodiments, the feature processing layer includes a feature selection layer, an upsampling layer, and a feature calculation layer;

[0069] The obtaining module is used to use the feature selection layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature; use the upsampling layer to upsample the first feature and each sub-feature to obtain the upsampled first feature and each upsampled sub-feature; use the feature calculation layer to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

[0070] In some embodiments, the apparatus further comprises:

[0071] a third determination module, configured to determine a confidence level of a phase difference between the first image and the second image according to a similarity between each sub-block in the first block and the second block;

[0072] The focusing module is used to control the focusing distance during image acquisition according to the phase difference when the confidence level is greater than a preset confidence threshold.

[0073] In some embodiments, the third determination module is used to obtain a similarity curve based on the similarity between each sub-block and the first block; determine the slope of a curve of the similarity corresponding to the phase difference between the first image and the second image in the similarity curve based on the similarity curve; and determine the confidence of the phase difference based on the slope of the curve.

[0074] In some embodiments, the apparatus further comprises:

[0075] A second acquisition module is used to acquire a first image sample and a second image sample collected at each lens position during the lens movement process; wherein the first image sample is a plurality of images generated after shielding a first area of ​​the photosensitive element; and the second image sample is a plurality of images generated after shielding a second area outside the first area of ​​the photosensitive element;

[0076] A generation module, used to generate training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determine the label values ​​corresponding to the training image samples;

[0077] A training module is used to train the neural network using the training image samples and the label values ​​corresponding to the training image samples to obtain the trained preset focusing model.

[0078] In some embodiments, the generation module is used to synthesize based on the first image samples and the second image samples corresponding to each lens position during the lens movement process to obtain the synthesized image samples corresponding to each lens position; determine the clarity of the synthesized image samples corresponding to each lens position; determine the phase difference corresponding to each synthesized image sample according to the clarity of each synthesized image sample and the lens position corresponding to each synthesized image sample; generate training image samples based on the first image samples and the second image samples corresponding to each lens position, and generate label values ​​corresponding to the training image samples associated with each lens position based on the phase difference corresponding to the synthesized image samples associated with each lens position.

[0079] In some embodiments, the phase difference corresponding to each synthetic image sample includes the phase difference corresponding to the synthetic block sample in each synthetic image sample; the generation module is used to divide each synthetic image sample into multiple synthetic block samples; determine the clarity of each synthetic block sample included in each synthetic image sample; determine the target synthetic block sample whose clarity corresponding to the coordinate position meets the preset clarity threshold according to the clarity of the synthetic block samples at the same coordinate position in each synthetic image sample; for each synthetic block sample in each synthetic image sample, determine the phase difference between the synthetic block sample and the associated target synthetic block sample according to the first lens position corresponding to the synthetic image sample where the synthetic block sample is located and the second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located.

[0080] In some embodiments, the generating module is used to determine the position difference between the first lens position and the second lens position; and determine the phase difference between the synthesized block sample and the associated target synthesized block sample according to the position difference and a preset mapping relationship between the position difference and the phase difference.

[0081] In some embodiments, the training image samples include a multi-block sample group; the generation module is used to divide the first image samples corresponding to each lens position into multiple first block samples; for each first block sample at each lens position, a second block sample including the coordinate position of the first block sample is determined in the second image sample corresponding to the lens position; wherein the size of the second block sample is larger than the size of the first block sample; for each first block sample, the first block sample and the second block sample corresponding to the first block sample are taken as a block sample group to obtain the training image sample including the multi-block sample group.

[0082] In some embodiments, the phase difference corresponding to the synthetic image sample includes the phase difference corresponding to the synthetic block sample; the training image sample includes a multi-block sample group; the generation module is used to use the phase difference corresponding to the synthetic block sample in the synthetic image sample and the training image sample corresponding to the same lens position as the label value of the block sample group in the training image sample that is consistent with the coordinate position of the synthetic block sample.

[0083] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0084] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method as described in the first aspect above.

[0085] According to a fourth aspect of an embodiment of the present disclosure, there is provided a storage medium, including:

[0086] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in the first aspect above.

[0087] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0088] Since the classification result including scene depth information or lens position information is obtained by using the model in the related art, and the focus distance corresponding to the classification result is obtained by combining the preset mapping relationship, the accuracy of the classification result is low, which is difficult to meet different shooting scenes, and the accuracy of the scene depth calculated by the model is difficult to judge, and the mapping relationship between the lens position and the focus distance is coupled with the influence of the mechanical displacement parameters of the lens motor, resulting in the low accuracy of the focus distance determined in the related art. The solution provided by the embodiment of the present disclosure, on the one hand, uses the preset focus model after training convergence to output the phase difference between the first image and the second image, and then controls the focus distance during image acquisition based on the phase difference, reduces the influence of the mechanical displacement parameters of the lens motor, meets the engineering needs of guiding the focus based on the phase difference, and has a high generalization ability; on the other hand, since the similarity between the left image and the right image is highly correlated with the phase difference between the left image and the right image, the preset focus model of the embodiment of the present disclosure calculates the similarity between the first image and the second image, and obtains the phase difference between the first image and the second image according to the similarity, so as to improve the accuracy of determining the focus distance based on the phase difference, achieve focusing to obtain a clearer image, and improve the user shooting experience.

[0089] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0091] Figure 1 This is an example diagram of calculating the phase difference based on the SAD algorithm in the related art.

[0092] Figure 2 The present invention is a flowchart showing an example of a focusing method according to an embodiment of the present invention.

[0093] Figure 3 This is an example diagram of a focusing model shown in an embodiment of the present disclosure.

[0094] Figure 4 The figure is an example diagram of a similarity curve shown in an embodiment of the present disclosure.

[0095] Figure 5 This is an example diagram of a first block sample according to an embodiment of the present disclosure.

[0096] Figure 6 This is an example diagram of a second block sample shown in an embodiment of the present disclosure.

[0097] Figure 7 A diagram of a focusing device according to an embodiment of the present disclosure is shown.

[0098] Figure 8 The present invention is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0099] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0100] In a related technology, the phase focusing technology generally includes the following steps: selecting the focus area of ​​the phase detection (PD) image, such as taking the central 1 / 3 area as the focus area; wherein the PD image includes a left image and a right image, and the pixels in the PD image correspond to grayscale values. Further dividing the grid based on the focus area, for example, dividing the focus area into multiple 8*8 blocks; using the phase difference calculation method to calculate the phase difference (PD value) and confidence for each block, so that the focus area will have M*N phase differences and confidences, and selecting the phase difference with the highest confidence as the PD result of the focus area; wherein the phase difference calculation method includes the sum of difference (SAD) method, the difference cosine method, the grayscale similarity method, and the like.

[0101] Figure 1 This is an example diagram of phase difference calculation based on SAD algorithm in related technology. Figure 1 The L1 mark is the right picture, and the L2 mark is the left picture. Figure 1 In the related art, the calculation of phase difference based on SAD algorithm includes the following steps: determining the sliding window size, which is usually n*n; obtaining the preset maximum disparity Dmax; generating the left image by blocking the right area of ​​the photosensitive element, and generating the right image by blocking the left area of ​​the photosensitive element; constructing a window with (x, y) as the center on the left image, and constructing a window with (x, y+d) or (x, yd) as the center on the right image; wherein the d value range is 0 to Dmax. Calculate the grayscale difference of the pixels in the left and right image windows within the disparity range of 0 to Dmax, such as calculating Figure 1 The grayscale difference between the p1 pixel in the middle right image and the p2 pixel in the left image, the grayscale difference between the q1 pixel in the right image and the q2 pixel in the left image, etc.; sum the grayscale differences corresponding to each pixel in the window, and select the d with the smallest sum of absolute grayscale differences as the disparity value at (x, y), that is, the phase difference. In addition, based on the sliding process of the sliding window, a SAD curve is generated with the coordinate x of the center pixel of the sliding window as the horizontal coordinate and the grayscale difference as the vertical coordinate. The curvature of the curve is used as the confidence level to assist in judging whether the PD result is credible.

[0102] However, the phase difference calculated by the above scheme is easily affected by image noise, blur, low brightness, etc., resulting in low accuracy of the determined phase difference.

[0103] In a related technology, the principle of binocular vision is used to simultaneously shoot a scene through multiple cameras, and the multiple left and right images captured by the multiple cameras are input into a preset model to calculate the depth of the scene, thereby determining the actual focus distance range corresponding to different depth ranges based on the mapping relationship between the preset depth and focus distance.

[0104] However, the actual accuracy of the depth calculated by this solution is difficult to judge. If the depth calculation error is large, the focus distance range error determined based on the preset mapping relationship between depth and focus distance will also be large.

[0105] In a related technology, a plurality of left and right images generated during the entire lens movement process are collected by moving the lens and input into a preset model, wherein the model divides the lens movement stroke into 49 categories (i.e., 49 positions as classification results) with a movement interval of 10 mm, calculates the image contrast corresponding to each lens position respectively, takes the classification result with the maximum contrast as the output result, and determines the focus distance based on the mapping relationship between the preset classification result and the focus distance. The mapping relationship between the preset classification result and the focus distance is shown in Table 1 below:

[0106] Table 1

[0107]

[0108] It can be seen that the classification result output by the above-mentioned preset model is a lens position range divided with a coarse precision, and the mapping between the lens position and the focusing distance is coupled with the influence of the mechanical displacement parameters of the lens motor. Therefore, the focusing distance determined based on the model output results and the preset mapping relationship in the related art has low accuracy.

[0109] In this regard, an embodiment of the present disclosure provides a focusing method, and the execution subject may be a focusing device with an imaging function, such as a user equipment (UE), a mobile device, a user terminal, a mobile phone, a tablet computer, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the focusing method may be implemented by a processor in the focusing device calling a computer-readable instruction stored in a memory.

[0110] Figure 2 is an example diagram of a focusing method flow chart shown in an embodiment of the present disclosure, comprising Figure 2 It can be seen that the following steps are included:

[0111] S201, acquiring a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​a photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element;

[0112] S202, calculating the similarity between the first image and the second image using a preset focus model, and obtaining a phase difference between the first image and the second image according to the similarity; wherein the preset focus model is trained based on a neural network;

[0113] S203: Control the focus distance during image acquisition according to the phase difference.

[0114] The focusing device of the embodiment of the present disclosure includes a photosensitive element, wherein the photosensitive element includes a plurality of photosensitive pixels.

[0115] In step S201, the focusing device obtains a first image generated by blocking a first area of ​​the photosensitive element, and obtains a second image generated by blocking a second area outside the first area of ​​the photosensitive element; wherein, blocking the first area of ​​the photosensitive element and blocking the second area outside the first area of ​​the photosensitive element can be achieved based on a manufacturing process such as a phase sensor, for example, the first area of ​​some photosensitive pixels in the phase sensor is covered with a metal cover to block the first area of ​​the photosensitive element; the second area of ​​another part of the photosensitive pixels in the phase sensor is covered with a metal cover to block the second area of ​​the photosensitive element; wherein the phase sensor is such as a dualPD sensor, a sparsePD sensor, etc.

[0116] In the embodiment of the present disclosure, if the first area is the left area of ​​part or all of the photosensitive pixels in the photosensitive element, the first image generated after blocking the first area of ​​the photosensitive element can be regarded as the right image, and the second image generated after blocking the second area (right area) outside the first area of ​​the photosensitive element can be regarded as the left image; if the first area is the right area of ​​part or all of the photosensitive pixels, the first image generated after blocking the first area of ​​the photosensitive element can be regarded as the left image, and the second image generated after blocking the second area (left area) outside the first area of ​​the photosensitive element can be regarded as the right image.

[0117] In step S202, the focusing device calculates the similarity between the first image and the second image using a preset focusing model, and obtains the phase difference between the first image and the second image based on the similarity; in some embodiments, the preset focusing model may include a first processing layer, a second processing layer, and a third processing layer; the focusing device extracts features from the first image and the second image through the first processing layer of the preset focusing model, and determines the similarity based on the difference between the extracted features through the second processing layer, and then determines the phase difference between the first image and the second image based on the similarity through the third processing layer; wherein the preset focusing model can be obtained by using left image samples and right image samples as training samples, using the preset phase difference between the left image samples and the right image samples as a label value to train a neural network, and adjusting the parameters of the first processing layer, the second processing layer, and the third processing layer.

[0118] In some embodiments, the preset focusing model may include a cascaded first sub-model and a second sub-model, and the output result of the first sub-model is used as the input of the second sub-model. The focusing device processes the first image and the second image through the first sub-model to obtain the similarity between the first image and the second image; and processes the similarity between the first image and the second image through the second sub-model to obtain the phase difference between the first image and the second image. Among them, the first sub-model is obtained by training the neural network using the left image sample and the right image sample as training samples, and the preset similarity between the left image sample and the right image sample as the label value; the second sub-model is obtained by training another neural network using the similarity between the left image sample and the right image sample as training samples, and the preset phase difference between the left image sample and the right image sample as the label value.

[0119] In step S203, the focusing device controls the focusing distance of the focusing device when performing image acquisition according to the phase difference output by the preset focusing model; in some embodiments, the focusing distance corresponding to the phase difference output by the preset focusing model is determined according to the mapping relationship between the preset phase difference and the focusing distance, and then the lens is controlled to move based on the focusing distance to perform image acquisition and imaging. In some embodiments, the focusing device can also calculate the phase difference using a preset focusing equation to obtain the focusing distance, and then control the lens movement based on the focusing distance to improve the imaging clarity; wherein the preset focusing equation can be obtained by fitting the conversion relationship between the focusing distance and the phase difference after multiple experiments.

[0120] It is understandable that, since the classification result including scene depth information or lens position information is obtained by using the model in the related art, and the focus distance corresponding to the classification result is obtained by combining the preset mapping relationship, the accuracy of the classification result is low, and it is difficult to meet different shooting scenes. In addition, the accuracy of the scene depth calculated by the model is difficult to judge, and the mapping relationship between the lens position and the focus distance is coupled with the influence of the mechanical displacement parameters of the lens motor, resulting in the low accuracy of the focus distance determined in the related art. The scheme provided by the embodiment of the present disclosure, on the one hand, uses the preset focus model after training convergence to output the phase difference between the first image and the second image, and then controls the focus distance during image acquisition based on the phase difference, reduces the influence of the mechanical displacement parameters of the lens motor, and meets the engineering needs of guiding the focus based on the phase difference while having a high generalization ability. On the other hand, since the similarity between the left image and the right image is highly correlated with the phase difference between the left image and the right image, the preset focus model of the embodiment of the present disclosure calculates the similarity between the first image and the second image, and obtains the phase difference between the first image and the second image according to the similarity, so as to improve the accuracy of determining the focus distance based on the phase difference, achieve focusing to obtain a clearer image, and improve the user shooting experience.

[0121] In some embodiments, the method further comprises:

[0122] determining a first block in the first image;

[0123] Determine a second block including the coordinate position of the first block in the second image; wherein the size of the second block is larger than the size of the first block;

[0124] The calculating the similarity between the first image and the second image by using a preset focus model, and obtaining the phase difference between the first image and the second image according to the similarity, includes:

[0125] Processing the first block and the second block using the preset focus model to determine the similarity between each sub-block in the second block and the first block; wherein the size of the sub-block in the second block is consistent with the size of the first block;

[0126] The phase difference between the first image and the second image is determined according to the similarity between the first block and each sub-block in the second block.

[0127] In the embodiment of the present disclosure, the focusing device determines the first block in the first image by taking a central block of a preset proportion of the first image as the first block, for example, taking a 1 / 3 central block of the first image as the first block; or by using a target detection algorithm to take a block such as a face block, a human body block, or a foreground block as the first block, etc., which is not limited to the embodiment of the present disclosure.

[0128] After determining the first block, the focusing device of the embodiment of the present disclosure determines a second block including the coordinate position of the first block in the second image. For example, the portion of the second image including the coordinate position of the first block and the neighborhood portion of the coordinate position of the first block are taken as the second block; wherein the neighborhood portion of the coordinate position of the first block may be a portion adjacent to the coordinate position of the first block in a preset direction (such as the x direction), or a portion adjacent to the coordinate position of the first block in all directions (including the x direction and the y direction), so that the size of the second block in the second image of the embodiment of the present disclosure is larger than the size of the first block in the first image.

[0129] The focusing device of the disclosed embodiment uses a preset focusing model to determine the similarity between each sub-block in the second block and the first block. In some embodiments, the focusing device uses the preset focusing model to divide the second block into multiple sub-blocks according to the size of the first block, and obtains multiple sub-blocks with the same size as the first block; wherein the sub-blocks may overlap or may not overlap. After dividing the second block into multiple sub-blocks, the first block and each sub-block are processed by the preset focusing model to obtain the similarity between each sub-block in the second block and the first block.

[0130] In some embodiments, the focusing device uses a preset focusing model to slide a preset sliding window in the second block, and determines the similarity between the sub-blocks in each sliding window and the first block; wherein the size of the preset sliding window is consistent with the size of the first block, and adjacent preset sliding windows may overlap or may not overlap.

[0131] After determining the similarity between each sub-block in the first block and the second block, the embodiment of the present disclosure uses a preset focus model, such as the aforementioned third processing layer or the second sub-model in the preset focus model, to determine the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block. In some embodiments, the preset focus model determines the maximum similarity among the similarities between each sub-block in the first block and the second block, and based on the aforementioned method, determines the phase difference between the first image and the second image according to the maximum similarity.

[0132] In some embodiments, a preset focusing model determines the similarities between the first block and each sub-block in the second block that are greater than a preset similarity threshold, and determines the mean of the similarities that are greater than the preset similarity threshold, thereby determining the phase difference between the first image and the second image based on the mean of the similarities in the aforementioned manner.

[0133] It can be understood that, on the one hand, the focusing device of the embodiment of the present disclosure determines the first block in the first image and the second block in the second image, and uses the preset focusing model to process each sub-block in the first block and the second block to determine the phase difference between the first image and the second image, which can reduce the amount of calculation of the preset focusing model and improve the immediacy of determining the phase difference between the first image and the second image. On the other hand, the block offset in the image caused by the unclear focus is usually not too large. Therefore, the embodiment of the present disclosure determines the second block in the second image based on the first block in the first image, and determines the similarity between each sub-block in the second block and the first block, which can improve the rationality of finding a sub-block with high similarity in the second block after determining that the first block is offset, thereby improving the accuracy of determining the phase difference between the first image and the second image based on the similarity between each sub-block in the second block and the first block.

[0134] In some embodiments, the preset focus model includes a feature extraction layer, a feature processing layer, and a prediction layer; the using the preset focus model to process the first block and the second block to determine the similarity between each sub-block in the second block and the first block includes:

[0135] Using the feature extraction layer to extract features from the first block and the second block respectively, to obtain a first feature corresponding to the first block and a second feature corresponding to the second block;

[0136] Using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculate the similarity between the first feature and each sub-feature;

[0137] The determining the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes:

[0138] The prediction layer is used to determine a phase difference between the first image and the second image according to a similarity between the first feature and each sub-feature.

[0139] In the disclosed embodiment, the preset focusing model includes a feature extraction layer, a feature processing layer, and a prediction layer. The focusing device uses the feature extraction layer in the preset focusing model, such as the feature extraction layer in the convolutional neural network (CNN) and the deep learning network (DNN), to extract features from the first block and the second block respectively, and obtain a first feature corresponding to the first block and a second feature corresponding to the second block.

[0140] After obtaining the first feature corresponding to the first block and the second feature corresponding to the second block, using the feature processing layer in the preset focusing model and the preset sliding window, according to the first size of the first feature, a sub-feature consistent with the first size is slidably selected in the second feature, and the similarity between the first feature and each sub-feature is calculated, that is, the similarity between the first block and each sub-block in the second block is obtained.

[0141] The disclosed embodiment utilizes the prediction layer in the focusing model to determine the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature; wherein the prediction layer in the focusing model corresponds to the aforementioned third processing layer, or corresponds to the processing layer in the aforementioned second sub-model.

[0142] It can be understood that the preset focusing model in the embodiment of the present disclosure includes a feature extraction layer, a feature processing layer, and a prediction layer. Compared with the preset focusing model obtained after cascading multiple sub-models, the embodiment of the present disclosure can reduce the computing resource consumption and model training time caused by cascading.

[0143] In some embodiments, determining the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature by using the prediction layer includes:

[0144] Through the prediction layer, the phase difference between the first image and the second image is determined according to the position difference between the position of the sub-block corresponding to the sub-feature whose similarity meets the preset similarity threshold in the second image and the position of the first block in the first image.

[0145] In the embodiment of the present disclosure, after determining the similarity between the first feature and each sub-feature, the prediction layer is used to determine the sub-features whose similarity meets the preset similarity threshold, such as determining the sub-features whose similarity is greater than the preset similarity threshold, or the sub-features corresponding to the maximum similarity, etc.

[0146] The disclosed embodiment determines the phase difference between the first image and the second image through a prediction layer according to the position difference between the position of a sub-block corresponding to a sub-feature whose similarity satisfies a preset similarity threshold and the position of the first block in the first image; wherein the position difference between the position of the sub-block in the second image and the position of the first block in the first image may be the distance between the coordinates of the center of the sub-block in the second image and the coordinates of the center of the first block in the first image, or the distance between the coordinates of the center of the sub-block in the second image and the coordinates of the center of the first block in the first image in a preset direction (such as the x direction, the corresponding coordinate is the horizontal coordinate).

[0147] In some embodiments, the position difference between the position of the sub-block corresponding to the sub-feature whose similarity meets the preset similarity threshold in the second image and the position of the first block in the first image is determined as the phase difference between the first image and the second image through the prediction layer. The phase difference is determined by the distance between the horizontal coordinate of the center of the sub-block in the second image and the horizontal coordinate of the center of the first block in the first image, which can be expressed by the following formula (1):

[0148] PD=S1-S0; (1)

[0149] Wherein, PD represents the phase difference, S1 represents the horizontal coordinate of the center of the sub-block most similar to the first block in the second image, and S0 represents the horizontal coordinate of the center of the first block in the first image.

[0150] In some embodiments, through the prediction layer, based on a preset focus distance equation, the position difference between the position of the sub-block corresponding to the sub-feature whose similarity meets the preset similarity threshold in the second image and the position of the first block in the first image is processed to obtain the phase difference between the first image and the second image. In the focus distance equation, the position difference and the phase difference are linearly related, which can be expressed by the following formula (2):

[0151] AF defocus value = Final PD value * DCC * VCM Sensitivity; (2)

[0152] Among them, AF defocus value represents the position difference, Final PD value represents the phase difference, DCC is the preset defocus conversion coefficient, and VCM Sensitivity is the preset lens motor mechanical displacement parameter.

[0153] It is understandable that blocks usually shift when the image is unclear due to defocus, so the block offset size is highly correlated with the focus distance. In this regard, the embodiment of the present disclosure utilizes the prediction layer in the preset focusing model to characterize the offset size of the first block based on the position difference between the position of the sub-block corresponding to the sub-feature that satisfies the preset similarity threshold in the second image and the position of the first block in the first image, thereby improving the accuracy of determining the phase difference between the first image and the second image based on the position difference.

[0154] In some embodiments, the feature processing layer includes a feature selection layer, an upsampling layer, and a feature calculation layer;

[0155] The using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculating the similarity between the first feature and each sub-feature, includes:

[0156] using the feature selection layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature;

[0157] Using the upsampling layer, upsampling the first feature and each sub-feature to obtain the upsampled first feature and each upsampled sub-feature;

[0158] The feature calculation layer is used to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

[0159] In the embodiment of the present disclosure, the feature processing layer includes a feature selection layer, an upsampling layer and a feature calculation layer; wherein the feature selection layer is used to use a preset sliding window to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature; the upsampling layer is used to upsample the first feature and each slidingly selected sub-feature to obtain the upsampled first feature and each upsampled sub-feature; the feature calculation layer is used to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

[0160] It can be understood that since extracting features in the feature extraction layer usually reduces the resolution, after the embodiment of the present disclosure uses the feature extraction layer to obtain the first feature and each sub-feature, it uses the upsampling layer to upsample the first feature and each sub-feature to improve the feature resolution, which helps to improve the accuracy of the similarity between the upsampled first feature and each upsampled sub-feature calculated by the feature calculation layer.

[0161] Figure 3 An example diagram of a focusing model shown in an embodiment of the present disclosure, comprising Figure 3It can be seen that the focusing model includes a first feature extraction layer, a second feature extraction layer, a first upsampling layer, a second upsampling layer, a feature calculation layer and a prediction layer; wherein the first feature extraction layer and the second feature extraction layer correspond to the aforementioned feature extraction layer, and the first upsampling layer, the second upsampling layer and the feature calculation layer correspond to the aforementioned feature processing layer. The first feature extraction layer extracts features from the first block of the first image to obtain the first feature corresponding to the first block; the first upsampling layer upsamples the first feature to obtain the upsampled first feature; the second feature extraction layer extracts features from the second block of the second image to obtain the second feature corresponding to the second block; according to the first size of the first feature, a sub-feature consistent with the first size is selected from the second feature by sliding (corresponding to the aforementioned feature selection layer), and each sub-feature is upsampled through the second upsampling layer to obtain the upsampled second feature; the feature calculation layer calculates the similarity between the first feature and each sub-feature in the second feature; finally, the prediction layer determines the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature.

[0162] It can be understood that the preset focusing model in the embodiment of the present disclosure does not limit the sizes of the first block and the second block of the input, and has better model generalization capability.

[0163] In some embodiments, the method further comprises:

[0164] Determining a confidence level of a phase difference between the first image and the second image according to a similarity between each sub-block in the first block and the second block;

[0165] The controlling the focus distance during image acquisition according to the phase difference comprises:

[0166] When the confidence level is greater than a preset confidence threshold, the focusing distance during image acquisition is controlled according to the phase difference.

[0167] In an embodiment of the present disclosure, the focusing device determines the confidence of the phase difference between the first image and the second image based on the similarity between each sub-block in the first block and the second block. For example, the proportion of the number of sub-blocks corresponding to a preset similarity threshold in the total number of sub-blocks is determined as the confidence of the phase difference between the first image and the second image; or, the confidence of the phase difference between the first image and the second image is determined based on the sum of the differences between each similarity, wherein the confidence of the phase difference between the first image and the second image is positively correlated with the sum of the differences between each similarity.

[0168] The focusing device of the embodiment of the present disclosure controls the focus distance during image acquisition according to the phase difference when the confidence corresponding to the phase difference is greater than a preset confidence threshold. This solution can reduce the situation where the image is not clear due to adjusting the focus distance based on the phase difference with low confidence.

[0169] In some embodiments, determining the confidence of the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes:

[0170] Obtaining a similarity curve according to the similarities between each of the sub-blocks and the first block;

[0171] Determining, according to the similarity curve, a slope of a curve of a similarity corresponding to a phase difference between the first image and the second image in the similarity curve;

[0172] The confidence level of the phase difference is determined according to the slope of the curve.

[0173] In an embodiment of the present disclosure, the focusing device obtains a similarity curve based on the similarity between each sub-block and the first block; wherein the similarity curve includes a curve whose horizontal axis is the center coordinate of each sub-block (such as the x-coordinate) and whose vertical axis is the similarity between each sub-block and the first block.

[0174] The focusing device of the embodiment of the present disclosure determines the slope of the curve of the similarity corresponding to the phase difference between the first image and the second image in the similarity curve based on the similarity curve, such as determining the slope of the curve of the maximum similarity in the similarity curve, or determining the slopes of the curves of multiple similarities greater than a preset similarity threshold in the similarity curve, thereby determining the slope of the curve of the maximum similarity in the similarity curve as the confidence of the phase difference based on the slope of the curve, or determining the weighted value of the slopes of the curves of multiple similarities greater than the preset similarity threshold in the similarity curve as the confidence of the phase difference, etc.

[0175] Figure 4 An example diagram of a similarity curve shown in an embodiment of the present disclosure, from Figure 4It can be seen that the slope of the vertex in the similarity curve marked by L3 is large, indicating that the similarity between the sub-block corresponding to the vertex and the first block is much greater than the similarity between other sub-blocks and the first block. Therefore, the confidence in determining the phase difference between the first image and the second image based on the similarity between the sub-block corresponding to the vertex and the first block is high. The slope of the vertex in the similarity curve marked by L4 is small, indicating that the similarity between the sub-block corresponding to the vertex and the first block is small, and the similarity between other sub-blocks and the first block is small. Therefore, the confidence in determining the phase difference between the first image and the second image based on the similarity between the sub-block corresponding to the vertex and the first block is low. The sharper the similarity curve is, the more the sub-block obtained by sliding matches the first block. Therefore, the confidence can be calculated by measuring the sharpness of the curve (the slope of the curve).

[0176] It is understandable that, combined with Figure 4 It can be seen from the description that the similarity between the sub-block and the first block has a large correlation in the slope of the curve in the similarity curve. Therefore, the embodiment of the present disclosure determines the slope of the curve in the similarity curve corresponding to the similarity of the phase difference between the first image and the second image based on the similarity curve, and then determines the confidence of the phase difference based on the slope of the curve with high accuracy.

[0177] In some embodiments, the method further comprises:

[0178] Acquire a first image sample and a second image sample collected at each lens position during the lens movement process; wherein the first image sample is a plurality of images generated after a first area of ​​the photosensitive element is shielded; and the second image sample is a plurality of images generated after a second area outside the first area of ​​the photosensitive element is shielded;

[0179] Generate training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determine the label values ​​corresponding to the training image samples;

[0180] The neural network is trained using the training image samples and the label values ​​corresponding to the training image samples to obtain the trained preset focusing model.

[0181] The disclosed embodiment describes in detail the process of training a neural network to obtain a preset focus model. First, the first image sample and the second image sample collected at each lens position during the lens movement are obtained; wherein the first image sample is a plurality of images generated after the first area of ​​the photosensitive element is blocked; and the second image sample is a plurality of images generated after the second area outside the first area of ​​the photosensitive element is blocked. If the first area is the left area of ​​part or all of the photosensitive pixels, the plurality of first image samples generated after blocking the first area of ​​the photosensitive element can be regarded as right image samples, and the plurality of second image samples generated after blocking the second area (right area) outside the first area of ​​the photosensitive element can be regarded as left image samples; if the first area is the right area of ​​part or all of the photosensitive pixels, the plurality of first image samples generated after blocking the first area of ​​the photosensitive element can be regarded as left image samples, and the plurality of second image samples generated after blocking the second area (left area) outside the first area of ​​the photosensitive element can be regarded as right image samples.

[0182] After obtaining the first image sample and the second image sample collected at each lens position during the lens movement process, training image samples are generated based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and label values ​​corresponding to the training image samples are determined; wherein the first image sample and the second image sample collected at each lens position correspond to label values, and the label value can be a manually marked phase difference, or a phase difference negatively correlated with the similarity determined after determining the similarity of the first image sample and the second image sample based on a preset algorithm, etc.

[0183] The disclosed embodiment uses training image samples and label values ​​corresponding to the training image samples to train a neural network to obtain a preset focus model after training; wherein the neural network is such as a CNN or DNN model. Exemplarily, after the training image samples are input into the neural network, the loss between the phase difference prediction value output by the neural network and the label value (phase difference) is calculated based on a preset loss function, and then the parameters of the neural network, such as weights, biases, etc., are adjusted based on the loss, and the neural network after the parameters are adjusted and converged is determined as the preset focus model after training; wherein the preset loss function is such as a focal loss function, a Hinge loss function, a cross entropy loss function, etc.

[0184] It can be understood that the embodiment of the present disclosure generates training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and trains the neural network in combination with the label values ​​corresponding to the training image samples to obtain a preset focusing model that converges after training. Compared with detecting phase difference based on an algorithm, the embodiment of the present disclosure has higher accuracy in detecting phase difference based on the preset focusing model and better generalization ability.

[0185] In some embodiments, generating training image samples based on first image samples and second image samples corresponding to each lens position during the lens movement process, and determining label values ​​corresponding to the training image samples, includes:

[0186] Based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, synthesize to obtain a synthesized image sample corresponding to each lens position;

[0187] Determining the clarity of the synthetic image samples corresponding to each of the lens positions;

[0188] Determine the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample;

[0189] A training image sample is generated based on the first image sample and the second image sample corresponding to each lens position, and a label value corresponding to the training image sample associated with each lens position is generated based on the phase difference corresponding to the synthetic image sample associated with each lens position.

[0190] In the embodiment of the present disclosure, synthesis is performed based on the first image samples and the second image samples corresponding to each lens position during the lens movement process. That is, for each lens position, the grayscale values ​​of the pixel points at the same position coordinates in the left image samples and the right image samples collected at the lens position are added to obtain the synthesized image samples corresponding to each lens position, and the pixel points in the synthesized image samples have corresponding grayscale values.

[0191] After obtaining the synthetic image samples corresponding to each lens position, the clarity of each synthetic image sample is determined using algorithms such as grayscale variance algorithm, Brenner function, Roberts gradient sum, Laplace gradient sum, etc.

[0192] In the embodiments of the present disclosure, for each synthetic image sample, based on the clarity of the synthetic image sample and the lens position corresponding to the synthetic image sample, the lens position difference between the synthetic image sample and the synthetic image sample with the highest clarity is used to represent the phase difference between the synthetic image sample and the synthetic image sample with the highest clarity; or a weighted result of the lens position difference between the synthetic image sample and the synthetic image sample with the highest clarity is used as the phase difference between the synthetic image sample and the synthetic image sample with the highest clarity, etc.

[0193] In the disclosed embodiment, for each lens position, the first image sample and the second image sample corresponding to the lens position are used as a group of training image samples, and the phase difference corresponding to the synthetic image sample associated with the lens position is used as the label value corresponding to a group of training image samples associated with the lens position, so as to train the neural network based on each group of training image samples and the label value corresponding to each group of training image samples to obtain a preset focusing model.

[0194] It can be understood that the grayscale values ​​of the pixels in the synthetic image samples of the embodiments of the present disclosure are more comprehensive, so that the accuracy of the clarity determined based on the synthetic image samples is higher, and further, the accuracy of the phase difference corresponding to each synthetic image sample determined based on the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample is higher, which helps to generate training image samples based on the first image samples and the second image samples corresponding to each lens position, and generate label values ​​corresponding to the training image samples associated with the lens position based on the phase difference corresponding to the synthetic image samples associated with each lens position to train the neural network, thereby improving the accuracy of the preset focusing model.

[0195] In some embodiments, the phase difference corresponding to each synthetic image sample includes a phase difference corresponding to a synthetic block sample in each synthetic image sample; and determining the clarity of the synthetic image samples corresponding to each lens position includes:

[0196] For each synthetic image sample, divide it into a plurality of synthetic block samples;

[0197] Determining the sharpness of each synthesized block sample included in each synthesized image sample;

[0198] The determining of the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample comprises:

[0199] According to the clarity of the synthetic block samples at the same coordinate position in each synthetic image sample, determine the target synthetic block sample whose clarity corresponding to the coordinate position meets the preset clarity threshold;

[0200] For each synthetic block sample in each synthetic image sample, the phase difference between the synthetic block sample and the associated target synthetic block sample is determined according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where a target synthetic block sample associated with the synthetic block sample is located.

[0201] In the embodiment of the present disclosure, each first image sample includes a plurality of block samples, each second image sample includes a plurality of block samples, and thus each synthesized image sample includes a plurality of synthesized block samples.

[0202] In the disclosed embodiment, for each synthetic image sample, a sliding window is used to divide the synthetic image sample to obtain a plurality of synthetic sample blocks in each synthetic image sample, and the clarity of each synthetic block sample included in each synthetic image sample is determined based on the aforementioned grayscale variance algorithm, Brenner function, Roberts gradient sum, Laplace gradient sum and other algorithms.

[0203] After determining the sharpness of multiple synthetic block samples included in each synthetic image sample, determine the target synthetic block sample whose sharpness corresponding to the coordinate position meets the preset sharpness threshold according to the sharpness of the synthetic block sample at the same coordinate position in each synthetic image sample, and for each synthetic block sample in each synthetic image sample, determine the position difference between the first lens position and the second lens position as the phase difference between the synthetic block sample and the associated target synthetic block sample according to the first lens position corresponding to the synthetic image sample where the synthetic block sample is located and the second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located, or determine the weighted result of the position difference between the first lens position and the second lens position as the phase difference between the synthetic block sample and the associated target synthetic block sample, etc. Wherein, the target synthetic block sample whose sharpness meets the preset sharpness threshold may be the target synthetic block sample whose sharpness is greater than the preset sharpness threshold; or after generating a sharpness curve based on the coordinate position and the sharpness, determine the synthetic block sample whose coordinate position corresponding to the maximum sharpness falls as the target synthetic block sample. It should be noted that the coordinate position corresponding to the maximum clarity indicates that at this focusing distance, the clearest image can be formed when the lens moves to this lens position.

[0204] It can be understood that the embodiment of the present disclosure divides the synthetic image samples to obtain multiple synthetic block samples, and then determines the phase difference corresponding to the synthetic block samples based on the block dimension, which helps to subsequently expand the number of training samples.

[0205] In some embodiments, determining the phase difference between the synthetic block sample and the associated target synthetic block sample according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located comprises:

[0206] determining a position difference between the first lens position and the second lens position;

[0207] According to the position difference and a preset mapping relationship between the position difference and the phase difference, a phase difference between the synthesized block sample and an associated target synthesized block sample is determined.

[0208] In the embodiment of the present disclosure, for each synthetic block sample in each synthetic image sample, the position difference between the first lens position and the second lens position is determined according to the first lens position corresponding to the synthetic image sample where the synthetic block sample is located and the second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located, and the phase difference between the synthetic block sample and the associated target synthetic block sample is determined according to the position difference and the mapping relationship between the preset position difference and the phase difference. The preset mapping relationship between the position difference and the phase difference can be obtained after multiple test fittings, such as the mapping between the position difference represented by the AF defocus value and the phase difference represented by the Final PD value in the aforementioned formula (2).

[0209] It can be understood that the embodiment of the present disclosure determines the phase difference between the synthetic block sample and the associated target synthetic block sample based on the mapping relationship between the preset position difference and the phase difference. Compared with directly using the position difference to represent the phase difference between the synthetic block sample and the associated target synthetic block sample, the accuracy of the phase difference determination can be adjusted by adjusting the mapping relationship, which has better flexibility.

[0210] In some embodiments, the training image samples include a multi-block sample group; the generating of the training image samples based on the first image samples and the second image samples corresponding to each lens position includes:

[0211] Divide the first image sample corresponding to each lens position into a plurality of first block samples;

[0212] For each first block sample at each lens position, determining a second block sample including a coordinate position of the first block sample in a second image sample corresponding to the lens position; wherein a size of the second block sample is larger than a size of the first block sample;

[0213] For each first block sample, the first block sample and the second block sample corresponding to the first block sample are taken as a block sample group to obtain the training image sample including multiple block sample groups.

[0214] In the embodiment of the present disclosure, for each first image sample corresponding to a lens position, a plurality of first block samples are divided using a sliding window, such as Figure 5 The embodiment of the present disclosure shown in the figure shows an example diagram of a first block sample, wherein L5 identifies the first image sample and L6 identifies the first block sample.

[0215] For each first block sample at each lens position, the embodiment of the present disclosure determines a second block sample including the coordinate position of the first block sample in the second image sample corresponding to the lens position. For example, the portion of the second image sample including the coordinate position of the first block sample and the neighborhood portion of the coordinate position of the first block sample are used as the second block sample; wherein the neighborhood portion of the coordinate position of the first block sample may be a portion adjacent to the coordinate position of the first block sample in a preset direction (such as the x direction), or a portion adjacent to the coordinate position of the first block sample in all directions (including the x direction and the y direction), so that the size of the second block sample in the second image sample of the embodiment of the present disclosure is larger than the size of the first block sample in the first image sample. Figure 6 The embodiment of the present disclosure shown in the figure shows an example diagram of a second block sample, wherein L7 identifies a second image sample and L8 identifies a second block sample.

[0216] In the embodiment of the present disclosure, for each first block sample, the first block sample and the second block sample corresponding to the first block sample are used as a block sample group to obtain a training image sample including multiple block sample groups. Through this solution, the number of training samples can be expanded.

[0217] In some embodiments, the phase difference corresponding to the synthetic image sample includes the phase difference corresponding to the synthetic block sample; the training image sample includes a multi-block sample group; and the step of generating a label value corresponding to the training image sample associated with the lens position based on the phase difference corresponding to the synthetic image sample associated with each lens position includes:

[0218] For the synthetic image samples and the training image samples corresponding to the same lens position, the phase difference corresponding to the synthetic block samples in the synthetic image samples is used as the label value of the block sample group in the training image samples that has the same coordinate position as the synthetic block samples.

[0219] In the disclosed embodiment, for the synthetic image samples and training image samples corresponding to the same lens position, the phase difference corresponding to the synthetic block samples in the synthetic image samples is associated with the coordinate position of the synthetic block samples (such as the horizontal coordinate of the center of the synthetic block), so that the phase difference corresponding to the synthetic block samples in the synthetic image samples is used as the label value of the block sample group in the training image samples that is consistent with the coordinate position of the synthetic block samples. Through this scheme, the accuracy of automatic association between training samples and label values ​​can be improved, which in turn helps to improve the accuracy of obtaining a preset focusing model by training a neural network based on training samples and label values.

[0220] Figure 7 is a diagram of a focusing device shown in an embodiment of the present disclosure, comprising Figure 7 It is known that, including:

[0221] The first acquisition module 701 is used to acquire a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​the photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element;

[0222] An obtaining module 702 is used to calculate the similarity between the first image and the second image using a preset focusing model, and obtain a phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network;

[0223] The focusing module 703 is used to control the focusing distance during image acquisition according to the phase difference.

[0224] In some embodiments, the apparatus further comprises:

[0225] A first determination module 704, configured to determine a first block in the first image;

[0226] A second determination module 705 is configured to determine a second block including the coordinate position of the first block in the second image; wherein the size of the second block is larger than the size of the first block;

[0227] The obtaining module 702 is used to process the first block and the second block using the preset focusing model to determine the similarity between each sub-block in the second block and the first block; wherein the size of the sub-blocks in the second block is consistent with the size of the first block; and determine the phase difference between the first image and the second image based on the similarity between each sub-block in the first block and the second block.

[0228] In some embodiments, the preset focusing model includes a feature extraction layer, a feature processing layer, and a prediction layer; the obtaining module 702 is used to use the feature extraction layer to extract features from the first block and the second block respectively, to obtain a first feature corresponding to the first block, and a second feature corresponding to the second block; use the feature processing layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature, and calculate the similarity between the first feature and each sub-feature; use the prediction layer to determine the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature.

[0229] In some embodiments, the obtaining module 702 is used to determine the phase difference between the first image and the second image through the prediction layer according to the position difference between the position of the sub-block corresponding to the sub-feature whose similarity satisfies a preset similarity threshold in the second image and the position of the first block in the first image.

[0230] In some embodiments, the feature processing layer includes a feature selection layer, an upsampling layer, and a feature calculation layer;

[0231] The obtaining module 702 is used to use the feature selection layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature; use the upsampling layer to upsample the first feature and each sub-feature to obtain the upsampled first feature and each upsampled sub-feature; use the feature calculation layer to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

[0232] In some embodiments, the apparatus further comprises:

[0233] A third determination module 706, configured to determine a confidence level of a phase difference between the first image and the second image according to a similarity between each sub-block in the first block and the second block;

[0234] The focusing module 703 is used to control the focusing distance during image acquisition according to the phase difference when the confidence level is greater than a preset confidence threshold.

[0235] In some embodiments, the third determination module 706 is used to obtain a similarity curve based on the similarity between each sub-block and the first block; determine the slope of a curve of the similarity corresponding to the phase difference between the first image and the second image in the similarity curve based on the similarity curve; and determine the confidence of the phase difference based on the slope of the curve.

[0236] In some embodiments, the apparatus further comprises:

[0237] The second acquisition module 707 is used to acquire the first image sample and the second image sample collected at each lens position during the lens movement process; wherein the first image sample is a plurality of images generated after the first area of ​​the photosensitive element is shielded; and the second image sample is a plurality of images generated after the second area outside the first area of ​​the photosensitive element is shielded;

[0238] A generating module 708, configured to generate training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determine label values ​​corresponding to the training image samples;

[0239] The training module 709 is used to train the neural network using the training image samples and the label values ​​corresponding to the training image samples to obtain the trained preset focusing model.

[0240] In some embodiments, the generation module 708 is used to synthesize based on the first image samples and the second image samples corresponding to each lens position during the lens movement process to obtain a synthesized image sample corresponding to each lens position; determine the clarity of the synthesized image samples corresponding to each lens position; determine the phase difference corresponding to each synthesized image sample according to the clarity of each synthesized image sample and the lens position corresponding to each synthesized image sample; generate training image samples based on the first image samples and the second image samples corresponding to each lens position, and generate a label value corresponding to the training image sample associated with each lens position based on the phase difference corresponding to the synthesized image sample associated with each lens position.

[0241] In some embodiments, the phase difference corresponding to each synthetic image sample includes the phase difference corresponding to the synthetic block sample in each synthetic image sample; the generation module 708 is used to divide each synthetic image sample into multiple synthetic block samples; determine the clarity of each synthetic block sample included in each synthetic image sample; determine the target synthetic block sample whose clarity corresponding to the coordinate position meets the preset clarity threshold according to the clarity of the synthetic block samples at the same coordinate position in each synthetic image sample; for each synthetic block sample in each synthetic image sample, determine the phase difference between the synthetic block sample and the associated target synthetic block sample according to the first lens position corresponding to the synthetic image sample where the synthetic block sample is located and the second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located.

[0242] In some embodiments, the generating module 708 is used to determine the position difference between the first lens position and the second lens position; and determine the phase difference between the synthesized block sample and the associated target synthesized block sample according to the position difference and a preset mapping relationship between the position difference and the phase difference.

[0243] In some embodiments, the training image samples include a multi-block sample group; the generation module 708 is used to divide the first image samples corresponding to each lens position into multiple first block samples; for each first block sample at each lens position, a second block sample including the coordinate position of the first block sample is determined in the second image sample corresponding to the lens position; wherein the size of the second block sample is larger than the size of the first block sample; for each first block sample, the first block sample and the second block sample corresponding to the first block sample are taken as a block sample group to obtain the training image sample including the multi-block sample group.

[0244] In some embodiments, the phase difference corresponding to the synthetic image sample includes the phase difference corresponding to the synthetic block sample; the training image sample includes a multi-block sample group; the generation module 708 is used to use the phase difference corresponding to the synthetic block sample in the synthetic image sample as the label value of the block sample group in the training image sample that has the same coordinate position as the synthetic block sample for the synthetic image sample and the training image sample corresponding to the same lens position.

[0245] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0246] Figure 8 8 is a block diagram of an electronic device 800 shown in an embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, a mobile computer, etc.

[0247] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0248] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0249] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0250] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0251] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0252] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0253] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0254] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0255] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0256] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0257] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0258] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the aforementioned focusing method.

[0259] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims above.

[0260] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A focusing method, characterized in that: The method comprises: Acquire a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​the photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element; Calculating the similarity between the first image and the second image using a preset focusing model, and obtaining a phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network; The focusing distance during image acquisition is controlled according to the phase difference.

2. The method according to claim 1, characterized in that The method further comprises: determining a first block in the first image; Determine a second block including the coordinate position of the first block in the second image; wherein the size of the second block is larger than the size of the first block; The calculating the similarity between the first image and the second image by using a preset focus model, and obtaining the phase difference between the first image and the second image according to the similarity, includes: Processing the first block and the second block using the preset focus model to determine the similarity between each sub-block in the second block and the first block; wherein the size of the sub-block in the second block is consistent with the size of the first block; The phase difference between the first image and the second image is determined according to the similarity between the first block and each sub-block in the second block.

3. The method according to claim 2, characterized in that The preset focus model includes a feature extraction layer, a feature processing layer, and a prediction layer; the using the preset focus model to process the first block and the second block to determine the similarity between each sub-block in the second block and the first block includes: Using the feature extraction layer to extract features from the first block and the second block respectively, to obtain a first feature corresponding to the first block and a second feature corresponding to the second block; Using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculate the similarity between the first feature and each sub-feature; The determining the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes: The prediction layer is used to determine a phase difference between the first image and the second image according to a similarity between the first feature and each sub-feature.

4. The method according to claim 3, characterized in that: The using the prediction layer to determine the phase difference between the first image and the second image according to the similarity between the first feature and each sub-feature includes: Through the prediction layer, the phase difference between the first image and the second image is determined according to the position difference between the position of the sub-block corresponding to the sub-feature whose similarity meets the preset similarity threshold in the second image and the position of the first block in the first image.

5. The method according to claim 3, characterized in that: The feature processing layer includes a feature selection layer, an upsampling layer and a feature calculation layer; The using the feature processing layer to slide and select sub-features consistent with the first size in the second feature according to the first size of the first feature, and calculating the similarity between the first feature and each sub-feature, includes: using the feature selection layer to slide and select a sub-feature consistent with the first size in the second feature according to the first size of the first feature; Using the upsampling layer, upsampling the first feature and each sub-feature to obtain the upsampled first feature and each upsampled sub-feature; The feature calculation layer is used to calculate the similarity between the upsampled first feature and each upsampled sub-feature.

6. The method according to claim 2, characterized in that The method further comprises: Determining a confidence level of a phase difference between the first image and the second image according to a similarity between each sub-block in the first block and the second block; The step of controlling the focus distance during image acquisition according to the phase difference comprises: When the confidence level is greater than a preset confidence threshold, the focus distance during image acquisition is controlled according to the phase difference.

7. The method according to claim 6, characterized in that The determining the confidence of the phase difference between the first image and the second image according to the similarity between each sub-block in the first block and the second block includes: Obtaining a similarity curve according to the similarities between each of the sub-blocks and the first block; Determining, according to the similarity curve, a slope of a curve of a similarity corresponding to a phase difference between the first image and the second image in the similarity curve; The confidence level of the phase difference is determined according to the slope of the curve.

8. The method according to claim 1, characterized in that: The method further comprises: Acquire a first image sample and a second image sample collected at each lens position during the lens movement process; wherein the first image sample is a plurality of images generated after shielding a first area of ​​the photosensitive element; and the second image sample is a plurality of images generated after shielding a second area outside the first area of ​​the photosensitive element; Generate training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determine the label values ​​corresponding to the training image samples; The neural network is trained using the training image samples and the label values ​​corresponding to the training image samples to obtain the trained preset focusing model.

9. The method according to claim 8, characterized in that The generating of training image samples based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, and determining the label values ​​corresponding to the training image samples, comprises: Based on the first image samples and the second image samples corresponding to each lens position during the lens movement process, synthesize to obtain a synthesized image sample corresponding to each lens position; Determining the clarity of the synthetic image samples corresponding to each of the lens positions; Determine the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample; A training image sample is generated based on the first image sample and the second image sample corresponding to each lens position, and a label value corresponding to the training image sample associated with each lens position is generated based on the phase difference corresponding to the synthetic image sample associated with each lens position.

10. The method according to claim 9, characterized in that The phase difference corresponding to each synthetic image sample includes a phase difference corresponding to a synthetic block sample in each synthetic image sample; and determining the clarity of the synthetic image samples corresponding to each lens position includes: For each synthetic image sample, divide it into a plurality of synthetic block samples; Determining the sharpness of each synthesized block sample included in each synthesized image sample; The determining of the phase difference corresponding to each synthetic image sample according to the clarity of each synthetic image sample and the lens position corresponding to each synthetic image sample comprises: According to the clarity of the synthetic block samples at the same coordinate position in each synthetic image sample, determine the target synthetic block sample whose clarity corresponding to the coordinate position meets the preset clarity threshold; For each synthetic block sample in each synthetic image sample, the phase difference between the synthetic block sample and the associated target synthetic block sample is determined according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where a target synthetic block sample associated with the synthetic block sample is located.

11. The method according to claim 10, characterized in that The determining of the phase difference between the synthetic block sample and the associated target synthetic block sample according to a first lens position corresponding to the synthetic image sample where the synthetic block sample is located and a second lens position corresponding to the synthetic image sample where the target synthetic block sample associated with the synthetic block sample is located comprises: determining a position difference between the first lens position and the second lens position; According to the position difference and a preset mapping relationship between the position difference and the phase difference, a phase difference between the synthesized block sample and an associated target synthesized block sample is determined.

12. The method according to claim 9, characterized in that The training image samples include a plurality of block sample groups; the generating of the training image samples based on the first image samples and the second image samples corresponding to each lens position includes: Divide the first image sample corresponding to each lens position into a plurality of first block samples; For each first block sample at each lens position, determining a second block sample including a coordinate position of the first block sample in a second image sample corresponding to the lens position; wherein a size of the second block sample is larger than a size of the first block sample; For each first block sample, the first block sample and the second block sample corresponding to the first block sample are taken as a block sample group to obtain the training image sample including multiple block sample groups.

13. The method according to claim 9, characterized in that The phase difference corresponding to the synthetic image sample includes the phase difference corresponding to the synthetic block sample; the training image sample includes a multi-block sample group; the generating of the label value corresponding to the training image sample associated with the lens position based on the phase difference corresponding to the synthetic image sample associated with each lens position includes: For the synthetic image samples and the training image samples corresponding to the same lens position, the phase difference corresponding to the synthetic block samples in the synthetic image samples is used as the label value of the block sample group in the training image samples that has the same coordinate position as the synthetic block samples.

14. A focusing device, characterized in that: The device comprises: A first acquisition module, used to acquire a first image and a second image; wherein the first image is an image generated after shielding a first area of ​​the photosensitive element; and the second image is an image generated after shielding a second area outside the first area of ​​the photosensitive element; An obtaining module, configured to calculate the similarity between the first image and the second image using a preset focusing model, and obtain a phase difference between the first image and the second image according to the similarity; wherein the preset focusing model is trained based on a neural network; A focusing module is used to control the focusing distance during image acquisition according to the phase difference.

15. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method as claimed in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 13.

Images