Autofocus method, apparatus, terminal, storage medium and program product

Abstract

The present disclosure relates to an auto-focusing method, device, terminal, storage medium and program product, wherein the method comprises: acquiring a to-be-detected image and noise information corresponding to the to-be-detected image; wherein the noise information is used to indicate a noise level of an image sensor in a current shooting environment; performing phase detection on the to-be-detected image based on the noise information to obtain a phase difference; and performing a focusing operation based on the phase difference.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, and in particular to autofocus methods, devices, terminals, storage media, and software products. Background Technology

[0002] Phase Detection Autofocus (PDAF) technology primarily determines image sharpness by performing phase detection on acquired pixel images, and is widely used in the focusing systems of digital devices. In recent years, with the development of deep learning technology, PDAF systems based on Artificial Intelligence (AI) models have gradually gained attention. These systems can be trained on large-scale data and possess certain representation learning and pattern recognition capabilities.

[0003] However, most existing AI-driven PDAF systems model the focusing task as a classification problem, with input only raw pixel images and the detection of the sharpest image from these pixel images. This results in focusing accuracy being affected in low-light and low-texture scenes, and the predicted values ​​are often inaccurate, making it difficult to effectively identify the focus position. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an autofocus method, apparatus, terminal, storage medium, and program product.

[0005] According to a first aspect of the present disclosure, an autofocus method is provided, comprising:

[0006] Acquire an image to be detected, and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor;

[0007] Based on the noise information, phase detection is performed on the image to be detected to obtain the phase difference;

[0008] Focusing is performed based on the phase difference.

[0009] According to a second aspect of the present disclosure, an autofocus device is provided, comprising:

[0010] An image acquisition unit is used to acquire an image to be detected and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor;

[0011] A phase detection unit is used to perform phase detection on the image to be detected based on the noise information to obtain the phase difference;

[0012] A focusing unit is used to perform focusing operations based on the phase difference.

[0013] According to a third aspect of the present disclosure, a terminal is provided, comprising:

[0014] processor;

[0015] Memory used to store computer programs or instructions that can be executed by a processor;

[0016] The processor is configured to execute computer programs or instructions to implement the steps of the autofocus method described in the first aspect.

[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, which, when a computer program or instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the autofocus method as described in the first aspect.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the autofocus method as described in the first aspect.

[0019] In this embodiment of the disclosure, by acquiring noise information, the noise level of the current shooting environment can be effectively calibrated, thereby enhancing the accuracy and robustness of phase detection. This technology enables the system to more accurately identify phase differences in low signal-to-noise ratio environments, improving autofocus performance, especially in extremely low light, backlight, or weak texture scenes, significantly improving focusing reliability and speed. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a flowchart illustrating an autofocus method according to some embodiments of the present disclosure.

[0022] Figure 2 This is a schematic diagram of an autofocus system apparatus according to some embodiments of the present disclosure.

[0023] Figure 3 This is a schematic diagram of the structure of an autofocus device according to some embodiments of the present disclosure.

[0024] Figure 4 This is a block diagram illustrating an apparatus according to some embodiments of the present disclosure.

[0025] Figure 5This is a block diagram of a chip system according to some embodiments of the present disclosure. Detailed Implementation

[0026] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.

[0027] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0028] Phase Detection Autofocus (PDAF) technology primarily determines image sharpness by performing phase detection on the acquired pixel images, and is widely used in the focusing systems of terminals. However, due to the influence of many ambient light, physical, and optical constraints, the predictions of the system algorithm may be prone to errors, such as geometric distortion caused by optical refraction of the camera lens, or additional noise related to photoelectric conversion.

[0029] In recent years, with the development of deep learning technology, PDAF systems based on artificial intelligence (AI) models have gradually attracted attention. These systems can be trained on large-scale data, possess certain representation learning and pattern recognition capabilities, and can still identify phase differences between left and right images even in challenging scenarios.

[0030] However, most existing AI-driven PDAF systems model the focusing task as an N-choose-1 classification problem, with input consisting only of the original pixel image. An AI model determines which image in a stack of input images (containing several pixels) is the sharpest. Because this approach lacks calibration for sensor noise levels, focusing accuracy is affected in low-light and low-texture scenes. In low signal-to-noise ratio scenarios, predictions are often inaccurate, making it difficult to effectively identify the focus point.

[0031] Figure 1 This is a flowchart illustrating an autofocus method according to some embodiments of the present disclosure, such as... Figure 1As shown, the autofocus method can be applied to camera equipment or terminals with camera functions, and includes the following steps.

[0032] In step S11, the image to be detected and the noise information corresponding to the image to be detected are acquired; wherein, the noise information is used to indicate the noise level of the current shooting environment on the image sensor.

[0033] After focusing is activated, at least one image to be detected can be acquired through the image sensor, and noise information corresponding to the image to be detected can be obtained. The noise information is used to calibrate the noise level of the current shooting environment.

[0034] In one implementation, detection pixel units for acquiring an image to be detected can be deployed on an image sensor. After focusing is initiated, the detection pixel units acquire image data to form the image to be detected, which may also be referred to as a pixel image.

[0035] The distribution of detection pixel units on the image sensor can be configured according to actual needs. For example, all pixel units on the image sensor can be designated as detection pixel units; or some pixel units can be designated as detection pixel units and deployed using patterns such as stripe distribution, staggered distribution, or region distribution. Furthermore, the distribution of detection pixel units can be dynamically adjusted according to the characteristics of the scene to optimize focusing performance.

[0036] In one implementation, the noise information corresponding to the image to be detected can be represented in various forms, such as: the analog signal gain of the image sensor under the current shooting environment; exposure time; sensor operating temperature; image texture information, etc.

[0037] In one implementation, a fixed total light intake (Light_quantity) can be used to obtain the exposure time of the image sensor when it receives the total light intake during the exposure operation. Based on the exposure time (exposure_time) of the total light intake, the analog signal gain (gain) under the current shooting environment can be calculated as a calibration value (temp_gain) for the noise level of the current shooting environment. For example, it can be calculated based on the following formula: Light_quantity = gain × exposure_time.

[0038] In step S12, phase detection is performed on the image to be detected based on noise information to obtain the phase difference.

[0039] In one implementation, phase detection can be performed on the image to be detected based on the influence of noise information on the phase difference, thus obtaining the phase difference corresponding to the image to be detected. The current focus accuracy can be determined by the phase difference; the larger the absolute value of the phase difference, the less accurate the current focus.

[0040] For example, the influence factor of noise on the phase difference can be predetermined, and the obtained phase difference can be calibrated based on this influence factor. Alternatively, machine learning algorithms can be used to predict the phase difference under noise conditions.

[0041] In step S13, focusing is performed based on the phase difference.

[0042] The focusing operation can include autofocus mode and manual focus mode. In autofocus mode, the lens position can be adjusted in real time based on the phase difference; in manual focus mode, the user can make fine adjustments based on the feedback information of the phase difference.

[0043] In one implementation, the lens position can be adjusted based on the obtained phase difference to adjust the focal length. Specifically, the lens can be moved back and forth by a motor to achieve precise focusing.

[0044] In one implementation, a phase threshold can be preset, and the phase difference obtained in the current round can be compared with the phase threshold. If the phase difference is greater than the phase threshold, the lens position can be adjusted based on the phase difference. For example, feedback can be sent to the motor system to instruct the motor to move the lens to a specified position and enter the next round of focusing operation. The above process is repeated to obtain the phase difference of the next round. Until the phase difference obtained in a certain round is less than or equal to the phase difference threshold, it indicates that the sharpness of the image acquired in that round meets the requirements, the focusing state is in focus, and focusing is completed.

[0045] In the above embodiments, by acquiring noise information, the noise level of the current shooting environment can be effectively calibrated, thereby enhancing the accuracy and robustness of phase detection. This technology enables the system to more accurately identify phase differences in low signal-to-noise ratio environments, improving autofocus performance, especially in extremely low light, backlight, or weak texture scenes, significantly improving focusing reliability and speed.

[0046] During a single focusing operation, one or more images to be detected can be acquired for phase detection. For example, if only one image to be detected is acquired, the phase difference can be calculated by comparing the image data acquired from adjacent or related pixel units.

[0047] If multiple images to be detected are acquired, the phase difference can be calculated by comparing multiple image data acquired from the same pixel unit. This method provides richer information and helps improve the accuracy of phase difference calculation, especially in complex scenes.

[0048] In one embodiment, the detection pixel unit in the image sensor may include two acquisition subunits, respectively used to acquire image data from a first channel and image data from a second channel, forming a first image and a second image, which may also be referred to as a left pixel image and a right pixel image. By performing phase detection on the two image data acquired by the same detection pixel unit, the corresponding phase difference can be obtained.

[0049] In one implementation, different lens positions may affect optical refraction during image acquisition, thereby causing geometric distortion. This indicates a correlation between lens position and phase difference. Therefore, when acquiring at least one image to be detected using an image sensor, first position information corresponding to that image can be acquired simultaneously; this first position information is used to indicate the specific position of the lens when acquiring the image to be detected.

[0050] The position of the lens can refer to its specific location within its movable range, such as the lens's focal length setting, rotation angle, and coordinates in three-dimensional space.

[0051] Based on the influence of the initial position information and noise information on the phase difference, phase detection can be performed on the image to be detected, thereby obtaining the phase difference corresponding to the image. By considering the changes in lens position and noise level, the measurement accuracy and reliability of the phase difference can be improved.

[0052] For example, the influence factors of the first position information and noise information on the phase difference can be determined in advance, and the obtained phase difference can be calibrated based on these influence factors. Alternatively, a machine learning algorithm can be used to predict the phase difference under the first position information and noise information.

[0053] In one implementation, two regions with the same depth on the same imaging plane may have different phase differences. This indicates that there is a correlation between the location of the Region of Interest (ROI) and the phase difference. Therefore, when acquiring at least one image to be detected using an image sensor, second location information corresponding to that image can be obtained; this second location information is used to indicate the location of the ROI in the image.

[0054] The location of the region of interest (ROI) in the image can be determined based on the specific application scenario and requirements. For example, it can be based on image content, using edge detection algorithms, image segmentation algorithms, and other technologies to identify the main target (such as a person or object) in the image; or it can be based on user interaction selection, delineating the ROI through the user's touch operation on the interactive interface; or it can be based on motion detection, using the moving region as the ROI.

[0055] Based on the influence of second position information and noise information on the phase difference, phase detection can be performed on the image to be detected to obtain the phase difference corresponding to the image. By comprehensively considering the influence of different positions and noise conditions, the accuracy of the phase difference can be further improved.

[0056] For example, the influence factors of the first position information and noise information on the phase difference can be determined in advance, and the obtained phase difference can be calibrated based on these influence factors. Alternatively, a machine learning algorithm can be used to predict the phase difference under the second position information and noise information.

[0057] In one implementation, when at least one image to be detected is acquired by an image sensor, first position information, second position information, and noise information corresponding to the image to be detected can be obtained; then, based on the influence of the first position information, second position information, and noise information on the phase difference, phase detection is performed on the image to be detected to obtain the phase difference corresponding to the image to be detected.

[0058] Based on the above embodiments, by acquiring the first position information and the second position information, the problem of inaccurate phase difference measurement caused by the difference between the lens position and the ROI position can be solved, and the autofocus performance and image quality in complex shooting environments can be improved.

[0059] In one implementation, the acquired first position information may include a normalized value corresponding to the lens position. This normalized value is a value obtained by normalizing the lens position. Specifically, the lens adjustment range (motor travel) Range can be obtained first, for example, it can be calculated using the following formula: Range = (macro_dac - inf_dac) * (1 + macro_margin - inf_margin); where macro_dac represents the lens position at the macro limit, inf_dac represents the lens position at the telephoto limit, macro_margin expresses the macro outward expansion ratio, and inf_margin expresses the telephoto outward expansion ratio. The maximum value of normalization is set to max_step, that is, the normalized lens adjustment range is [0, max_step]. If the lens position before normalization is DAC, then the normalized lens position can be expressed as Normalized_DAC = max_step - DAC * ((max_step + 1) / Range).

[0060] By using normalized values, inaccurate phase difference measurements caused by hardware differences between lenses can be effectively avoided, thereby improving the versatility of the autofocus method. This normalization process ensures consistent performance across different devices and lenses, enhancing the system's adaptability.

[0061] In one implementation, the acquired first position information may include an encoded value corresponding to the lens position. This encoded value can be a value obtained by encoding the lens position using a preset encoding method, or it can be a value obtained by encoding a normalized value, or it can be a value obtained by encoding both the lens position and the normalized value.

[0062] In one implementation, the position of the lens can be obtained first, and then normalized to obtain a normalized value. Subsequently, the normalized value is encoded using a preset encoding method to obtain an encoded value corresponding to the lens position, which is then used as the first position information.

[0063] The encoding method can be linear encoding, non-linear encoding, or feature extraction encoding, and the specific choice can be determined according to the needs of the actual application scenario.

[0064] Linear coding: This coding method maps the input to a space of the same dimension through a linear transformation. Specifically, it divides the fixed-length discrete data into segments, denoted as `Segmented_value`. The coding formula is: `Encoded_DAC = (Normalized_DAC / / Segmented_value + 1) * Segmented_value`. This method extends the values ​​of discrete features to Euclidean space, where each segment corresponds to a discrete value in Euclidean space. For example, assuming `max_step` is 899, a possible segment value is 200; if the normalized value corresponding to the camera position is 300, the corresponding encoded value could be 00010; if the normalized value is 500, the corresponding encoded value could be 00100. This coding method effectively reduces sensitivity to small changes in the input, improving system stability.

[0065] Nonlinear coding: This coding method maps the input to a space of the same dimension through a nonlinear transformation. For example, it encodes the normalized values ​​corresponding to the camera position into a vector using a combination of sine and / or cosine functions. This vector is unique for each position in the input. For example, `Encoded_DAC = cos(Normalized_DAC / pow(10000,Normalized_DAC / max_step))` or `Encoded_DAC = sin(Normalized_DAC / pow(10000,Normalized_DAC / max_step))`, where `sin()` and `cos()` are sine or cosine functions. This coding method can learn the relationships between different positions, even if these positions are far apart in the input sequence.

[0066] Feature extraction encoding: This encoding method maps the input to a high-dimensional space using a deep neural network. Specifically, it constructs a small feature extraction network using multiple convolutional layers or multiple perceptrons, extracts features from the camera position using this feature extraction network, and then inputs the obtained feature map into the subsequent model.

[0067] Encoding can further enhance the efficiency and accuracy of data processing. Encoded values ​​can effectively reduce the complexity of data transmission and storage, and in subsequent model training and inference processes, encoded values ​​can serve as feature inputs, helping the model better learn the relationship between lens position and phase difference.

[0068] In one embodiment, the acquired second position information can be represented by a first coordinate and / or a second coordinate. The first coordinate can be defined as the center coordinate of the region of interest (ROI) in the imaging plane, and the second coordinate represents the relative position between the center of the ROI and the center of the imaging plane.

[0069] Specifically, by calculating in the horizontal and vertical directions, the relationship between the ROI center and the imaging plane center can be calibrated, thereby accurately describing the lens optical characteristics at different positions. This calibration process not only saves computational resources but also significantly improves the prediction accuracy of subsequent systems.

[0070] Assume the original resolution of the imaging plane is HxW, where H represents the height and W represents the width, and the top-left and bottom-right coordinates of the Region of Interest (ROI) are (x1, y1) and (x2, y2), respectively. Based on this, the relative distance between the center of the ROI and the center of the imaging plane can be calculated using the following formulas: Horizontal relative distance: Relative_Distance_X = (W - (x1 + x2)) / 2 and (H - (y1 + y2)) / 2; Vertical relative distance: Relative_Distance_Y = (W - (x1 + x2)) / W and (H - (y1 + y2)) / H.

[0071] In this way, the system can effectively capture the positional features of the ROI in the imaging plane and convert them into relative coordinate information.

[0072] In one implementation, AI-driven phase detection technology can be used to pre-train a phase detection model. This model performs phase detection by inputting first position information, second position information, and noise information corresponding to the image to be detected, and combining these with the image itself. The phase detection model analyzes the image to be detected and predicts the phase difference, thereby achieving efficient autofocus.

[0073] In one implementation, before inputting the first location information, second location information, noise information, and the first and second images from the image to be detected into the phase detection model, all input information can be integrated. Specifically, the first location information, second location information, noise information, and the first and second images can be uniformly normalized. During this process, the system extracts pixel unit data located within the region of interest (ROI) from the first and second images, and binds the image data of these pixel units with the corresponding first location information, second location information, and noise information at the pixel level. In this way, each pixel unit will be simultaneously bound to two sets of image data and its corresponding first location information, second location information, and noise information. Finally, the integrated data is input into the trained phase detection model to predict the phase difference.

[0074] The phase detection model can be a supervised machine learning model. Its training samples consist of image samples and their corresponding first position information, second position information, and noise information, while the sample label is the phase difference corresponding to the image sample. Depending on the specific needs, the machine learning model can be flexibly selected, including but not limited to the following types: Convolutional Neural Networks (CNN) models, Recurrent Neural Networks (RNN) models, Long Short-Term Memory (LSTM) models, Transformer models, and a series of variants based on Transformer models. These models can effectively capture complex features in the input data, improving the accuracy of phase detection and the overall performance of the system.

[0075] Figure 2 This is a schematic diagram of a module of an autofocus system according to some embodiments of the present disclosure. (Refer to...) Figure 2 The system includes: a lens optical calibration module 201, a sensor noise calibration module 202, an input fusion module 203, a phase difference determination module 204, and an autofocus module 205.

[0076] The system comprises several modules: a lens optical calibration module, a sensor noise calibration module, and an autofocus module. The former calibrates the phase difference caused by different lens positions and ROI positions, acquiring first and second position information to mask the effects of hardware differences such as lens optical characteristics and position ranges. The latter calibrates the strength of noise signals received by the sensor, acquiring noise information to enhance the system's predictive and recognition capabilities in high-noise scenes. The latter normalizes and integrates the received calibration results (first, second, and noise information) to provide input to the phase difference determination module. The latter feeds the received input into a trained phase detection model, which performs phase detection on the left and right pixel images to determine the phase difference between them. The final autofocus module provides feedback to the entire system, feeding back the obtained phase difference to the autofocus system. Based on the strategy, the system moves the lens position using the motor, completing the current focusing operation and initiating subsequent focusing operations.

[0077] 1. Lens Optical Calibration Module: This module can include a lens position normalization unit, a lens position calibration unit, and a ROI position calibration unit. Since different types of lenses have different travel distances, performing individual optical calibration for all lenses is difficult and impractical. First, the lens position normalization unit uses a linear transformation to convert the different travel distances into a fixed range, thus masking the hardware differences between different lenses. Specifically, the lens adjustment range (motor travel) Range(macro_dac-inf_dac)*(1+macro_margin-inf_margin) can be obtained first. The lens position before normalization is DAC, and the maximum normalized value is set to max_step. The normalized lens position Normalized_DAC = max_step - DAC * ((max_step+1) / Range), and the normalized lens position range is [0, max_step]. Next, the lens position calibration unit encodes the input lens position. The encoded lens position Encoded_DAC = f(Normalized_DAC), where f(x) represents the encoding method. Optional encoding methods can include linear encoding, nonlinear encoding, and feature extraction encoding. Finally, the region of interest (ROI) location calibration unit calibrates the relative position of the ROI on the imaging plane. By calculating in the horizontal and vertical directions, the relative positional relationship between the ROI center and the center of the imaging plane is determined, thereby accurately describing the lens optical characteristics at different positions. This saves computational resources while improving the prediction accuracy of subsequent systems. Assuming the original resolution of the imaging plane is HxW, the upper-left and lower-right coordinates of the ROI are (x1, y1) and (x2, y2) respectively, and the relative distances between the ROI center and the center of the imaging plane are (W-(x1+x2)) / 2 and (H-(y1+y2)) / 2. The calibrated coordinates of the ROI center relative to the center of the imaging plane are (W-(x1+x2)) / W and (H-(y1+y2)) / H.

[0078] 2. Sensor Noise Calibration Module: This module includes an environmental noise calibration unit and a noise augmentation unit. The environmental noise calibration unit is responsible for actually calibrating the total amount of light received by the sensor and breaking it down into the sensor's analog signal gain (gain) multiplied by the exposure time. The gain value calibrates the noise level of the current exposure environment, i.e., Light_quantity() = gain * exposure_time. Specifically, the training scene is first fixed, and the optimal gain and exposure_time for the current shooting environment are obtained through automatic exposure. The gain value is extracted separately as the environmental noise calibration value temp_gain. Similarly, the noise augmentation unit can control the noise intensity by keeping the total amount of light constant and dynamically adjusting the noise level and exposure time, thereby achieving the goal of covering different levels of noise and improving the system's ability to identify noise. Assuming the range of gain is [0, 256], a reference gain list can be set as [1, 2, 4, 8, 32, 64, 120, 144, 168, 192, 216, 240]. In the same calibration scenario, for each gain, the exposure time is adjusted sequentially to keep the total amount of light constant. After both settings are established, the lens is set to the furthest point of macro photography and moved towards infinity at fixed intervals. The original left and right pixel images (first image and second image) at each position throughout the process are collected as training image samples. Explicitly calibrated noise information helps the system enhance its prediction ability in low signal-to-noise ratio, high-noise environments, thereby improving the model's generality. In some embodiments, the sensor noise calibration module includes a smart mobile device containing the hardware sensor to be calibrated, a fixed tripod, and corresponding debugging tools. For example, for Android system mobile devices, the corresponding debugging device is an Android Debug Bridge.

[0079] 3. Input Integration Module: This module integrates the inputs provided by the two auxiliary calibration modules and provides the required inputs to the trained supervised learning model. The module first normalizes the calibrated lens optical information (first and second position information) and noise information along with the original left and right pixel images, and then fills in the corresponding ROI positions according to the input. This achieves pixel-by-pixel information alignment, ensuring that each input original pixel image has strongly bound optical and noise calibration information explicitly provided to the subsequent supervised learning model, greatly simplifying the task of predicting image phase difference.

[0080] 4. Phase Difference Determination Module: This module mainly includes a trained machine learning model. This model treats the phase detection autofocus task as a regression task, taking the integrated pixel-aligned ROI information as input and the sub-pixel level predicted phase difference of the entire ROI as output. In some embodiments, the trained machine learning model includes at least one of the following: a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN) model, a Long Short-Term Memory (LSTM) model, a Transformer model, and a series of variants based on the Transformer model.

[0081] 5. Autofocus Module: This module is responsible for the final action decision of the overall system. This module judges the phase difference output by the phase difference determination module. If the current output focus status is "focused", the current focusing process of the system ends; otherwise, the output phase difference value is fed back to the motor system, pushing the motor to move a specified distance and start the next focusing.

[0082] Figure 3 This is a block diagram illustrating an autofocus device according to some embodiments of the present disclosure. (Refer to...) Figure 3 The image acquisition unit 301, the phase detection unit 302, and the focusing unit 303 are included.

[0083] The image acquisition unit 301 is configured to acquire an image to be detected and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor.

[0084] The phase detection unit 302 is configured to perform phase detection on the image to be detected based on the noise information to obtain the phase difference.

[0085] The focusing unit 303 is configured to perform focusing operations based on phase difference.

[0086] Optionally, the image acquisition unit 301 is further configured to acquire first position information and second position information corresponding to the image to be detected; wherein, the first position information is used to indicate the position of the lens when acquiring the image to be detected; and the second position information is used to indicate the position of the region of interest (ROI) in the image.

[0087] The phase detection unit 302 is configured to perform phase detection on the image to be detected based on first position information, second position information and noise information to obtain the phase difference.

[0088] Optionally, the image to be detected includes: a first image and a second image; wherein the first image includes image data acquired through a first channel, and the second image includes image data acquired through a second channel.

[0089] Optionally, the first location information includes at least one of the following:

[0090] The position of the camera;

[0091] The normalized value corresponding to the position of the lens; where the normalized value is the value obtained after normalizing the position of the lens based on the adjustment range of the lens;

[0092] The encoded value corresponding to the position of the lens; wherein, the encoded value is the value obtained by encoding the position or normalized value of the lens through a preset encoding method.

[0093] Optionally, the encoding method includes at least one of the following:

[0094] Linear encoding;

[0095] Nonlinear coding;

[0096] Feature extraction encoding.

[0097] Optionally, the second location information is used to indicate at least one of the following:

[0098] The first coordinate is used to represent the center of the ROI in the imaging plane;

[0099] The second coordinate is used to represent the relative position between the center of the ROI and the center of the imaging plane.

[0100] Optionally, the phase detection unit 302 is configured to input the first position information, the second position information, and the noise information into the image to be detected into a pre-trained phase detection model, and output the phase difference; wherein, the phase detection model is a machine learning model obtained by supervised training with image samples and training samples corresponding to the image samples as inputs, and the phase difference corresponding to the image samples as sample labels.

[0101] Optionally, the machine learning model includes at least one of the following:

[0102] Convolutional Neural Network (CNN) model;

[0103] Recurrent neural network model;

[0104] Long Short-Term Memory (LSTM) network model;

[0105] Transformer model.

[0106] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0107] Figure 4 This is a block diagram illustrating an autofocus device 400 according to some embodiments of the present disclosure. For example, device 400 may be a terminal such as a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0108] Reference Figure 4 The device 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0109] Processing component 402 typically controls the overall operation of device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0110] Memory 404 is configured to store various types of data to support the operation of device 400. Examples of this data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0111] The power supply component 406 provides power to the various components of the device 400. The power supply component 406 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 400.

[0112] Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or a system with focal length and optical zoom capabilities.

[0113] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0114] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0115] Sensor assembly 414 includes one or more sensors for providing status assessments of various aspects of device 400. For example, sensor assembly 414 may detect the on / off state of device 400, the relative positioning of components such as the display and keypad of device 400, changes in the position of device 400 or a component of device 400, the presence or absence of user contact with device 400, the orientation or acceleration / deceleration of device 400, and temperature changes of device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0116] Communication component 416 is configured to facilitate wired or wireless communication between device 400 and other devices. Device 400 can access wireless networks based on communication standards, such as WiFi, 3G, 4G, 5G, other communication standards, or combinations thereof. In some embodiments of this disclosure, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In some embodiments of this disclosure, communication component 416 further includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0117] In some embodiments of this disclosure, the apparatus 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0118] In some embodiments of this disclosure, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions that can be executed by a processor 420 of device 400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0119] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform an autofocus method, the method comprising: acquiring an image to be detected and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor; performing phase detection on the image to be detected based on the noise information to obtain a phase difference; and performing a focusing operation based on the phase difference.

[0120] Some embodiments of this disclosure also provide a chip system, such as Figure 5As shown, the chip system includes at least one processor 501 and at least one interface circuit 502. The processor 501 and the interface circuit 502 are interconnected via lines. For example, the interface circuit 502 can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit 502 can be used to send signals to other devices (e.g., the processor 501). Exemplarily, the interface circuit 502 can read instructions stored in memory and send those instructions to the processor 501. When the instructions are executed by the processor 501, the device for autofocus can perform the steps in the above embodiments. Of course, the chip system may also include other discrete components, and some embodiments of this disclosure do not specifically limit this.

[0121] In some embodiments of this disclosure, the interface circuit 502 can acquire data, program instructions, and / or information from the internal storage area of ​​the chip system; it can also acquire data, program instructions, and / or information from outside the chip system.

[0122] Optionally, the chip system also includes a memory 503 for storing necessary computer programs and data.

[0123] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0124] In the above detailed description, reference has been made to the accompanying drawings, which illustrate specific aspects of this disclosure by way of illustration. In this regard, terms indicating direction or positional relationship, such as “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “rear,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” “counterclockwise,” “axial,” “radial,” and “circumferential,” are used with reference to the orientation of the described figures. Since components of the described device can be positioned in multiple different orientations, directional terms are used for illustrative purposes and not for limitation. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concept of this disclosure. Therefore, the following detailed description should not be considered limiting.

[0125] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.

[0126] It should be understood that, unless otherwise expressly specified and limited, the terms "joining," "attaching," "installing," "connecting," "linking," "fixing," etc., used in the embodiments of this disclosure should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms herein based on the specific circumstances.

[0127] Furthermore, the term "above" as used herein with respect to components, elements, or material layers formed or located "above" a surface may be used to indicate that the component, element, or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements, or layers are arranged between the surface and the component, element, or material layer. However, the term "above" as used with respect to components, elements, or material layers formed or located "above" a surface may also optionally have a specific meaning: that the component, element, or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, for example, in direct contact with the surface.

[0128] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0129] It should be understood that spatial relative terms, such as “above,” “upper,” “below,” and “lower,” are used herein to describe the relationship between one element and another shown in the figures. In addition to the orientation depicted in the figures, these spatial relative terms are also intended to encompass different orientations of the device in use or operation. For example, if the device in the figures is flipped, an element described as “above” or “upper” relative to another element would be “below” or “lower” relative to that other element. Thus, depending on the spatial orientation of the device, the term “above” encompasses both above and below orientations. Devices may have other orientations (e.g., rotated 90 degrees or in other orientations), and the spatial relative terms used herein should be interpreted accordingly.

[0130] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0131] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0132] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0133] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An autofocus method, characterized in that, include: Acquire an image to be detected, and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor; Based on the noise information, phase detection is performed on the image to be detected to obtain the phase difference; Focusing is performed based on the phase difference.

2. The method according to claim 2, characterized in that, The method further includes: Acquire first position information and second position information corresponding to the image to be detected; wherein, the first position information is used to indicate the position of the lens when acquiring the image to be detected; and the second position information is used to indicate the position of the region of interest (ROI) in the image; The step of performing phase detection on the image to be detected based on the noise information to obtain the phase difference includes: Phase detection is performed on the image to be detected based on the first location information, the second location information, and the noise information to obtain the phase difference.

3. The method according to claim 1 or 2, characterized in that, The image to be detected includes: a first image and a second image; wherein the first image includes image data acquired through a first channel, and the second image includes image data acquired through a second channel.

4. The method according to claim 2, characterized in that, The first location information includes at least one of the following: The position of the lens; The normalized value corresponding to the position of the lens; wherein the normalized value is the value obtained after normalizing the position of the lens based on the adjustment range of the lens; The encoded value corresponding to the position of the lens; wherein the encoded value is a value obtained by encoding the position or normalized value of the lens through a preset encoding method.

5. The method according to claim 4, characterized in that, The encoding method includes at least one of the following: Linear encoding; Nonlinear coding; Feature extraction encoding.

6. The method according to claim 2, characterized in that, The second location information is used to indicate at least one of the following: A first coordinate, which is used to represent the center of the ROI in the imaging plane; The second coordinate is used to represent the relative position between the center of the ROI and the center of the imaging plane.

7. The method according to claim 1, characterized in that, The step of performing phase detection on the image to be detected based on the first location information, the second location information, and the noise information to obtain the phase difference includes: The first position information, the second position information, and the noise information are input into the image to be detected into a pre-trained phase detection model, and the phase difference is output. The phase detection model is a machine learning model obtained by supervised training with image samples and training samples corresponding to the image samples as inputs, and the phase difference corresponding to the image samples as sample labels.

8. The method according to claim 3, characterized in that, The machine learning model includes at least one of the following: Convolutional Neural Network (CNN) model; Recurrent neural network model; Long Short-Term Memory (LSTM) network model; Transformer model.

9. An automatic focusing device, characterized in that, include: An image acquisition unit is used to acquire an image to be detected and noise information corresponding to the image to be detected; wherein the noise information is used to indicate the noise level of the current shooting environment on the image sensor; A phase detection unit is used to perform phase detection on the image to be detected based on the noise information to obtain the phase difference; A focusing unit is used to perform focusing operations based on the phase difference.

10. A terminal, characterized in that, include: processor; Memory used to store computer programs or instructions that can be executed by a processor; The processor is configured to execute a computer program or instructions to implement the steps of the autofocus method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium, wherein when a computer program or instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the autofocus method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the autofocus method as described in any one of claims 1 to 8.

Images