Aperture position determination method, aperture position determination apparatus, and storage medium

Abstract

The present disclosure relates to a method and device for determining aperture position and storage medium. The method comprises: in response to a change in lens position during a camera application shooting process of a terminal, acquiring a current variable aperture measurement value, a current optical image stabilization measurement value and a current autofocus measurement value during the lens movement; and determining a current variable aperture position corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value and the current autofocus measurement value based on an aperture position determination model. The present disclosure can ensure the accuracy of the variable aperture position determination result by determining the variable aperture position based on the variable aperture measurement value, the current optical image stabilization measurement value and the current autofocus measurement value acquired in real time during the camera shooting process of the terminal.

Description

Technical Field

[0001] This disclosure relates to the field of photography, and in particular to a method for determining aperture position, a device for determining aperture position, and a storage medium. Background Technology

[0002] The variable aperture (Aperture) of a camera controls the amount of light entering the camera during shooting. A larger aperture allows for greater light throughput, resulting in a brighter image and enabling clear imaging in low-light scenes, while also providing a shallower depth of field. Conversely, a smaller aperture results in less light throughput, a darker image, but sharper images and a deeper depth of field. Based on the different imaging effects at different apertures, variable aperture represents a new direction in the development of camera imaging.

[0003] In related technologies, there is a strong coupling relationship between the variable aperture, optical image stabilization, and focusing devices of the terminal camera. This results in changes in the sensor values ​​corresponding to the aperture even when the variable aperture position remains unchanged, due to changes in the optical image stabilization and focusing devices. Therefore, when determining the variable aperture position (aperture stop) of the terminal camera, the aperture position cannot be determined solely based on the sensor values ​​corresponding to the aperture. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an aperture position determination method, an aperture position determination device, and a storage medium.

[0005] According to a first aspect of the present disclosure, an aperture position determination method is provided, comprising: responding to a change in the lens position during a camera application's photo-taking process on a terminal, acquiring a current variable aperture measurement value, a current optical image stabilization measurement value, and a current autofocus measurement value during the lens movement process; and determining the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on an aperture position determination model.

[0006] In one embodiment, determining the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on the aperture position determination model includes: inputting the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value into the aperture position determination model, and the aperture position determination model outputting the current position of the variable aperture.

[0007] In one embodiment, the aperture position determination model is determined by the correspondence between the actual position of the variable aperture and the measured values ​​of the variable aperture, optical image stabilization, and autofocus.

[0008] In one embodiment, the aperture position determination model is determined as follows: a training dataset and a test dataset are collected. The training dataset includes corresponding variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the actual position of the variable aperture. The test dataset includes variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the corresponding predicted variable aperture position, which are different from those in the training dataset. The aperture position determination model is trained based on the training dataset. The trained aperture position determination model is verified based on the test dataset.

[0009] In one embodiment, the acquisition of the training dataset and the test dataset includes: for each parameter among the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value, keeping one parameter constant, adjusting the other two parameters, and determining the corresponding actual position of the variable aperture; acquiring multiple sets of data corresponding to the actual positions of the variable aperture and the variable aperture measurement value, the optical image stabilization measurement value, and the autofocus measurement value; determining a portion of the multiple sets of corresponding data as the training dataset, and determining another portion of the multiple sets of corresponding data as the test dataset.

[0010] In one embodiment, training the aperture position determination model based on the training dataset includes: normalizing the training dataset to obtain normalized data; and training the aperture position determination model based on the normalized data and a prediction neural network.

[0011] In one embodiment, training the aperture position determination model based on the normalized data and the predictive neural network includes: obtaining the mean square error corresponding to the training data; determining that the accuracy of the aperture position determination model meets the accuracy requirements in response to the mean square error being less than or equal to an error threshold, and completing the training of the aperture position determination model.

[0012] In one embodiment, verifying the trained aperture position determination model based on the test dataset includes: normalizing the variable aperture measurement values, optical image stabilization measurement values, and autofocus measurement values ​​in the test dataset; inputting the normalized variable aperture value, optical image stabilization value, and autofocus value into the aperture position determination model, and the aperture position determination model outputting predicted data; denormalizing the predicted data to obtain the predicted aperture position; determining the difference between the predicted variable aperture position and the variable aperture position in the test dataset; and completing the verification of the aperture position determination model in response to the difference being less than or equal to a difference threshold.

[0013] According to a second aspect of the present disclosure, an aperture position determination device is provided, comprising: an acquisition unit, configured to acquire a current variable aperture measurement value, a current optical image stabilization measurement value, and a current autofocus measurement value during the lens movement process in response to a change in the lens position during a camera application photo capture on a terminal; and a processing unit, configured to determine the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on an aperture position determination model.

[0014] In one embodiment, the processing unit determines the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on the aperture position determination model in the following manner: the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value are input into the aperture position determination model, and the aperture position determination model outputs the current position of the variable aperture.

[0015] In one embodiment, the aperture position determination model is determined by the processing unit using the correspondence between the actual position of the variable aperture and the measured values ​​of the variable aperture, optical image stabilization, and autofocus.

[0016] In one embodiment, the processing unit of the aperture position determination model determines the aperture position as follows: A training dataset and a test dataset are collected. The training dataset includes corresponding variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the actual position of the variable aperture. The test dataset includes variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the corresponding predicted variable aperture position, which are different from those in the training dataset. The aperture position determination model is trained based on the training dataset. The trained aperture position determination model is verified based on the test dataset.

[0017] In one embodiment, the processing unit collects training and test datasets in the following manner: for each parameter among the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value, one parameter is kept constant while the other two parameters are adjusted, and the corresponding actual position of the variable aperture is determined. Multiple sets of data corresponding to the actual positions of the variable aperture and the variable aperture measurement value, the optical image stabilization measurement value, and the autofocus measurement value are obtained respectively.

[0018] A portion of the multiple sets of corresponding data is determined as the training dataset, and another portion of the multiple sets of corresponding data is determined as the test dataset.

[0019] In one embodiment, the processing unit trains the aperture position determination model based on the training dataset in the following manner: normalizing the training dataset to obtain normalized data; and training the aperture position determination model based on the normalized data and the prediction neural network.

[0020] In one embodiment, the processing unit trains the aperture position determination model based on the normalized data and the prediction neural network in the following manner: obtaining the mean square error corresponding to the training data; in response to the mean square error being less than or equal to an error threshold, determining that the accuracy of the aperture position determination model meets the accuracy requirements, and completing the training of the aperture position determination model.

[0021] In one embodiment, the processing unit verifies the trained aperture position determination model based on the test dataset in the following manner: normalizing the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value in the test dataset; inputting the normalized variable aperture value, optical image stabilization value, and autofocus value into the aperture position determination model, and the aperture position determination model outputs predicted data; denormalizing the predicted data to obtain the predicted aperture position; determining the difference between the predicted variable aperture position and the variable aperture position in the test dataset; and verifying the aperture position determination model in response to the difference being less than or equal to a difference threshold.

[0022] According to a third aspect of the present disclosure, an aperture position determination apparatus is provided, comprising: a processor: a memory for storing processor-executable instructions; wherein the processor is configured to: execute the aperture position determination method described in the first aspect or any embodiment of the first aspect.

[0023] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the aperture position determination method described in the first aspect or any embodiment of the first aspect.

[0024] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: When the lens position changes during the camera application's photo-taking process on a terminal, the current position of the variable aperture is determined through an aperture position determination model based on the current camera sensor data collected in real time during the camera's photo-taking process. The camera sensor data includes variable aperture measurement values, optical image stabilization measurement values, and autofocus measurement values. This disclosure ensures the accuracy of the variable aperture position determination result.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0027] Figure 1 This is a schematic diagram illustrating a partial structure of a terminal camera according to an exemplary embodiment of the present disclosure.

[0028] Figure 2 This is a flowchart illustrating an aperture position determination method according to an exemplary embodiment.

[0029] Figure 3 This is a flowchart illustrating a method for determining the actual position of an aperture according to an exemplary embodiment.

[0030] Figure 4 This is a flowchart illustrating a method for determining an aperture position determination model according to an exemplary embodiment.

[0031] Figure 5 This is a flowchart illustrating a method for collecting training and testing datasets according to an exemplary embodiment.

[0032] Figure 6 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment.

[0033] Figure 7 This is a schematic diagram of a neural network structure for predicting aperture position according to an exemplary embodiment of the present disclosure.

[0034] Figure 8 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment.

[0035] Figure 9 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment.

[0036] Figure 10 This is a block diagram illustrating an aperture position determining device according to an exemplary embodiment.

[0037] Figure 11 This is a block diagram illustrating an apparatus for determining the aperture position according to an exemplary embodiment. Detailed Implementation

[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0039] The aperture position determination method provided in this disclosure is applied to scenarios where the current aperture level is determined during the terminal photography process.

[0040] When taking a photo, the size of the camera aperture affects the amount of light entering the camera. A larger aperture allows more light in, resulting in a brighter image and clearer images in low-light conditions. It also provides a shallower depth of field, enabling background blurring and highlighting the subject. Conversely, a smaller aperture allows less light in, resulting in a darker image, but sharper images and a deeper depth of field. Based on the different imaging effects of different apertures and considering the diverse imaging needs of end users, variable aperture represents a new development direction for photographic and video shooting on mobile devices.

[0041] In related technologies, such as Figure 1 As shown in the schematic diagram of a portion of the terminal camera's structure, there is a strong coupling relationship between the terminal camera's variable aperture, optical image stabilization (OIS), and autofocus. The aperture adjustment, OIS, and autofocus interact with each other. Based on this structure, even if the variable aperture position remains unchanged, the sensor values ​​corresponding to the aperture will change due to the OIS and autofocus mechanisms. Therefore, determining the variable aperture position (aperture stop) cannot be based solely on the corresponding sensor values. Furthermore, due to the strong coupling between the variable aperture, OIS, and autofocus, it is impossible to establish an explicit model that includes the correspondence between various sensor values ​​(variable aperture value, OIS value, autofocus value) and the aperture position (aperture stop), thus preventing the direct determination of the aperture position based solely on the sensor values ​​collected during image capture.

[0042] In view of this, this application proposes an aperture position determination method, which collects the sensor values ​​of various sensors of the terminal camera under multiple sets of different variables and the corresponding aperture position information as training data, trains a neural network model to obtain an aperture position determination model, and outputs the current sensor values ​​of the terminal to the trained aperture position determination model to predict the true aperture value.

[0043] Figure 2This is a flowchart illustrating an aperture position determination method according to an exemplary embodiment. Figure 2 As shown, the method includes steps S101 to S102.

[0044] In step S101, in response to a change in the lens position during the camera application's photo-taking process on the terminal, the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value are obtained during the lens movement process.

[0045] In step S102, based on the aperture position determination model, the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value is determined.

[0046] In this embodiment of the disclosure, such as Figure 1 As shown, due to the strong coupling between the variable aperture, optical image stabilization, and autofocus of the terminal camera, the variable aperture measurement value is affected by the optical image stabilization and autofocus during the shooting process. This results in different aperture measurement values ​​corresponding to the same variable aperture position, and the same aperture measurement value corresponding to different variable aperture positions. Based on the strong coupling between the terminal camera and the terminal camera, the determination of the camera aperture position needs to consider the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value of the terminal camera at the same time.

[0047] In this embodiment of the disclosure, the current position of the variable aperture is determined by the aperture position determination model based on the current camera sensor values ​​(including the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value) collected in real time during the shooting process. That is, multiple parameters are used to locate the current position of the variable aperture, which can accurately locate the aperture position and avoid misjudgment caused by determining the aperture position only based on the aperture measurement value, resulting in an unsatisfactory final imaging effect.

[0048] In this embodiment, due to the strong coupling relationship between the variable aperture, optical image stabilization, and camera focusing devices of the terminal camera, it is impossible to establish an explicit model that includes the correspondence between various sensor values ​​(variable aperture value, optical image stabilization value, camera focusing value) and aperture position (aperture stop). Therefore, this disclosure uses a pre-trained aperture position determination model to determine the current position of the variable aperture. The following embodiments of this disclosure further illustrate the method for determining the aperture position.

[0049] Figure 3 This is a flowchart illustrating a method for determining the actual position of an aperture according to an exemplary embodiment. Figure 3 As shown, the method includes steps S201 to S202.

[0050] In step S201, the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value are input into the aperture position determination model.

[0051] The aperture position determination model takes the current variable aperture measurement, the current optical image stabilization measurement, and the current autofocus measurement as inputs, and outputs the current position of the variable aperture.

[0052] In step S202, the aperture position determination model outputs the current position of the variable aperture.

[0053] In this embodiment, the pre-trained neural network model (aperture position determination model) outputs the current position of the variable aperture based on the input of the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value. This enables the terminal to quickly and efficiently obtain the current position of the aperture and adjust the aperture size according to needs to obtain the desired depth of field and brightness, resulting in an ideal photographed image.

[0054] It is understood that the aperture position determination model needs to be trained based on the collected real data. The following embodiments of this disclosure illustrate the method for determining the aperture position determination model.

[0055] In one embodiment of this disclosure, the aperture position determination model is determined by the correspondence between the actual position of the variable aperture and the measured values ​​of the variable aperture, optical image stabilization, and autofocus.

[0056] In this embodiment of the disclosure, based on the acquired camera sensor parameters and the corresponding variable aperture positions, the correspondence between the camera sensor parameters and the corresponding variable aperture positions is abstracted, i.e., AP rea =f(OIS) x OIS y ,AF z AP mea The aperture position is determined and the model is established based on the correspondence at the abstract point.

[0057] Among them, AP rea This refers to the actual position of the variable aperture, which can be the aperture setting (e.g., 0, 1); OIS x OIS y For optical image stabilization measurements, AF z For optical zoom measurements, AP mea The aperture measurement value is the actual aperture position, which can be determined by observation. The optical image stabilization measurement value, optical zoom measurement value, and aperture measurement value can be obtained by the Hall sensor in the terminal camera device.

[0058] In this embodiment of the disclosure, the process of determining the aperture position determination model includes model training and post-training validation, wherein the data used for training and the data used for validation are two different datasets. The following embodiments of this disclosure illustrate the method for determining the aperture position determination model.

[0059] Figure 4 This is a flowchart illustrating a method for determining an aperture position determination model according to an exemplary embodiment. Figure 4 As shown, the method includes steps S301 to S303.

[0060] In step S301, the training dataset and the test dataset are collected.

[0061] The training dataset includes corresponding variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the actual position of the variable aperture. The test dataset includes variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the corresponding predicted position of the variable aperture, which are different from those in the training dataset.

[0062] In step S302, an aperture position determination model is trained based on the training dataset.

[0063] In step S303, the aperture position determination model is verified based on the test dataset.

[0064] In this embodiment of the disclosure, a training dataset for model training and a test dataset for model verification are collected respectively. After training the neural network with the training dataset to obtain an aperture position determination model with accuracy that meets the accuracy requirements, the accuracy of the aperture position determination model is further verified a second time with the test dataset to obtain an aperture position determination model with accuracy that meets the requirements.

[0065] In this embodiment, the actual collected data are used as training and testing data. Since determining the variable aperture position requires simultaneous consideration of variable aperture measurements, optical image stabilization measurements, and autofocus measurements—meaning that if one of these three measurements remains constant while the other two change—the variable aperture position may also change. Therefore, this disclosure uses a controlled variable method to collect training data, resulting in training and testing datasets. The following embodiments of this disclosure illustrate the method for collecting training and testing datasets.

[0066] Figure 5 This is a flowchart illustrating a method for collecting training and testing datasets according to an exemplary embodiment. Figure 5 As shown, the method includes steps S401 to S402.

[0067] In step S401, for each parameter among the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value, one parameter is kept constant while the other two parameters are adjusted, and the corresponding actual position of the variable aperture is determined. Multiple sets of data corresponding to the actual position of the variable aperture and the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value are obtained respectively.

[0068] In step S402, a portion of the multiple sets of corresponding data is determined as the training dataset, and another portion of the multiple sets of corresponding data is determined as the test dataset.

[0069] In this embodiment, one of the variable aperture measurement, optical image stabilization measurement, and autofocus measurement remains constant, while the other two change, and the position of the variable aperture may also change. Different aperture measurements correspond to the same aperture position, and the same aperture value corresponds to different aperture positions. Based on the combined influence of multiple measurements on the aperture position, this disclosure uses a controlled variable method to obtain training and testing datasets. For example, different groups of variable aperture measurements are set, and for each group of variable aperture measurements, the other two measurements are adjusted and the corresponding aperture position is determined. The same process is then performed for the other two measurements to finally obtain the training dataset.

[0070] In this embodiment of the disclosure, the training dataset and test dataset are obtained by controlling the variable method to ensure the richness of the training data and test data, so that the final aperture position determination model can make accurate judgments on the aperture position for different combinations of measurement values.

[0071] It is understood that the variable aperture measurement, optical image stabilization measurement, and autofocus measurement each have different measurement standards and corresponding ranges. Therefore, the training data in the training dataset needs further processing before it can be used for model training. The following embodiments of this disclosure further illustrate the method for determining the aperture position to determine the model.

[0072] Figure 6 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment. For example... Figure 6 As shown, the method includes steps S501 to S502.

[0073] In step S501, the training dataset is normalized to obtain normalized data.

[0074] In step S502, an aperture position determination model is trained based on the normalized data and the prediction neural network.

[0075] In this embodiment of the disclosure, since the measurement standards for the variable aperture measurement value, optical image stabilization measurement value, autofocus measurement value, and variable aperture position are different, it is necessary to normalize the above measurement values ​​and constrain them to the same range (e.g., constrain to 0-1). This facilitates data processing by the neural network model to be trained and ensures the accuracy of the final aperture position determination model when determining the position.

[0076] In an exemplary embodiment of this disclosure, a backpropagation neural network is used as the neural network model to be trained, such as... Figure 7 This is a schematic diagram of a neural network structure used to predict aperture position. The backpropagation neural network consists of an input layer, hidden layers, and an output layer. The first layer is the input layer for the variables, the middle layer is the hidden layer, and the last layer is the output layer for predicting the variables. Each neuron in the neural network receives all the outputs of the neurons in the previous layer. The weight parameters are trained through network training, and the network output is... The activation function of the hidden layer is the sigmoid function, w i It is variable x i The weights are denoted by , and b represents the bias of the neuron. It is understandable that the activation functions of hidden layers can also include other activation functions such as trigonometric functions.

[0077] In this embodiment, the mean squared error (MSE) is obtained through the training set (please refer to relevant resources for specific concepts). The MSE is used to determine the prediction accuracy, thus obtaining an aperture position determination model whose prediction accuracy meets the accuracy requirements. The following embodiments further illustrate the method for determining the aperture position determination model.

[0078] Figure 8 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment. For example... Figure 8 As shown, the method includes steps S601 to S602.

[0079] In step S601, the mean square error corresponding to the training data is obtained.

[0080] The prediction data includes variable aperture measurements, optical image stabilization measurements, autofocus measurements, and corresponding predicted variable aperture positions, which are different from the training data in the training dataset.

[0081] In step S602, in response to the mean square error being less than or equal to the error threshold, it is determined that the accuracy of the aperture position determination model meets the accuracy requirements, and the training of the aperture position determination model is completed.

[0082] In this embodiment, the mean squared error (MSE) of the neural network model is obtained based on the actual collected training data to determine the prediction accuracy. This disclosure pre-sets an error threshold representing the critical value at which the prediction accuracy of the network model reaches a critical value. By comparing the MSE with the preset error threshold, it is determined whether the prediction accuracy of the prediction model meets the requirements. When the prediction accuracy of the neural network model meets the requirements for further accuracy assessment, the training of the aperture position determination model is completed.

[0083] Understandably, when the mean squared error is greater than the error threshold, the prediction accuracy of the neural network model does not meet the requirements and cannot be used as the final aperture position determination model. The prediction model needs to be repeatedly trained until the mean squared error is less than or equal to the error threshold and the prediction accuracy of the model meets the requirements.

[0084] It is understood that variable aperture measurements, optical image stabilization measurements, and autofocus measurements each have different measurement standards and corresponding ranges. Therefore, the training data in the training dataset needs further processing before it can be used for model training. The following embodiments of this disclosure further illustrate the method for determining the aperture position to determine the model.

[0085] Figure 9 This is a flowchart illustrating a method for determining an aperture position determination model according to yet another exemplary embodiment. For example... Figure 9 As shown, the method includes steps S701 to S705.

[0086] In step S701, the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value in the test dataset are normalized.

[0087] In step S702, the normalized variable aperture value, optical image stabilization value, and autofocus value are input into the aperture position determination model, and the aperture position determination model outputs predicted data.

[0088] In step S703, the predicted data is denormalized to obtain the aperture prediction position.

[0089] In step S704, the degree of difference between the predicted variable aperture position and the variable aperture position in the test dataset is determined.

[0090] In step S705, in response to the difference degree being less than or equal to the difference degree threshold, the verification of the aperture position determination model is completed.

[0091] In this embodiment, after training the aperture position determination model, the model is validated using a test dataset. Corresponding to the processing method of the training dataset, the test dataset also needs to undergo normalization. The variable aperture measurement values, optical image stabilization measurement values, and autofocus measurement values ​​in the test dataset are normalized. The normalized data is then input into the aperture position determination model. The model outputs predicted data, which is the associated data of the predicted variable aperture position. This predicted data is then denormalized to obtain the predicted variable aperture position. The accuracy of the aperture position determination model is verified by comparing the predicted variable aperture position with the actual variable aperture position collected in the test dataset. It is understood that the actual variable aperture position and the predicted variable aperture position used for comparison are data under the same variable aperture value, optical image stabilization value, and autofocus value. This disclosure compares multiple sets of actual variable aperture positions with predicted variable aperture positions to determine the proportion (i.e., the degree of difference) of inaccurate data in the multiple sets of predicted variable aperture positions, and compares it with a preset degree of difference threshold. When the degree of difference is less than or equal to the degree of difference threshold, it indicates that the accuracy of the aperture position determination model has met the accuracy requirements and can be used for real-time determination of the variable aperture position during shooting.

[0092] Understandably, similar to the judgment process for mean squared error, when the difference between the actual and predicted positions of multiple variable apertures exceeds the difference threshold, the prediction accuracy of the neural network model does not meet the requirements and cannot be used as the final aperture position determination model. A model training and model validation process is required until the prediction accuracy of the aperture position determination model meets the requirements.

[0093] In this embodiment, the actual position of the variable aperture and the corresponding optical image stabilization, optical zoom, and aperture measurements are used as training and testing datasets. A neural network model is trained using the training dataset, and the accuracy of the aperture position determination model's predictions is verified using validation data. This results in an aperture position determination model that meets the required accuracy and can be used for real-time aperture position determination. During actual camera shooting, in response to changes in camera position, the current variable aperture measurement, current autofocus measurement, and current optical image stabilization measurement are acquired. These real-time measurements are then input into the pre-trained aperture position determination model to obtain the current variable aperture position. This disclosure, by determining the variable aperture position based on the real-time acquisition of current variable aperture, current optical image stabilization, and current autofocus measurements during camera shooting, ensures the accuracy of the variable aperture position determination result, guarantees the effectiveness of the camera's depth-of-field and brightness adjustments, and ultimately ensures the final camera imaging effect.

[0094] Based on the same concept, this disclosure also provides an aperture position determining device 100.

[0095] It is understood that the aperture position determination device 100 provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0096] Figure 10 This is a block diagram illustrating an aperture position determining device 100 according to an exemplary embodiment. (Refer to...) Figure 10 The device includes an acquisition unit 101 and a processing unit 102.

[0097] The acquisition unit 101 is used to acquire the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value during the lens movement process in response to a change in the lens position during the camera application of the terminal.

[0098] The processing unit 102 is used to determine the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on the aperture position determination model.

[0099] In one embodiment, the processing unit 102 determines the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on the aperture position determination model in the following manner: the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value are input into the aperture position determination model, and the aperture position determination model outputs the current position of the variable aperture.

[0100] In one embodiment, the aperture position determination model is determined by the processing unit 102 using the correspondence between the actual position of the variable aperture and the measured values ​​of the variable aperture, optical image stabilization, and autofocus.

[0101] In one embodiment, the aperture position determination model processing unit 102 determines the aperture position as follows: It collects a training dataset and a test dataset. The training dataset includes corresponding variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the actual position of the variable aperture. The test dataset includes variable aperture measurements, optical image stabilization measurements, autofocus measurements, and the corresponding predicted variable aperture position, which are different from those in the training dataset. Based on the training dataset, the aperture position determination model is trained. Based on the test dataset, the trained aperture position determination model is validated.

[0102] In one embodiment, the processing unit 102 collects the training dataset and the test dataset in the following manner:

[0103] For each parameter in the variable aperture measurement, optical image stabilization measurement, and autofocus measurement, keep one parameter constant while adjusting the other two parameters, and determine the corresponding actual position of the variable aperture. Obtain multiple sets of data corresponding to the actual position of the variable aperture and the variable aperture measurement, optical image stabilization measurement, and autofocus measurement.

[0104] One portion of the multiple sets of corresponding data is selected as the training dataset, and another portion of the multiple sets of corresponding data is selected as the test dataset.

[0105] In one embodiment, the processing unit 102 trains the aperture position determination model based on the training dataset in the following manner: The training dataset is normalized to obtain normalized data. Based on the normalized data and the prediction neural network, the aperture position determination model is trained.

[0106] In one embodiment, the processing unit 102 trains an aperture position determination model based on normalized data and a predictive neural network in the following manner: It obtains the mean square error corresponding to the training data. In response to the mean square error being less than or equal to an error threshold, it determines that the accuracy of the aperture position determination model meets the accuracy requirements, thus completing the training of the aperture position determination model.

[0107] In one embodiment, the processing unit 102 verifies the trained aperture position determination model based on a test dataset in the following manner: normalizing the variable aperture measurement values, optical image stabilization measurement values, and autofocus measurement values ​​in the test dataset; inputting the normalized variable aperture values, optical image stabilization values, and autofocus values ​​into the aperture position determination model, which then outputs predicted data; denormalizing the predicted data to obtain the predicted aperture position; determining the degree of difference between the predicted variable aperture position and the variable aperture position in the test dataset; and verifying the aperture position determination model when the degree of difference is less than or equal to a difference threshold.

[0108] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0109] Figure 11 This is a block diagram illustrating an apparatus 200 for determining aperture position according to an exemplary embodiment. The apparatus 200 can be provided as a terminal. For example, the apparatus 200 can be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0110] Reference Figure 11 The device 200 may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.

[0111] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.

[0112] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0113] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.

[0114] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0115] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.

[0116] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0117] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0118] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0119] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0120] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0121] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0122] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0123] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0124] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0125] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0126] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0127] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for determining the aperture position, characterized in that, include: In response to a change in lens position during the camera application's photo-taking process on the terminal, the current variable aperture measurement value, current optical image stabilization measurement value, and current autofocus measurement value are obtained during the lens movement process. Based on the aperture position determination model, the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value is determined; The aperture position determination model is determined in the following way: A training dataset and a test dataset are collected. The training dataset includes corresponding variable aperture measurement values, optical image stabilization measurement values, autofocus measurement values, and the actual position of the variable aperture. The test dataset includes variable aperture measurement values, optical image stabilization measurement values, autofocus measurement values, and the corresponding actual position of the variable aperture, which are different from those in the training dataset. The aperture position determination model is trained based on the training dataset. The trained aperture position determination model is validated based on the test dataset.

2. The method according to claim 1, characterized in that, The aperture position determination model determines the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value, including: The current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value are input into the aperture position determination model, and the aperture position determination model outputs the current position of the variable aperture.

3. The method according to claim 1, characterized in that, The aperture position determination model is determined by the correspondence between the actual position of the variable aperture and the measured values ​​of the variable aperture, optical image stabilization, and autofocus.

4. The method according to claim 1, characterized in that, The collected training and test datasets include: For each parameter among the variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value, one parameter is kept constant while the other two parameters are adjusted, and the corresponding actual position of the variable aperture is determined. Multiple sets of data corresponding to the actual position of the variable aperture and the variable aperture measurement value, the optical image stabilization measurement value, and the autofocus measurement value are obtained respectively. A portion of the multiple sets of corresponding data is determined as the training dataset, and another portion of the multiple sets of corresponding data is determined as the test dataset.

5. The method according to claim 1, characterized in that, Training the aperture position determination model based on the training dataset includes: The training dataset is normalized to obtain normalized data; The aperture position determination model is trained based on the normalized data and the predictive neural network.

6. The method according to claim 5, characterized in that, The step of training the aperture position determination model based on the normalized data and the prediction neural network includes: Obtain the mean square error corresponding to the training data; In response to the mean square error being less than or equal to the error threshold, it is determined that the accuracy of the aperture position determination model meets the accuracy requirements, and the training of the aperture position determination model is completed.

7. The method according to claim 1, characterized in that, The step of verifying the trained aperture position determination model based on the test dataset includes: The variable aperture measurement value, optical image stabilization measurement value, and autofocus measurement value in the test dataset are normalized. The normalized variable aperture measurement value, the optical image stabilization measurement value, and the autofocus measurement value are input into the aperture position determination model, and the aperture position determination model outputs predicted data. The predicted data is denormalized to obtain the aperture prediction position; Determine the degree of difference between the predicted position of the variable aperture and the actual position of the variable aperture in the test dataset; In response to the difference being less than or equal to the difference threshold, the verification of the aperture position determination model is completed.

8. An aperture position determining device, characterized in that, include: The acquisition unit is used to acquire the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value during the lens movement process in response to a change in the lens position during the camera application of the terminal. The processing unit is used to determine the current position of the variable aperture corresponding to the current variable aperture measurement value, the current optical image stabilization measurement value, and the current autofocus measurement value based on the aperture position determination model; The aperture position determination model is determined by the processing unit in the following manner: A training dataset and a test dataset are collected. The training dataset includes corresponding variable aperture measurement values, optical image stabilization measurement values, autofocus measurement values, and the actual position of the variable aperture. The test dataset includes variable aperture measurement values, optical image stabilization measurement values, autofocus measurement values, and the corresponding actual position of the variable aperture, which are different from those in the training dataset. The aperture position determination model is trained based on the training dataset. The trained aperture position determination model is validated based on the test dataset.

9. An aperture position determining device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the aperture position determination method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the processor to perform the aperture position determination method according to any one of claims 1 to 7.

Images