Lens control method and device, equipment and storage medium

Abstract

The invention provides a lens control method and device, equipment and a storage medium. The method comprises the following steps: in a shooting mode, determining a first shaking degree of the electronic equipment and a first target position of a first lens assembly in the electronic equipment according to measurement data of at least one inertial sensor in the electronic equipment in a first time window; and under the condition that the first shaking degree is large-angle shaking, controlling an optical anti-shaking unit in the electronic equipment to move the first lens assembly to the first target position and then lock the first lens assembly at the first target position.

Description

Technical Field

[0001] The present application relates to electronic technology, and is related to but not limited to lens control methods and devices, equipment, and storage media. Background Art

[0002] When a user holds an electronic device to shoot, the hand will shake to a certain extent even when it is stationary, or the hand will shake when the user is in motion. The shaking of the hand will be transmitted to the handheld electronic device, causing the electronic device to shake in one or more directions, which in turn causes the image captured by the lens assembly to be blurred. Figure 1 There are water ripples in the captured image (such as the captured content in the rectangular frame 101 ). It can be seen that the captured content is blurred due to the presence of the water ripples. Summary of the invention

[0003] The lens control method, device, equipment, and storage medium provided by the present application include:

[0004] In a first aspect, an embodiment of the present application provides a lens control method, comprising: in a shooting mode, determining a first shake degree of the electronic device and a first target position of a first lens assembly in the electronic device based on measurement data of at least one inertial sensor in the electronic device in a first time window; when the first shake degree is a large-angle shake, controlling an optical image stabilization unit in the electronic device to move the first lens assembly to the first target position and then lock it at the first target position.

[0005] In a second aspect, an embodiment of the present application provides a lens control device, comprising: a determination module, configured to, in a shooting mode, determine a first shake degree of the electronic device and a first target position of a first lens assembly in the electronic device based on measurement data of at least one inertial sensor in the electronic device in a first time window; and a control module, configured to, when the first shake degree is a large-angle shake, control an optical image stabilization unit in the electronic device to move the first lens assembly to the first target position and then lock it at the first target position.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a lens control device, an optical image stabilization unit, at least one inertial sensor, a first lens assembly and a display screen; wherein the lens control device is configured to: in a shooting mode, determine a first shake degree of the electronic device and a first target position of the first lens assembly based on measurement data of at least one inertial sensor in a first time window; and when the first shake degree is a large-angle shake, control the optical image stabilization unit to move the first lens assembly to the first target position and then lock it at the first target position; the display screen is configured to display a picture taken by the first lens assembly after the movement.

[0007] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, the method described in the embodiment of the present application is implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the embodiment of the present application.

[0009] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the method provided in the embodiment of the present application.

[0010] It can be understood that in the embodiment of the present application, in the shooting mode, when the shaking degree of the electronic device is large-angle shaking, the first lens assembly is controlled to move to the first target position and then lock the first target position unchanged; in this way, under large-angle shaking, the difference between the theoretical value of the first target position (i.e., the value determined based on the measurement data output by the inertial sensor) and the actual position of the first lens assembly (i.e., the actual position when the first lens assembly moves to the first target position) is fixed, thereby achieving the purpose of optimizing the water ripple problem and further improving the image quality of the picture taken by the first lens assembly.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0014] Figure 1 This is an example of a shot with water ripples;

[0015] Figure 2A schematic diagram of the relationship between the real-time data of the hall and the target provided in the embodiment of the present application;

[0016] Figure 3 A schematic diagram of the implementation flow of the lens control method provided in the embodiment of the present application;

[0017] Figure 4 A schematic diagram of an implementation flow of a method for determining a first shaking degree and a first target position provided in an embodiment of the present application;

[0018] Figure 5 A schematic diagram of an implementation flow of a method for determining a current position of a first lens assembly provided in an embodiment of the present application;

[0019] Figure 6 A schematic diagram of an interrupt hardware circuit provided in an embodiment of the present application;

[0020] Figure 7 A schematic diagram of the judgment logic of large-angle shaking provided in an embodiment of the present application;

[0021] Figure 8 A schematic diagram of an optimized target-hall waveform provided in an embodiment of the present application;

[0022] Fig. 9 A schematic diagram of the structure of a lens control device provided in an embodiment of the present application;

[0023] Fig.10 A schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 1 ;

[0024] Fig.11 A schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the specific technical scheme of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0027] In the following description, reference is made to “some embodiments”, “this embodiment”, “embodiments of the present application” and examples, etc., which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] The descriptions of “first, second, third” etc. that appear in the embodiments of the present application are only for illustration and distinction of the description objects. There is no distinction of order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0029] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies or terms of the embodiments of the present application. The following related technologies or related terms can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0030] (1) Optical image stabilization technology

[0031] Optical Image Stabilization (OIS) technology is a technology widely used in imaging devices such as cameras and smartphones. It aims to reduce or eliminate image blur caused by device shaking through physical means, thereby improving image quality.

[0032] The core of optical image stabilization technology is to compensate for the jittery optical path through movable components (such as lens components). That is, the principle of optical image stabilization is: according to the measurement data of the inertial sensor in the electronic device, the target position of the lens component in the shooting mode of the electronic device is determined; based on this, the lens component is controlled to move to the target position, thereby offsetting the impact of the jitter, and then ensuring the stability of the optical path to reduce the blur of the image.

[0033] Exemplarily, in a possible implementation, a first offset value (ie, a compensation displacement) between a target position and a current position of the lens assembly may be determined, and the movement of the lens assembly may be controlled based on the compensation displacement.

[0034] In the embodiment of the present application, the functional module that implements the optical image stabilization technology is called an optical image stabilization unit. For example, the optical image stabilization unit may be an OIS IC, etc.

[0035] (2) Inertial sensor

[0036] Inertial sensors are an important sensor technology that is mainly used to measure the change in motion of electronic devices (i.e., measurement data), such as measuring the acceleration, tilt, shock, vibration, rotation, and multi-degree-of-freedom (DoF) motion of electronic devices.

[0037] According to the measurement principle, inertial sensors include accelerometers and gyroscopes; accelerometers can measure the linear acceleration of the device in three-dimensional space, including the acceleration of gravity. When the device shakes at a large angle, the accelerometer can capture the acceleration changes generated by the device during the shaking process. The gyroscope can measure the rotational angular velocity / shaking angular velocity of the device around its axes, providing real-time information about the rotation of the device. When the device shakes at a large angle, the gyroscope can accurately sense the rotation / shaking direction and speed of the device. By integrating the angular velocity output by the gyroscope, the rotation / shaking angle of the device relative to the initial position can be obtained.

[0038] According to the degree of integration, inertial sensors include single-component inertial sensors and inertial measurement units (IMUs); among them, single-component inertial sensors: such as a single accelerometer or gyroscope, are used to measure a single physical quantity. Inertial measurement unit (IMU): is a device that integrates multiple inertial sensors, usually including accelerometers and gyroscopes, or other sensors such as magnetometers. IMU can simultaneously measure information such as the acceleration, angular velocity, and angle of the device, and fuse the data of the two to provide a more comprehensive and accurate state of motion.

[0039] According to the classification of technology type, inertial sensors include MEMS inertial sensors, fiber optic gyroscopes and laser gyroscopes; among them, MEMS inertial sensors: miniature inertial sensors manufactured using micro-nano technology, with the characteristics of miniaturization, low power consumption, high precision, high reliability and low cost. Fiber optic gyroscope: obtains the angular velocity information of the device by measuring the change of the light path, with high precision and stability. Laser gyroscope: uses laser technology to measure angular velocity, also with high precision and stability.

[0040] In the embodiment of the present application, there is no limitation on the type of the inertial sensor, and the inertial sensor includes one or more types mentioned above.

[0041] (3) Displacement sensor

[0042] Displacement sensors are a type of sensor used to measure the position or displacement of lens components. They can use a variety of working principles and technologies to achieve different displacement measurement tasks.

[0043] According to the working principle, displacement sensors can include: Hall effect displacement sensor, ultrasonic displacement sensor, photoelectric displacement sensor, capacitive displacement sensor, inductive displacement sensor, etc.

[0044] Among them, the working principle of the Hall effect displacement sensor is to measure the displacement of the lens assembly by detecting the Hall effect in the magnetic field. When the lens assembly is close to the displacement sensor, the magnetic field distribution around the displacement sensor will change, thereby generating a Hall voltage, and the displacement of the lens assembly can be inferred by measuring the change in this voltage.

[0045] Among them, the working principle of the capacitive displacement sensor is to use the capacitance effect to measure the displacement of the lens assembly and estimate the position of the lens assembly by measuring the capacitance change between two electrodes. When the distance between the lens assembly and the electrode changes, the capacitance value will change accordingly.

[0046] Among them, the working principle of the inductive displacement sensor is to use the change in inductance of the lens assembly to measure the displacement, usually through a coil and a magnetic core. When the relative position between the lens assembly and the magnetic core changes, the inductance in the coil will also change accordingly, and the displacement of the lens assembly can be inferred by measuring this change.

[0047] The electronic device has a built-in displacement sensor, which senses the corresponding displacement data (that is, the target physical quantity, such as Hall voltage, capacitance change, inductance change, etc.) to obtain the current position of the corresponding lens assembly, that is, the actual position of the lens assembly.

[0048] In one possible implementation, a calibration operation is performed on the electronic device before the electronic device leaves the factory. Calibration refers to establishing a mapping from the sensor code corresponding to the target physical quantity to the displacement. The calibration can obtain a mapping relationship between the sensor code output by the displacement sensor and the actual displacement. In this way, during the shooting process, the electronic device can use the above-mentioned calibrated mapping relationship to convert the output code of the displacement sensor into displacement, and thus obtain the current position of the corresponding lens assembly.

[0049] In the embodiments of the present application, there is no limitation on the type of displacement sensor, and the displacement sensor can be various types of sensors.

[0050] In mobile phones, OIS-based telephoto shooting modes are increasingly used, but due to the limitations of image sensors and motor structures, all telephoto module previews will have varying degrees of water ripples when they are shaken. Examples of water ripples are as mentioned above. Figure 1 ,from Figure 1It can be seen that the image is deformed, blurred, and distorted. Based on this, the inventor of the present application found through research and analysis that the main reason for the generation of water ripples is that the hall information of the lens assembly (i.e., the actual position of the lens assembly) cannot keep up with the target parameter (i.e., the target position of the lens assembly), resulting in image rotation. The optimization solution in the industry for this problem is to reduce the gain of the gyroscope (gyro gain), thereby sacrificing the anti-shake capability of the overall OIS IC to reduce water ripples.

[0051] However, although the optimization scheme of reducing the gain of the gyroscope is simple to implement, it cannot effectively optimize the water ripple problem, and it does not optimize the problem in principle. For example, this scheme has the following problems:

[0052] 1) Reducing the gain of the gyroscope will reduce the overall anti-shake effect and have a long impact period; 2) This solution does not solve the problem that the water ripple hall cannot keep up with the target, but only improves it by reducing the anti-shake capability, but the effect is not good.

[0053] Based on this, the inventor of the present application conducted research and analysis based on the principle of the water ripple problem, and based on this, formulated a corresponding optimization plan to achieve the optimization of water ripple distortion.

[0054] The analysis of the principle of water ripple problem is as follows.

[0055] Figure 2 A schematic diagram of the relationship between the real-time data of the hall and the target provided in the embodiment of the present application; Figure 2 As shown, curve 201 is the real-time data of target, curve 202 is the real-time data of hall, and curve 203 is the difference between target and hall; wherein, Figure 2 The horizontal axis is the time unit in ms, and the vertical axis is the step code.

[0056] It should be noted that target is the target value for controlling the position of the lens assembly, that is, controlling the lens assembly to move to the target position corresponding to the target value; hall represents the actual position of the lens assembly;

[0057] The inventor of this application is based on Figure 2The analysis of the data shown in the figure shows that the water ripples are caused by the large fluctuation and long duration of the difference between the target and the hall when the phone is shaken at a large angle. Simply put, the proportional, integral, differential coefficient (PID) control cannot keep up with the change of the target. In other words, the fluctuation of the difference between the target and the hall at the same time is large, that is, there is a large difference between the target that controls the target position of the lens assembly and the actual position of the lens assembly, and the PID control accuracy is poor.

[0058] In view of the analysis of the principle of the above technical problem, an embodiment of the present application provides a lens control method, in which a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device are determined according to measurement data of at least one inertial sensor in the electronic device in a first time window; wherein the first lens assembly is in a shooting mode; when the first shaking degree is a large-angle shaking, an optical image stabilization unit in the electronic device is controlled to move the first lens assembly to a first target position and then lock it at the first target position unchanged;

[0059] In this way, under large-angle shaking, the difference between the theoretical value of the first target position (i.e., the target value determined based on the measurement data output by the inertial sensor) and the actual position of the first lens assembly (i.e., the actual position hall when the first lens assembly moves to the first target position) is fixed, that is, the difference between the target and the hall is locked unchanged, thereby achieving the purpose of optimizing the water ripple problem and further improving the image quality of the picture taken by the first lens assembly.

[0060] The lens control method provided in the embodiment of the present application is applied to an electronic device, and the electronic device can be various types of devices with shooting functions during the implementation process, for example, the electronic device can include a mobile phone, a tablet computer, smart glasses, a smart watch, a pan-tilt camera, a drone, a sports camera, a portable camera, etc. The function implemented by the method can be implemented by calling a program code by a processor in the electronic device, and of course the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0061] Figure 3 A schematic diagram of the implementation flow of the lens control method provided in the embodiment of the present application is shown in FIG. Figure 3 As shown, the method may include the following steps 301 to 302:

[0062] Step 301, in a shooting mode, determining a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window;

[0063] Step 302: When the first shaking degree is a large-angle shaking, control the optical image stabilization unit in the electronic device to move the first lens assembly to a first target position and then lock it at the first target position.

[0064] It can be understood that in the embodiment of the present application, when the shaking degree of the electronic device is large-angle shaking, the first lens assembly is controlled to move to the first target position and then lock the first target position unchanged; in this way, under large-angle shaking, the difference between the theoretical value of the first target position (i.e., the value determined based on the measurement data output by the inertial sensor) and the actual position of the first lens assembly (i.e., the actual position when the first lens assembly moves to the first target position) is fixed, thereby achieving the purpose of optimizing the water ripple problem and further improving the image quality of the picture taken by the first lens assembly.

[0065] The following describes further optional implementations and related terms of each of the above steps.

[0066] Step 301, in a shooting mode, determining a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window.

[0067] In the embodiment of the present application, there is no limitation on the type of the first lens assembly, which may be various types of lens assemblies, for example, the lens assembly may be a telephoto lens assembly, a wide-angle lens assembly, an ultra-wide-angle lens assembly, a macro lens assembly, or a main camera lens assembly.

[0068] In some embodiments, Figure 4 As shown, step 301 can be implemented through the following steps 3011 to 3014:

[0069] Step 3011: Determine a first shake change of the electronic device according to measurement data of the at least one inertial sensor in the first time window.

[0070] In a possible implementation, the time length of the first time window can be predefined. The first time window can be the time length from the current moment to a certain historical moment; for example, the time length of the first time window is T, the current moment is recorded as t1, and the starting moment of the first time window is (t1-T). The measurement data of the same inertial sensor in the first time window includes the measurement data collected at different moments in the first time window, such as the angular velocity collected by the gyroscope at different moments in the first time window.

[0071] In some embodiments, the first shake variation includes one or more of a shake angle, a shake displacement, and an acceleration.

[0072] It can be understood that since different types of inertial sensors measure different physical quantities, the types of the corresponding first shake variation are also different. For example, for a gyroscope, it measures the angular velocity of rotation, and the angular velocity of rotation collected at different times in the first time window can be integrated, and the obtained integral result (i.e., rotation angle / shake angle) is the first shake variation, wherein the angular velocity of rotation collected at different times in the first time window can be understood as an example of the measurement data of the inertial sensor; for another example, for an accelerometer, it measures linear acceleration, and the acceleration collected at different times in the first time window can be integrated, and the obtained integral result (i.e., rotation speed / shake speed) can be the first shake variation, or, the shake displacement of the electronic device can be further determined based on the shake speed, and the shake displacement is used as the first shake variation; wherein, the acceleration collected at different times in the first time window can be understood as an example of the measurement data of the inertial sensor. Of course, the mean value of the acceleration in the first time window can also be used as the first shake variation; in short, the type of the first shake variation corresponding to the embodiment of the present application is not limited.

[0073] Step 3012: determine the first target position according to the first shake change amount.

[0074] Step 3013: determine a first offset value between the current position of the first lens assembly and the first target position.

[0075] In the embodiment of the present application, there is no limitation on the method for determining the actual position (i.e., the current position) of the first lens assembly. Figure 5 As shown, the current position of the first lens assembly can be determined through the following steps 501 to 503:

[0076] Step 501: Acquire first displacement data of a first lens assembly measured by a displacement sensor.

[0077] As described above for displacement sensors, different types of displacement sensors measure different types of first displacement data (i.e., target physical quantities). For example, for a Hall effect displacement sensor, the first displacement data measured is a Hall voltage, which includes Hall voltages collected at multiple different times; for another example, for a capacitive displacement sensor, the first displacement data measured is a capacitance change, which includes capacitance changes collected at multiple different times.

[0078] Step 502, filtering the first displacement data to obtain second displacement data; the filtering process includes low-pass filtering and / or mean filtering.

[0079] It can be understood that the purpose of low-pass filtering the first displacement data is to remove noise data in the first displacement data, and the purpose of mean filtering is the same. It can be understood that removing noise data in the first displacement data is beneficial to improving the accuracy / precision of the current position in step 503, thereby improving the accuracy / precision of the first offset value between the current position of the first lens assembly and the first target position, and further improving the accuracy / precision of the first shake degree.

[0080] Step 503: Determine the current position of the first lens assembly according to the second displacement data.

[0081] As described above, in one possible implementation, a calibration operation is performed on the electronic device before it leaves the factory. Calibration refers to establishing a mapping from the sensor code corresponding to the target physical quantity (i.e., characterizing the first displacement data) to the displacement. The calibration can obtain a mapping relationship between the sensor code output by the displacement sensor and the actual displacement. In this way, during the shooting process, the electronic device can use the above-mentioned calibrated mapping relationship to convert the output code of the displacement sensor into displacement, and thus obtain the current position of the corresponding lens assembly.

[0082] Step 3014: Determine a first shake degree of the electronic device according to a relationship between the first offset value and / or the first shake variation and a corresponding threshold value.

[0083] Exemplarily, in some embodiments, when the first offset value is greater than or equal to a first threshold value, and / or when at least one of the first shake changes is greater than or equal to a corresponding second threshold value, it is determined that the first shake degree of the electronic device is a large-angle shake. For example, when the first shake angle of the electronic device is greater than a first angle threshold value, and the first offset value is greater than or equal to the first threshold value, it is determined that the first shake degree of the electronic device is a large-angle shake.

[0084] Exemplarily, in some embodiments, when the first offset value is less than or equal to a third threshold value, and / or when at least one of the first shake changes is less than or equal to a fourth threshold value, the first shake degree of the electronic device is determined to be a small-angle shake; wherein the third threshold value is greater than or equal to the first threshold value, and the fourth threshold value is less than or equal to the second threshold value. For example, when the first shake angle of the electronic device is less than the second angle threshold value, the first shake degree of the electronic device is determined to be a small-angle shake; wherein the first angle threshold value is greater than the second angle threshold value.

[0085] Step 302: When the first shaking degree is a large-angle shaking, control the optical image stabilization unit in the electronic device to move the first lens assembly to a first target position and then lock it at the first target position.

[0086] Further, in some embodiments, step 302 may include: when the first shake degree is a large-angle shake, sending a first control signal to the optical image stabilization unit, the first control signal being used to instruct the first lens assembly to be moved to a first target position; after sending the first control signal or after determining that the first lens assembly has moved to the first target position, sending a second control signal to the optical image stabilization unit to trigger the optical image stabilization unit to stop working, thereby locking the first lens assembly at the first target position.

[0087] Furthermore, in some embodiments, the second control signal is an interrupt signal. After sending the first control signal or determining that the first lens assembly has moved to the first target position, the interrupt signal can be sent to the optical image stabilization unit by triggering the first pin (for example, a GPIO pin) to trigger the optical image stabilization unit to stop working, thereby locking the first lens assembly at the first target position; in this way, the optical image stabilization unit can be quickly triggered to stop working through the interrupt signal, thereby locking the first lens assembly at the first target position, which is beneficial to further enhance the optimization degree of the water ripple problem.

[0088] In the case where the first shake degree of the electronic device is a small-angle shake, after sending the first control signal to the optical image stabilization unit, the second control signal is no longer sent to the optical image stabilization unit. Instead, the third target position is determined based on the measurement data of at least one inertial sensor in the next time window, and the optical image stabilization unit is controlled to move the first lens assembly to the third target position.

[0089] In the embodiment of the present application, the next time window of the first time window may overlap with the first time window, or may not overlap with the first time window, which is related to the sliding step of the time window. For example, the time length of the first time window is T, and the end time is t1, then the starting time of the first time window is (t1-T); the starting time of the next time window of the first time window is (t1-T+sliding step), and the end time of the next time window of the first time window is (t1+sliding step); wherein the sliding step may be a predefined time length.

[0090] It should be noted that in the embodiment of the present application, the electronic device has different processing methods for different shaking degrees of the electronic device. When the shaking degree of the electronic device is large-angle shaking, the electronic device locks the first lens assembly at the first target position unchanged, and subsequently, even if the electronic device shakes at a large angle again, the first lens assembly is no longer controlled to move until the shaking state of the electronic device changes to small-angle shaking.

[0091] That is, in some embodiments, the method further includes: determining a second shake degree of the electronic device and a second target position of the first lens assembly based on measurement data of the at least one inertial sensor in a second time window; wherein the second time window is after the first time window; and when the second shake degree is a small-angle shake, controlling the optical image stabilization unit to move the first lens assembly from the first target position to the second target position.

[0092] The second time window can be understood by referring to the description of the first time window above. The second time window is different from the first time window in that the second time window is after the first time window, that is, the starting time of the second time window is after the starting time of the first time window, and the ending time of the second time window is after the ending time of the first time window. In a possible implementation, the time length of the second time window is equal to the time length of the first time window. The second time window can be the time length from the current moment to a certain historical moment; for example, the time length of the second time window is T, the current moment is recorded as t2, and the starting time of the second time window is (t2-T). The measurement data of the same inertial sensor in the first time window includes the measurement data collected at different times in the second time window, such as the angular velocity collected by the gyroscope at different times in the second time window.

[0093] It should be noted that the current moment t1 corresponding to the first time window mentioned above can also be understood as the end moment of the first time window, and the current moment t2 corresponding to the second time window can also be understood as the end moment of the second time window. Therefore, the current moment t2 described here and the current moment t1 mentioned above are two different moments, and t2 occurs before t1.

[0094] In the embodiment of the present application, the method for determining the second shaking degree is the same as the method for determining the first shaking degree described above, except that the second shaking degree is determined based on the measurement data of the second time window. The method for determining the second target position is also the same as the method for determining the first target position described above, except that the second target position is determined based on the measurement data of the second time window.

[0095] In the embodiment of the present application, the control method of the optical image stabilization unit to move the first lens assembly to the target position is not limited. The first lens assembly can be gradually moved to the target position based on a feedback mechanism, for example, based on a proportional, integral, differential coefficient (PID) control algorithm to gradually move the first lens assembly to the target position.

[0096] The following examples describe possible implementations of the lens control method described in one or more of the above embodiments.

[0097] In an embodiment of the present application, a complete dynamic optimization solution is proposed for the characteristics of water ripple distortion. The solution is to lock (severon) to the current target without changing when it is identified that the electronic device is shaking at a large angle and detects that |target-hall|>200code (set threshold), thereby forcibly fixing the position of the first lens assembly to the target. Target can be understood as the first target position mentioned above, hall can be understood as the current position of the first lens assembly, and |target-hall| can be understood as the first offset value mentioned above. When it is identified that the gyro change (an example of the shake change) is less than 0.2°, enter OIS ON and turn on the OIS function normally.

[0098] It should be noted that, here, “200 code” can be understood as an example of the first threshold value, and “0.2°” can be understood as an example of the fourth threshold value.

[0099] The following describes the relevant detection schemes of the dynamic optimization scheme of water ripples provided in the embodiments of the present application.

[0100] It can be understood that the detection scheme mainly includes three parts: recognition of large-angle shaking, detection of |target-hall|, and detection of gyro change.

[0101] When a large-angle shaking of the electronic device is detected, an interruption mode is used to trigger the device, such as Figure 6 As shown, the platform processor 601 processes the gyro data, and based on the processing result of the gyro data, when it is determined that the device has a large-angle shake, the OIS IC 602 is triggered by an interruption method to lock (severons) the first lens assembly to the target unchanged; as shown Figure 6 As shown, the platform processor 601 in the electronic device sends an interrupt signal to the pin INT of the OIS IC 602 through the GPIO pin to trigger the OIS IC 602 to lock (severons) the first lens assembly to the target unchanged.

[0102] Figure 6 The interrupt hardware circuit shown is similar in principle to the return interface. It can be understood that the processing performance of the platform processor 601 is better than that of the OIS IC 602. Therefore, this solution uses the platform processor 601 to process the gyro data, which has the advantages of faster processing speed, multi-channel data can be compared horizontally, and higher accuracy. The hardware interrupt circuit responds quickly and the overall performance is better.

[0103] Figure 7 A schematic diagram of the judgment logic of large-angle shaking provided in an embodiment of the present application; Figure 7 As shown, the platform processor 601 identifies large-angle shaking through GYRO data from both the UI and OIS channels.

[0104] like Figure 7 As shown, when the integral angles of the GYRO data of both the UI and OIS channels are greater than 1° (i.e., an example of the second threshold), it is determined that the device has experienced large-angle shaking; thus, the judgment logic based on multi-channel GYRO data can effectively avoid false triggering. The Gyro integral angle time is 1s, which is obtained through conventional simulation of user environments and can more accurately judge large angles.

[0105] |target-hall| is determined. The hall (i.e., an example of the first displacement data) and the target value can be read in real time through the OIS IC. The hall may have noise during use, resulting in a difference between the hall value and the actual lens value. The hall of this solution is collected at a sampling frequency of 20kHz. A 100Hz low-pass filter is first used in hall processing, and then the average is taken every 20 data to reduce the hall noise. Ultimately, the detection accuracy of |target-hall| is improved.

[0106] To determine whether the gyro change is less than 0.2°, the gyro change is determined by the GYRO data of the UI channel. Table 1 is the gyro data of the UI channel of the whole machine, and Table 2 is the gyro data of the OIS channel. Through comparative analysis, it can be seen that the accuracy of the gyro data of the UI channel is three times that of the OIS channel. Therefore, the gyro data based on the UI channel is used for small angle detection, and its accuracy is higher and the advantage is more obvious.

[0107] Table 1 Gyro data of UI road

[0108] axis Noise_RMS_AVG Noise_RMS_Density X 0.0412366 0.0018442 Y 0.0381026 0.001704 X 0.044921 0.0020089 Y 0.0408962 0.0018289 X 0.0889056 0.003976 Y 0.0813946 0.0036419

[0109] Table 2 Gyro data of OIS road

[0110]

[0111] After the inventor of the present application introduced the algorithm of the present solution (i.e. the lens control method), Figure 8 As shown, the optimized target-hall waveform 801 obtained in the process of recording a video through the first lens assembly has significantly improved overall performance and enhanced anti-water ripple distortion capability; wherein, Figure 8 The horizontal axis shows time, and the vertical axis shows the absolute value of the difference between the target and the hall.

[0112] It can be understood that the above solution provides a new solution to OIS water ripple distortion, and the advantages of this solution are as follows:

[0113] 1) Based on the principle of water ripple generation, the hall-target fluctuation is optimized, which is more versatile and can be used in all water ripple projects;

[0114] 2) Using UI’s high-precision auxiliary judgment to improve the accuracy of angle recognition and achieve better performance;

[0115] 3) This solution detects the integral angle and severons the current target, which can not only optimize the water ripples but also improve the picture jumps.

[0116] It should be noted that the above solution is not only applicable to the mobile phone industry, but also to all camera products; the above solution can not only be applied to water ripple optimization, but also to OIS micro-tremor optimization.

[0117] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.; or, steps in different embodiments may be combined into a new technical solution.

[0118] Based on the foregoing embodiments, an embodiment of the present application provides a lens control device, which includes the modules included and the units included in the modules, which can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an AI acceleration engine (such as NPU, etc.), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0119] Fig. 9 A schematic diagram of the structure of the lens control device provided in the embodiment of the present application is shown in FIG. Fig. 9 As shown, the lens control device 900 includes a determination module 901 and a control module 902, wherein:

[0120] The determination module 901 is configured to determine, in a shooting mode, a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window;

[0121] The control module 902 is configured to control the optical image stabilization unit in the electronic device to move the first lens assembly to the first target position and then lock it at the first target position when the first shaking degree is a large-angle shaking.

[0122] In some embodiments, the determination module 901 is further configured to: determine a second shake degree of the electronic device and a second target position of the first lens assembly based on measurement data of the at least one inertial sensor in a second time window; wherein the second time window is after the first time window; and the control module 902 is further configured to control the optical image stabilization unit to move the first lens assembly from the first target position to the second target position when the second shake degree is a small-angle shake.

[0123] In some embodiments, the determination module 901 is configured to: determine a first shake change amount of the electronic device based on measurement data of the at least one inertial sensor in the first time window; determine the first target position based on the first shake change amount; determine a first offset value between the current position of the first lens assembly and the first target position; determine a first shake degree of the electronic device based on the relationship between the first offset value and / or the first shake change amount and the corresponding threshold value.

[0124] Further, in some embodiments, the determination module 901 is configured to: determine that the first shake degree is a large-angle shake when the first offset value is greater than or equal to a first threshold, and / or the first shake change is greater than or equal to a corresponding second threshold; determine that the first shake degree is a small-angle shake when the first offset value is less than or equal to a third threshold, and / or the first shake change is less than or equal to a fourth threshold; wherein the third threshold is greater than or equal to the first threshold, and the fourth threshold is less than or equal to the second threshold.

[0125] In some embodiments, the determination module 901 is further configured to: obtain first displacement data of the first lens assembly measured by a displacement sensor; filter the first displacement data to obtain second displacement data; the filtering process includes low-pass filtering and / or mean filtering; and determine the current position of the first lens assembly according to the second displacement data.

[0126] In some embodiments, the control module 902 is configured to: send a first control signal to the optical image stabilization unit when the first shake degree is a large-angle shake, the first control signal being used to instruct the first lens assembly to be moved to the first target position; after sending the first control signal or after determining that the first lens assembly has moved to the first target position, send a second control signal to the optical image stabilization unit to trigger the optical image stabilization unit to stop working, thereby locking the first lens assembly at the first target position.

[0127] Further, in some embodiments, the second control signal is an interrupt signal; the control module 902 is configured to: after sending the first control signal or after determining that the first lens assembly has moved to the first target position, trigger the first pin to send an interrupt signal to the optical image stabilization unit to trigger the optical image stabilization unit to stop working, thereby locking the first lens assembly at the first target position.

[0128] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0129] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0130] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0131] An embodiment of the present application provides an electronic device, Fig.10 A schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 1 ,like Fig.10 As shown, the electronic device 100 includes: a lens control device 900, an optical image stabilization unit 1001, a first lens assembly 1002, a display screen 1003 and at least one inertial sensor 1004; wherein,

[0132] The lens control device 900 is configured to: in a shooting mode, determine a first shake degree of the electronic device 100 and a first target position of the first lens assembly 1002 according to measurement data of at least one inertial sensor 1004 in a first time window; and when the first shake degree is a large-angle shake, control the optical image stabilization unit 1001 to move the first lens assembly 1002 to the first target position and then lock it at the first target position;

[0133] The display screen 1003 is configured to display the image captured by the first lens assembly 1002 after the movement.

[0134] In some embodiments, the lens control device 900 is further configured to: determine a second shake degree of the electronic device 100 and a second target position of the first lens assembly 1002 based on measurement data of the at least one inertial sensor 1004 in a second time window; wherein the second time window is after the first time window; and when the second shake degree is a small-angle shake, control the optical image stabilization unit 1001 to move the first lens assembly 1002 from the first target position to the second target position.

[0135] For example, when the second shaking degree is a small-angle shaking, the lens control device 900 sends a third control signal to the optical image stabilization unit 1001, and the third control signal is used to instruct the optical image stabilization unit 1001 to move the first lens assembly 1002 to the second target position.

[0136] In some embodiments, the lens control device 900 is configured to: determine a first shake change amount of the electronic device 100 based on measurement data of at least one inertial sensor 1004 in a first time window; determine a first target position based on the first shake change amount; determine a first offset value between a current position of the first lens assembly 1002 and the first target position; and determine a first shake degree of the electronic device 100 based on a relationship between the first offset value and / or the first shake change amount and corresponding thresholds, respectively.

[0137] In some embodiments, the lens control device 900 is configured to: determine that the first shake degree is a large-angle shake when the first offset value is greater than or equal to a first threshold value, and / or the first shake change amount is greater than or equal to a corresponding second threshold value; determine that the first shake degree is a small-angle shake when the first offset value is less than or equal to a third threshold value, and / or the first shake change amount is less than or equal to a fourth threshold value; wherein the third threshold value is greater than or equal to the first threshold value, and the fourth threshold value is less than or equal to the second threshold value.

[0138] In some embodiments, the electronic device 100 also includes a displacement sensor; the lens control device 900 is further configured to: obtain first displacement data of the first lens assembly 1002 measured by the displacement sensor; filter the first displacement data to obtain second displacement data; the filtering process includes low-pass filtering and / or mean filtering; and determine the current position of the first lens assembly 1002 based on the second displacement data.

[0139] In some embodiments, the lens control device 900 is configured to: send a first control signal to the optical image stabilization unit 1001 when the first shake degree is a large-angle shake, and the first control signal is used to instruct the first lens assembly 1002 to move to the first target position; after sending the first control signal or determining that the first lens assembly 1002 moves to the first target position, send a second control signal to the optical image stabilization unit 1001 to trigger the optical image stabilization unit 1001 to stop working, thereby locking the first lens assembly 1002 at the first target position.

[0140] Furthermore, in some embodiments, the second control signal is an interrupt signal; the lens control device 900 is configured to: after sending the first control signal or after determining that the first lens assembly 1002 moves to the first target position, trigger the first pin (for example, a GPIO pin) to send an interrupt signal to the optical image stabilization unit 1001, so as to trigger the optical image stabilization unit 1001 to stop working.

[0141] In a possible implementation, the lens control device 900 is applied to a platform processor, the optical image stabilization unit 1001 is an OIS IC, and the processing performance of the platform processor is better than that of the OIS IC.

[0142] An embodiment of the present application provides an electronic device, Fig.11 A schematic diagram of the structure of the electronic device provided in the embodiment of the present application Figure 2 ,like Fig.11 As shown, the electronic device 110 includes a memory 1101 and a processor 1102 , wherein the memory 1101 stores a computer program that can be run on the processor 1102 , and the processor 1102 implements the steps in the method provided in the above embodiment when executing the program.

[0143] It should be noted that the memory 1101 is configured to store instructions and applications executable by the processor 1102, and can also cache data to be processed or already processed by the processor 1102 and various modules in the electronic device 110 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0144] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiment are implemented.

[0145] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the method provided in the above method embodiment.

[0146] It should be noted here that the description of the above electronic device, storage medium and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the electronic device, storage medium and computer program product embodiments of this application, please refer to the description of the method embodiment of this application for understanding.

[0147] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0148] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0149] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0150] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0151] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0152] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0153] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0154] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0155] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0156] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0157] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0158] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A lens control method, characterized in that: The method comprises: In a shooting mode, determining a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window; In the case where the first shaking degree is a large-angle shaking, the optical image stabilization unit in the electronic device is controlled to move the first lens assembly to the first target position and then lock it at the first target position.

2. The method according to claim 1, characterized in that The method further comprises: Determining a second shaking degree of the electronic device and a second target position of the first lens assembly according to measurement data of the at least one inertial sensor in a second time window; wherein the second time window is after the first time window; When the second shaking degree is a small-angle shaking, the optical image stabilization unit is controlled to move the first lens assembly from the first target position to the second target position.

3. The method according to claim 1 or 2, characterized in that: The method of determining a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window comprises: Determining a first shake change amount of the electronic device according to measurement data of the at least one inertial sensor in the first time window; determining the first target position according to the first shaking change amount; Determining a first offset value between a current position of the first lens assembly and the first target position; A first shake degree of the electronic device is determined according to a relationship between the first offset value and / or the first shake change and a corresponding threshold value.

4. The method according to claim 3, characterized in that The determining, according to the relationship between the first offset value and / or the first shake variation and the corresponding threshold value, a first shake degree of the electronic device includes: When the first offset value is greater than or equal to a first threshold value, and / or the first shaking variation is greater than or equal to a corresponding second threshold value, determining that the first shaking degree is a large-angle shaking; When the first offset value is less than or equal to a third threshold value, and / or the first shaking variation is less than or equal to a fourth threshold value, determining that the first shaking degree is a small-angle shaking; The third threshold is greater than or equal to the first threshold, and the fourth threshold is less than or equal to the second threshold.

5. The method according to claim 3, characterized in that: The method further comprises: Acquire first displacement data of the first lens assembly measured by a displacement sensor; Performing filtering processing on the first displacement data to obtain second displacement data; the filtering processing includes low-pass filtering and / or mean filtering; The current position of the first lens assembly is determined according to the second displacement data.

6. The method according to any one of claims 1 to 5, characterized in that: When the first shaking degree is a large-angle shaking, controlling the optical image stabilization unit in the electronic device to move the first lens assembly to the first target position and then lock it at the first target position includes: When the first shaking degree is a large-angle shaking, sending a first control signal to the optical image stabilization unit, wherein the first control signal is used to instruct to move the first lens assembly to the first target position; After sending the first control signal or determining that the first lens assembly moves to the first target position, sending a second control signal to the optical image stabilization unit to trigger the optical image stabilization unit to stop working, thereby locking the first lens assembly at the first target position.

7. The method according to claim 6, characterized in that The second control signal is an interrupt signal; and sending the second control signal to the optical image stabilization unit to trigger the optical image stabilization unit to stop working includes: The first pin is triggered to send an interrupt signal to the optical image stabilization unit, so as to trigger the optical image stabilization unit to stop working.

8. A lens control device, characterized in that: The device comprises: a determination module configured to determine, in a shooting mode, a first shaking degree of the electronic device and a first target position of a first lens assembly in the electronic device according to measurement data of at least one inertial sensor in the electronic device in a first time window; The control module is configured to control the optical image stabilization unit in the electronic device to move the first lens assembly to the first target position and then lock it at the first target position when the first shaking degree is a large-angle shaking.

9. An electronic device, characterized in that: The electronic device comprises: a lens control device, an optical image stabilization unit, at least one inertial sensor, a first lens assembly and a display screen; wherein, The lens control device is configured to: in a shooting mode, determine a first shake degree of the electronic device and a first target position of the first lens assembly according to measurement data of at least one inertial sensor in a first time window; and when the first shake degree is a large-angle shake, control the optical image stabilization unit to move the first lens assembly to the first target position and then lock it at the first target position; The display screen is configured to display the image captured by the first lens assembly after movement.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images