Method, device, electronic device, storage medium and program product for device control

Abstract

The present disclosure relates to the technical field of data processing, and specifically provides a device control method and device, electronic device, storage medium and program product. The method comprises: determining first movement data of a lens of an electronic device based on collected movement data of the electronic device, calculating second movement data of the lens based on collected original images; obtaining a predicted compensation value based on the first movement data and the second movement data using a focus compensation model; if a focus compensation event is detected, adjusting the second movement data based on the predicted compensation value to obtain third movement data; and controlling the lens movement based on the first movement data and the third movement data respectively. In this way, the first movement data for anti-shake and the second movement data for focus adjustment are combined for adjustment and optimization, which can improve the quality of the collected images.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to a method, apparatus, electronic device, storage medium, and program product for device control. Background Technology

[0002] During image acquisition, the device typically performs image stabilization while simultaneously adjusting focus to capture a clear image. Focus adjustment involves moving the lens back and forth to achieve focus. Image stabilization, on the other hand, counteracts random external vibrations (such as hand-held phone movement) by controlling the lens to move along a plane perpendicular to the lens's forward and backward direction.

[0003] However, the device's focus adjustment process and image stabilization process may interfere with each other, resulting in poor image quality. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a method, apparatus, electronic device, storage medium, and program product for device control.

[0005] On one hand, this disclosure provides a method for controlling a device, including:

[0006] Based on the collected motion data of the electronic device, the first motion data of the lens of the electronic device is determined. The first motion data is used to represent the direction and distance that the lens is to move on a specified plane.

[0007] Based on the acquired original image, second movement data of the lens is calculated. The second movement data is used to represent the distance that the lens needs to move in a specified direction in order to focus the lens.

[0008] A focus compensation model is used to obtain the predicted compensation value based on the first movement data and the second movement data;

[0009] If a focus compensation event is detected, the second motion data is adjusted based on the predicted compensation value to obtain the third motion data;

[0010] The lens movement is controlled based on the first movement data and the third movement data, respectively.

[0011] In one embodiment, the first movement data includes a first current for controlling the movement of the lens in a first direction and a second current for controlling the movement of the lens in a second direction; the second movement data includes an initial focusing current for controlling the movement of the lens in the specified direction; and the third movement data includes a target focusing current for controlling the movement of the lens in the specified direction.

[0012] The movement distance of the lens is positively correlated with the magnitude of the current; the specified direction is perpendicular to the specified plane.

[0013] In one embodiment, determining the first movement data of the lens of the electronic device based on the collected motion data of the electronic device includes:

[0014] The angular velocity of the electronic device is acquired using a motion sensor; the motion data includes angular velocity.

[0015] Based on the angular velocity, the offset vector of the electronic device on the specified plane is calculated;

[0016] The first current is obtained based on the component of the offset vector in the first direction;

[0017] The second current is obtained based on the component of the offset vector in the second direction; the first current and the second current are used to control the lens to move the offset vector in the opposite direction.

[0018] In one implementation, calculating the second movement data of the lens based on the acquired original image includes:

[0019] The original image is captured through the lens;

[0020] The original image is subjected to quality assessment to obtain image quality assessment results;

[0021] Based on the image quality assessment results, the initial focusing current is calculated.

[0022] In one implementation, detecting the focus compensation event includes:

[0023] Use at least one of the following:

[0024] If it is determined that the current lens adjustment state indicates focus adjustment, then the focus compensation event is detected.

[0025] If it is determined that the current lens adjustment state indicates focus adjustment, and both the first current and the second current are lower than the first current threshold and higher than the second current threshold, then it is determined that the focus compensation event has been detected.

[0026] In one embodiment, the method further includes:

[0027] The focus compensation model is trained using the following steps:

[0028] Multiple current combinations and their corresponding reference compensation values ​​are obtained, wherein the current combinations include a first current sample, a second current sample, and an initial focusing current sample.

[0029] Input each current combination into the initial compensation model to obtain the predicted compensation value corresponding to each current combination.

[0030] The predicted loss is determined based on the predicted compensation value and reference compensation value corresponding to each current combination.

[0031] Based on the predicted loss, the parameters of the initial compensation model are adjusted until a focus compensation model that meets the preset training conditions is obtained.

[0032] In one aspect, this disclosure provides a device for controlling equipment, comprising:

[0033] The determining unit is used to determine first motion data of the lens of the electronic device based on the collected motion data of the electronic device. The first motion data is used to indicate the direction and distance that the lens is to move on a specified plane.

[0034] A calculation unit is used to calculate second movement data of the lens based on the acquired original image. The second movement data is used to represent the distance that the lens needs to move in a specified direction in order to focus the lens.

[0035] The obtaining unit is used to obtain a predicted compensation value based on the first movement data and the second movement data using a focus compensation model;

[0036] An adjustment unit is used to adjust the second motion data based on the predicted compensation value to obtain the third motion data if a focus compensation event is detected.

[0037] The control unit is used to control the movement of the lens based on the first movement data and the third movement data, respectively.

[0038] In one embodiment, the first movement data includes a first current for controlling the movement of the lens in a first direction and a second current for controlling the movement of the lens in a second direction; the second movement data includes an initial focusing current for controlling the movement of the lens in the specified direction; and the third movement data includes a target focusing current for controlling the movement of the lens in the specified direction.

[0039] The movement distance of the lens is positively correlated with the magnitude of the current; the specified direction is perpendicular to the specified plane.

[0040] In one implementation, the determining unit is used to:

[0041] The angular velocity of the electronic device is acquired using a motion sensor; the motion data includes angular velocity.

[0042] Based on the angular velocity, the offset vector of the electronic device on the specified plane is calculated;

[0043] The first current is obtained based on the component of the offset vector in the first direction;

[0044] The second current is obtained based on the component of the offset vector in the second direction; the first current and the second current are used to control the lens to move the offset vector in the opposite direction.

[0045] In one embodiment, the computing unit is used for:

[0046] The original image is captured through the lens;

[0047] The original image is subjected to quality assessment to obtain image quality assessment results;

[0048] Based on the image quality assessment results, the initial focusing current is calculated.

[0049] In one embodiment, the adjustment unit is configured to: employ at least one of the following:

[0050] If it is determined that the current lens adjustment state indicates focus adjustment, then the focus compensation event is detected.

[0051] If it is determined that the current lens adjustment state indicates focus adjustment, and both the first current and the second current are lower than the first current threshold and higher than the second current threshold, then it is determined that the focus compensation event has been detected.

[0052] In one embodiment, the adjustment unit is further configured to:

[0053] The focus compensation model is trained using the following steps:

[0054] Multiple current combinations and their corresponding reference compensation values ​​are obtained, wherein the current combinations include a first current sample, a second current sample, and an initial focusing current sample.

[0055] Input each current combination into the initial compensation model to obtain the predicted compensation value corresponding to each current combination.

[0056] The predicted loss is determined based on the predicted compensation value and reference compensation value corresponding to each current combination.

[0057] Based on the predicted loss, the parameters of the initial compensation model are adjusted until a focus compensation model that meets the preset training conditions is obtained.

[0058] In one aspect, this disclosure provides an electronic device, including:

[0059] Processor; and

[0060] The memory stores computer instructions that cause the processor to perform the steps of the methods provided in the various alternative implementations of any of the aforementioned device controls.

[0061] In one aspect, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described device control methods.

[0062] On one hand, this disclosure provides a computer program product including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the steps of the method provided in any of the above-described alternative implementations of device control.

[0063] The device control method in this embodiment includes determining first motion data of the lens of the electronic device based on acquired motion data of the electronic device, wherein the first motion data represents the direction and distance the lens is to move on a specified plane; calculating second motion data of the lens based on the acquired original image, wherein the second motion data represents the distance the lens is to move in the specified direction for focusing the lens; obtaining a predicted compensation value based on the first motion data and the second motion data using a focus compensation model; adjusting the second motion data based on the predicted compensation value if a focus compensation event is detected to obtain third motion data; and controlling the lens movement based on the first motion data and the third motion data respectively. In this way, by combining the first motion data for image stabilization and the second motion data for focus adjustment, the second motion data is optimized, thereby optimizing the lens focus adjustment, reducing the impact of the conflict between image stabilization adjustment and focus adjustment on the image, and thus improving the quality of the acquired image. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of a translational OIS structure in an embodiment of this disclosure.

[0065] Figure 2 This is a flowchart of a device control method according to an embodiment of the present disclosure.

[0066] Figure 3 This is an example diagram of a three-dimensional coordinate system in an embodiment of this disclosure.

[0067] Figure 4 This is a detailed implementation flowchart of a device control method according to an embodiment of this disclosure.

[0068] Figure 5 This is a structural block diagram of a device for controlling an equipment according to an embodiment of this disclosure.

[0069] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0070] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.

[0071] The structure of an electronic device used for image acquisition, with focus adjustment and image stabilization capabilities, is described below. The electronic device includes an image acquisition unit (e.g., a camera). The image acquisition unit includes a lens module. The lens module includes a lens (also referred to as an optical lens) and a moving device for moving the lens. The moving device may include an optical image stabilization device. The optical image stabilization device may employ an optical image stabilizer (OIS) to perform optical image stabilization.

[0072] OIS (Optical Image Sensor) can employ a motor to power the lens, enabling its movement. Motors can be categorized into open-loop and closed-loop motors based on their function. Open-loop motors can be voice coil motors (VCMs), and the lens can be directly nested within the VCM. For example, a current signal can be sent to the lens drive section of the open-loop motor, allowing it to generate a corresponding driving force based on the magnitude and direction of the received current. This driving force can then move the lens in a specific direction, achieving functions such as focusing or zooming.

[0073] The primary function of a lens is to converge ambient light onto the sensor. It uses a series of optical lenses to collect light rays incident from different angles, focusing them to form a clear optical image, which is then projected onto the sensor's photosensitive surface. The optical lens can move within the lens module. The purpose of focusing is to control the movement of the lens so that a clear image is projected onto the sensor.

[0074] The focusing principle of the device is explained below. The device receives light through a sensor, generates a Raw Image Format (RAW) image, and calculates the lens's focus data based on the RAW image. This focus data is used to evaluate image sharpness, focus accuracy, and scene depth to obtain an image quality assessment result. The device performs a digital-to-analog converter (DAC) on the focus data to obtain the current used for focusing, and then transmits this current to the driver IC. The driver IC powers the VCM (Vibration Mirror Control Center) to move the VCM, thereby causing the lens to reach the specified focus position.

[0075] The principle of optical image stabilization is explained below. Electronic devices monitor device shake using a gyroscope and send the collected angular velocity to a driver integrated circuit (IC). The driver IC calculates the lens offset based on the angular velocity and controls the motor of the image stabilization module to move in the opposite direction to counteract the shake. In other words, through translation compensation, the negative impact of shake on the device's imaging is suppressed. For example, it can suppress the negative impact of hand shake on mobile phone imaging.

[0076] In image stabilization, translational OIS is typically used. (See also...) Figure 1 The diagram shown illustrates a translation-type OIS structure. The motor of a translation-type OIS typically has three sets of coils. These three sets of coils are used to control lens movement along the X, Y, and Z axes, respectively. The X and Y axis coils control camera shake, while the Z axis coil controls focusing. Z-axis control can be achieved through a focusing algorithm and is not necessarily related to the X and Y axes. Translation-type OIS is less expensive and easier to manufacture. It should be noted that... Figure 1 The text here is only used to describe the X, Y, and Z axes of the motor. If... Figure 1 The lines are not clear, but this does not affect the clarity of the instruction manual.

[0077] Open-loop motors do not contain Hall effect sensors. The lack of Hall effect sensors makes it impossible to accurately determine whether the motor is truly stable at a specific position. When an open-loop motor is subjected to significant external force, its coils will elastically expand and contract due to their inherent physical characteristics. This elastic expansion and contraction is directly reflected in the image, easily causing a shaky appearance. If the device detects this shaking, it typically adjusts the lens position using AutoFocus (AF) to achieve focus. Simultaneously, to reduce image blur caused by shaking, offset adjustments are made along the X and Y axes.

[0078] However, when the device moves significantly, these two processes can interfere with each other. Specifically, focus adjustments made on the Z-axis may conflict with image stabilization offsets on the X and Y axes. This superimposed effect can result in more noticeable distortion in the image, making the final image blurry even with autofocus enabled, and significantly reducing image stabilization performance.

[0079] In view of the deficiencies of the above-mentioned related technologies, this disclosure provides a method, apparatus, electronic device, storage medium and program product for device control, which aims to improve the quality of acquired images.

[0080] This disclosure provides a device control method applicable to any electronic device used for image acquisition. The electronic device includes an image acquisition unit. The image acquisition unit includes a lens module. The lens module includes a lens and a moving device for moving the lens. The moving device may include optical image stabilization (OIS). This disclosure does not limit the type of electronic device; it can be any suitable device type, such as a mobile phone, a handheld gimbal camera, or an action camera, etc., which will not be elaborated further in this disclosure.

[0081] See Figure 2 The diagram shown is a flowchart of a device control method according to an embodiment of this disclosure. The following is a description of the method in conjunction with... Figure 2 The method is described below, and the specific implementation process is as follows:

[0082] Step 201: Based on the collected motion data of the electronic device, determine the first motion data of the lens of the electronic device. The first motion data is used to represent the direction and distance that the lens is to move on the specified plane.

[0083] The motion data includes angular velocity; the first motion data includes a first current for controlling the lens's movement in a first direction and a second current for controlling the lens's movement in a second direction. The first current drives the lens to move a first offset distance in the first direction; the second current drives the lens to move a second offset distance in the second direction. Lens movement includes both forward and reverse movement. The magnitude of the current is positively correlated with the distance the lens moves.

[0084] In one implementation, when performing step 201, the following steps may be adopted:

[0085] The motion sensor collects the angular velocity of the electronic device; based on the angular velocity, the offset vector of the electronic device on a specified plane is calculated; a first current is obtained according to the component of the offset vector in the first direction; a second current is obtained according to the component of the offset vector in the second direction; the first current and the second current are used to control the lens to move the offset vector in the opposite direction.

[0086] For example, an OIS algorithm can be used to obtain the first and second currents based on angular velocity. Optionally, the motion sensor can be a gyroscope. Furthermore, the correspondence between distance and direction and the magnitude and direction of the current can be preset. The magnitude of the component of the offset vector in the first direction is the first offset distance, and the direction of the component is opposite to the direction of movement driven by the first current.

[0087] For example, if the offset vector (x, y) = (-1, 10), then the component on the X-axis (i.e., the first direction) is -1, and the component on the Y-axis (i.e., the second direction) is 10. This gives us the first current corresponding to -1, such as 1 mA, and the second current corresponding to 10, such as -10 mA. The first current is used to control the lens movement by 1 mm on the X-axis, and the second current is used to control the lens movement by -10 mm on the Y-axis. Therefore, the vector used by the first and second currents to control the lens movement is (1, -10).

[0088] Step 202: Based on the acquired original image, calculate the second movement data of the lens. The second movement data is used to represent the distance the lens needs to move in a specified direction in order to focus the lens.

[0089] The second movement data includes an initial focusing current for controlling the lens movement in a specified direction; this initial focusing current drives the lens to move a first focusing distance in the specified direction. Lens movement includes both forward and reverse movement. The specified direction can be perpendicular to a specified plane.

[0090] In this embodiment, a three-dimensional coordinate system can be established. (See also...) Figure 3 The image shown is an example of a three-dimensional coordinate system. Figure 3In this system, the three-dimensional coordinate system includes the X-axis, Y-axis, and Z-axis. Figure 3 In this system, each coordinate axis is equipped with a set of coils for the motor. The coils for the X and Y axes are used to control shake, and the coil for the Z axis is used to control focus. The plane formed by the X and Y axes is the designated plane, and the direction indicated by the Z axis is the designated direction.

[0091] In one implementation, step 202 may be performed using the following steps:

[0092] The original image is captured through the lens; the quality of the original image is assessed to obtain the image quality assessment result; based on the image quality assessment result, the initial focusing current is calculated.

[0093] Optionally, an AF algorithm can be used to obtain the initial focus current based on the original image. The image quality assessment result can be the image sharpness.

[0094] Furthermore, the correspondence between focusing distance and current can be preset. Then, based on the image quality assessment results, the first focusing distance to be moved in a specified direction during focusing adjustment can be determined, and the initial focusing current corresponding to the first focusing distance can be obtained according to the correspondence between focusing distance and current.

[0095] It should be noted that the first mobile data acquisition algorithm (e.g., OIS algorithm) and the second mobile data acquisition algorithm (e.g., AF algorithm) are independent of each other and do not affect each other.

[0096] Step 203: Using a focus compensation model, based on the first movement data and the second movement data, obtain the predicted compensation value.

[0097] In one implementation, the first motion data and the second motion data are input into a focus compensation model to obtain a predicted compensation value. The predicted compensation value can be a compensation current.

[0098] Step 204: If a focus compensation event is detected, the second motion data is adjusted based on the predicted compensation value to obtain the third motion data.

[0099] The third movement data includes a target focusing current used to control the lens to move in a specified direction; the target focusing current is used to drive the lens to move a second focusing distance in the specified direction.

[0100] In one implementation, when a focus compensation event is detected, at least one of the following methods may be used:

[0101] Method 1: Obtain the lens adjustment status; the lens adjustment status indicates whether focus adjustment is required; if the lens adjustment status indicates focus adjustment, then a focus compensation event is detected.

[0102] Method 2: Obtain the lens adjustment status; if the lens adjustment status indicates focus adjustment, and both the first current and the second current are lower than the first current threshold and higher than the second current threshold, then a focus compensation event is detected.

[0103] Optionally, the first current threshold can be the maximum current value of the OIS, and the second current threshold can be the minimum current value of the OIS, such as zero.

[0104] In practical applications, the first current threshold and the second current threshold can be set according to the actual application scenario, and there are no restrictions here.

[0105] Step 205: Control the camera movement based on the first movement data and the third movement data respectively.

[0106] In one embodiment, the motor includes three sets of coils. A first current drives the first set of coils to move the lens a first offset distance on the X-axis. A second current drives the second set of coils to move the lens a second offset distance on the Y-axis. A target focusing current drives the third set of coils to move the lens a second focusing distance on the Z-axis.

[0107] The following is combined Figure 4 The methods for controlling the above-mentioned equipment are described in detail below. Figure 4 The diagram shown is a detailed implementation flowchart of a device control method. The specific steps of this method include:

[0108] Step 401: Collect angular velocity using a gyroscope.

[0109] Step 402: Calculate the offset vector of the electronic device on the specified plane based on the angular velocity.

[0110] Step 403: Determine the first current and the second current based on the components of the offset vector in the first and second directions respectively, and then proceed to step 405.

[0111] Step 404: Using an autofocus algorithm, obtain the initial focus current based on the acquired original image.

[0112] Step 405: Input the first current, the second current, and the initial focusing current into the focusing compensation model to obtain the predicted compensation value.

[0113] Step 406: Calculate the sum of the initial focusing current and the predicted compensation value to obtain the target focusing current.

[0114] Step 407: Determine whether the current lens adjustment status indicates focus adjustment. If yes, proceed to step 408; otherwise, proceed to step 401.

[0115] In this way, if focusing is not required at the moment, focus compensation and image stabilization can be omitted.

[0116] Step 408: Determine whether both the first current and the second current are lower than the first current threshold and higher than the second current threshold. If yes, proceed to step 409; otherwise, proceed to step 410.

[0117] Step 409: Control the lens movement based on the first current, the second current, and the target focusing current.

[0118] In one embodiment, a first current drives a first set of coils of a motor to control the lens to move a first offset distance in a first direction; a second current drives a second set of coils of a motor to control the lens to move a second offset distance in a second direction; and a target focusing current drives a third set of coils of a motor to control the lens to move a second focusing distance in a specified direction.

[0119] Step 410: Based on the initial focusing current, control the lens movement.

[0120] In this way, when the values ​​of the first current and the second current are abnormal, no image stabilization adjustment is performed. Instead, the lens movement is controlled directly based on the initial focusing current, and no focus compensation is performed.

[0121] In this way, steps 401-410 can be repeated to continuously adjust the lens.

[0122] Furthermore, the focus compensation model can be trained using the following steps:

[0123] Multiple current combinations and their corresponding reference compensation values ​​are obtained. The current combinations include a first current sample, a second current sample, and an initial focusing current sample. Each current combination is input into the initial compensation model to obtain a predicted compensation value. Based on the predicted compensation value and reference compensation value corresponding to each current combination, the prediction loss is determined. Based on the prediction loss, the parameters of the initial compensation model are adjusted until a focusing compensation model that meets the preset training conditions is obtained.

[0124] In one model training scenario, the electronic device is a mobile phone, which is fixed in a laboratory lightbox. The lightbox can use a narrow-band fluorescent light source, such as the TL84, to provide 1000 lux of light. The maximum and minimum OIS currents can be determined based on the product model of the OIS in the phone; for example, the maximum and minimum OIS currents (e.g., 0) can be obtained from the phone's hardware specifications. Then, the phone's focus mode is set to a fixed-step motion mode from telephoto to near focus, and different OIS motion currents are set to keep the phone in continuous focusing motion and shaking.

[0125] Different fixed step sizes can be set for different rounds. Different fixed step sizes correspond to different initial focusing currents. The OIS motion current includes a first current and a second current, both of which are greater than the minimum current mentioned above but lower than the maximum current mentioned above. In this way, multiple different current combinations can be obtained, and each current combination includes a first current sample, a second current sample, and an initial focusing current sample.

[0126] By continuously capturing images of the power card through the phone's camera and recording each frame of the preview flow, the corresponding images for each current combination can be obtained.

[0127] Taking a set of current combinations as an example, the lens is moved according to the first current sample, the second current sample, and the initial focusing current sample in the set of current combinations. Then, the image is acquired through the lens, the image feature data is extracted, and the image sharpness is calculated based on the image feature data. The predicted compensation value is also calculated based on the sharpness.

[0128] In this way, if the image is blurry or significantly distorted, the determined sharpness will be relatively low. Based on this sharpness, the compensation current (i.e., the predicted compensation value) that needs to be compensated for the initial focusing current sample can be calculated so that the sharpness of the image obtained after compensation is higher.

[0129] In this embodiment, if image stabilization adjustment is required simultaneously with focus adjustment, a focus compensation model can be used. Based on the first and second currents used for image stabilization adjustment, and the initial focus current used for focus adjustment, a predicted compensation value is obtained. The initial focus current is then optimized based on the predicted compensation value to obtain the target focus current. This combination of image stabilization and focus adjustment algorithms for focus adjustment optimization can suppress image distortion or blurring caused by irregular device shaking during focusing, reducing the impact of conflicts between focus and image stabilization adjustments on image quality, improving image quality and user experience, without increasing additional hardware costs.

[0130] Based on the same inventive concept, this disclosure also provides a device control apparatus. Since the principle by which the above-described apparatus and device solve the problem is similar to that of a device control method, the implementation of the above-described apparatus can refer to the implementation of the method, and repeated details will not be elaborated further. This apparatus can be applied to electronic devices. This disclosure does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this disclosure. The apparatus embodiment can be implemented through software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logically defined device, it is formed by the processor of the electronic device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0131] See Figure 5 The diagram shown is a structural block diagram of a device control apparatus according to an embodiment of this disclosure. In some embodiments, the device control apparatus exemplified by this disclosure includes:

[0132] The determining unit 501 is used to determine the first movement data of the lens of the electronic device based on the collected motion data of the electronic device. The first movement data is used to indicate the direction and distance that the lens is to move on a specified plane.

[0133] The calculation unit 502 is used to calculate second movement data of the lens based on the acquired original image. The second movement data is used to represent the distance that the lens needs to move in a specified direction in order to focus the lens.

[0134] The obtaining unit 503 is used to obtain a predicted compensation value based on the first movement data and the second movement data using a focus compensation model;

[0135] The adjustment unit 504 is used to adjust the second movement data based on the predicted compensation value to obtain the third movement data if a focus compensation event is detected.

[0136] The control unit 505 is used to control the movement of the lens based on the first movement data and the third movement data respectively.

[0137] In one embodiment, the first movement data includes a first current for controlling the movement of the lens in a first direction and a second current for controlling the movement of the lens in a second direction; the second movement data includes an initial focusing current for controlling the movement of the lens in a specified direction; and the third movement data includes a target focusing current for controlling the movement of the lens in the specified direction.

[0138] The distance the lens moves is positively correlated with the magnitude of the current; the specified direction is perpendicular to the specified plane.

[0139] In one embodiment, the determining unit 501 is used to:

[0140] The motion sensor collects the angular velocity of the electronic device; the motion data includes angular velocity.

[0141] Based on angular velocity, calculate the offset vector of the electronic device on the specified plane;

[0142] The first current is obtained based on the component of the offset vector in the first direction;

[0143] The second current is obtained based on the component of the offset vector in the second direction; the first and second currents are used to control the lens to move the offset vector in the opposite direction.

[0144] In one embodiment, the computing unit 502 is used for:

[0145] Capture raw images through the lens;

[0146] The original image is subjected to quality assessment to obtain the image quality assessment results;

[0147] The initial focusing current is calculated based on the image quality assessment results.

[0148] In one embodiment, the adjustment unit 504 is configured to: employ at least one of the following:

[0149] If the current lens adjustment state indicates focus adjustment, then a focus compensation event is detected.

[0150] If the current lens adjustment state indicates focus adjustment, and both the first current and the second current are lower than the first current threshold and higher than the second current threshold, then a focus compensation event is detected.

[0151] In one embodiment, the adjustment unit 504 is further configured to:

[0152] The focus compensation model is trained using the following steps:

[0153] Multiple current combinations and their corresponding reference compensation values ​​are obtained. The current combinations include a first current sample, a second current sample, and an initial focusing current sample.

[0154] Input each current combination into the initial compensation model to obtain the predicted compensation value corresponding to each current combination.

[0155] The predicted loss is determined based on the predicted compensation value and reference compensation value corresponding to each current combination.

[0156] Based on the predicted loss, the parameters of the initial compensation model are adjusted until a focus compensation model that meets the preset training conditions is obtained.

[0157] The device control method in this embodiment includes determining first motion data of the electronic device's lens based on acquired motion data of the electronic device, the first motion data representing the direction and distance the lens is to move on a specified plane; calculating second motion data of the lens based on the acquired original image, the second motion data representing the distance the lens is to move in the specified direction for focusing the lens; using a focus compensation model, obtaining a predicted compensation value based on the first and second motion data; if a focus compensation event is detected, adjusting the second motion data based on the predicted compensation value to obtain third motion data; and controlling lens movement based on the first and third motion data respectively. In this way, by combining the first motion data used for image stabilization and the second motion data used for focus adjustment, the second motion data is optimized, thereby optimizing the lens's focus adjustment, reducing the impact of the conflict between image stabilization adjustment and focus adjustment on the image, and thus improving the quality of the acquired image.

[0158] In this embodiment of the disclosure, an electronic device is also provided, including:

[0159] Processor; and

[0160] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.

[0161] In this embodiment of the disclosure, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.

[0162] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described in any of the above embodiments.

[0163] Figure 6 A schematic diagram of the structure of an electronic device 6000 is shown. (See also...) Figure 6 As shown, the electronic device 6000 includes a processor 6010 and a memory 6020, and optionally may also include a power supply 6030, a display unit 6040, and an input unit 6050.

[0164] The processor 6010 is the control center of the electronic device 6000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 6000 by running or executing software programs and / or data stored in the memory 6020, thereby performing overall monitoring of the electronic device 6000.

[0165] In this embodiment of the present disclosure, the processor 6010 executes the steps in the above embodiments when it calls the computer program stored in the memory 6020.

[0166] Optionally, the processor 6010 may include one or more processing units; preferably, the processor 6010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 6010. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.

[0167] The memory 6020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 6000, etc. In addition, the memory 6020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.

[0168] Electronic device 6000 also includes a power supply 6030 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 6010 through a power management system, thereby enabling the management of charging, discharging, and power consumption.

[0169] The display unit 6040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 6000. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 6000, as well as text, images, and other objects displayed on the display interfaces. The display unit 6040 may include a display panel 6041. The display panel 6041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0170] The input unit 6050 can be used to receive information such as numbers or characters input by the user. The input unit 6050 may include a touch panel 6051 and other input devices 6052. The touch panel 6051, also known as a touch screen, can collect touch operations on or near the touch panel 6051 by the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 6051).

[0171] Specifically, the touch panel 6051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 6010, and receive and execute commands from the processor 6010. Furthermore, the touch panel 6051 can be implemented using various types of touch technologies, including resistive, capacitive, infrared, and surface acoustic wave. Other input devices 6052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0172] Of course, the touch panel 6051 can cover the display panel 6041. When the touch panel 6051 detects a touch operation on or near it, it transmits the information to the processor 6010 to determine the type of touch event. Subsequently, the processor 6010 provides corresponding visual output on the display panel 6041 based on the type of touch event. Although in Figure 6 In this embodiment, the touch panel 6051 and the display panel 6041 are two separate components to realize the input and output functions of the electronic device 6000. However, in some embodiments, the touch panel 6051 and the display panel 6041 can be integrated to realize the input and output functions of the electronic device 6000.

[0173] The electronic device 6000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 6000 may also include other components such as a camera. Since these components are not the focus of this disclosure, they will not be discussed further. Figure 6 It is not shown in the text and will not be described in detail here.

[0174] Those skilled in the art will understand that Figure 6 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0175] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this disclosure, the functions of each module (or unit) can be implemented in one or more software or hardware components.

Claims

1. A method for controlling equipment, characterized in that, The method includes: Based on the collected motion data of the electronic device, the first motion data of the lens of the electronic device is determined. The first motion data is used to indicate the direction and distance that the lens is to move on a specified plane. Based on the acquired original image, second movement data of the lens is calculated. The second movement data is used to represent the distance that the lens needs to move in a specified direction in order to focus the lens. A focus compensation model is used to obtain the predicted compensation value based on the first movement data and the second movement data; If a focus compensation event is detected, the second motion data is adjusted based on the predicted compensation value to obtain the third motion data; The lens movement is controlled based on the first movement data and the third movement data, respectively.

2. The method according to claim 1, characterized in that, The first movement data includes a first current for controlling the movement of the lens in a first direction and a second current for controlling the movement of the lens in a second direction; the second movement data includes an initial focusing current for controlling the movement of the lens in the specified direction. The third movement data includes a target focusing current for controlling the movement of the lens in the specified direction; The movement distance of the lens is positively correlated with the magnitude of the current; the specified direction is perpendicular to the specified plane.

3. The method according to claim 2, characterized in that, The determination of the first movement data of the lens of the electronic device based on the collected motion data of the electronic device includes: The angular velocity of the electronic device is acquired using a motion sensor; the motion data includes angular velocity. Based on the angular velocity, the offset vector of the electronic device on the specified plane is calculated; The first current is obtained based on the component of the offset vector in the first direction; The second current is obtained based on the component of the offset vector in the second direction; the first current and the second current are used to control the lens to move the offset vector in the opposite direction.

4. The method according to claim 2, characterized in that, The calculation of the second movement data of the lens based on the acquired original image includes: The original image is captured through the lens; The original image is subjected to quality assessment to obtain image quality assessment results; Based on the image quality assessment results, the initial focusing current is calculated.

5. The method according to any one of claims 2-4, characterized in that, The detection of the focus compensation event includes: Use at least one of the following: If it is determined that the current lens adjustment state indicates focus adjustment, then the focus compensation event is detected. If it is determined that the current lens adjustment state indicates focus adjustment, and both the first current and the second current are lower than the first current threshold and higher than the second current threshold, then it is determined that the focus compensation event has been detected.

6. The method according to any one of claims 2-4, characterized in that, The method further includes: The focus compensation model is trained using the following steps: Multiple current combinations and their corresponding reference compensation values ​​are obtained, wherein the current combinations include a first current sample, a second current sample, and an initial focusing current sample. Input each current combination into the initial compensation model to obtain the predicted compensation value corresponding to each current combination. The predicted loss is determined based on the predicted compensation value and reference compensation value corresponding to each current combination. Based on the predicted loss, the parameters of the initial compensation model are adjusted until a focus compensation model that meets the preset training conditions is obtained.

7. A device for controlling equipment, characterized in that, The device includes: The determining unit is used to determine first motion data of the lens of the electronic device based on the collected motion data of the electronic device. The first motion data is used to indicate the direction and distance that the lens is to move on a specified plane. A calculation unit is used to calculate second movement data of the lens based on the acquired original image. The second movement data is used to represent the distance that the lens needs to move in a specified direction in order to focus the lens. The obtaining unit is used to obtain a predicted compensation value based on the first movement data and the second movement data using a focus compensation model; An adjustment unit is used to adjust the second motion data based on the predicted compensation value to obtain the third motion data if a focus compensation event is detected. The control unit is used to control the movement of the lens based on the first movement data and the third movement data, respectively.

8. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device is the method according to any one of claims 1 to 6.

Images