Air recognition method, electronic device and storage medium

Abstract

Embodiments of the present application relate to the field of terminal technology, and in particular to a method for air recognition, an electronic device, and a storage medium. The method can be applied to an electronic device including a camera, the electronic device turns on a real-time online mode, and the camera includes an autofocus AF module. The above method includes: the electronic device captures an image through a camera. Afterwards, when the user's air operation is recognized based on the captured image, the electronic device performs a preset response operation corresponding to the air operation. In the above method, the AF module can be powered on when the electronic device is in a first posture, and not powered on when it is in a second posture. In other words, the AF module of the electronic device in the second posture does not work, and the AF module does not consume the power of the electronic device, or consumes a very small amount of power. Thereby, the power consumption of the AF module during the air recognition process can be reduced, the battery life of the electronic device can be improved, and the user experience can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to an air recognition method, an electronic device, and a storage medium. Background Art

[0002] Nowadays, mobile phones and other electronic devices can use the camera's always on (AO) function to realize functions such as air gesture control of the screen and air gesture screenshots.

[0003] At present, how to capture images through the camera of electronic devices and perform remote recognition is a problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide an air recognition method, an electronic device, and a storage medium, which can reduce the power consumption of the AF module during the air recognition process, thereby increasing the battery life of the electronic device and improving the user experience.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In the first aspect, a method for air recognition is provided, which can be applied to an electronic device including a camera, the electronic device turns on the real-time online AO ​​mode, and the camera includes an auto focus AF module. Among them, the above-mentioned electronic device can be an electronic device with a camera such as a mobile phone, a tablet computer, a notebook, etc. The above-mentioned method includes: the electronic device collects images through a camera. Afterwards, when the user's air operation is recognized based on the collected image, the electronic device performs a preset response operation corresponding to the air operation. The above-mentioned air operation includes but is not limited to air grasping gestures, air up / down gestures, air pressing gestures, or the user looking at the mobile phone, etc. The preset response operation includes but is not limited to screenshots, page turning, answering calls, lighting the screen, etc. When the camera collects images, if the electronic device is in a first posture, the AF module is in a power-on state; if the electronic device is in a second posture, the AF module is in a power-off state. The posture of the electronic device is used to characterize the relative position relationship between the electronic device and the ground plane.

[0007] In the above method, the AF module can be powered on when the electronic device is in the first posture, and not powered on when it is in the second posture. That is to say, the AF module of the electronic device in the second posture does not work, and the AF module does not consume the power of the electronic device, or consumes a very small amount of power. This can reduce the power consumption of the AF module during the air recognition process, increase the battery life of the electronic device, and enhance the user experience.

[0008] In a possible design of the first aspect, the first posture includes that the shooting direction of the camera is vertical, or the shooting direction of the camera is between vertical and horizontal; the second posture includes that the shooting direction of the camera of the electronic device is horizontal. The shooting direction of the camera can be understood as the direction in which the image sensor of the camera points to the lens.

[0009] In this design, in the second posture and when the AF module is powered off, the depth of field of the camera will be the initial depth of field, and a clear image can be captured. Since the AF module is powered off, that is to say, the power consumption of the AF module is very small, the AF module will not consume the power of the electronic device, or consume a very small amount of power. In this way, the power consumption of the AF module during the air recognition process can be further reduced.

[0010] In another possible design of the first aspect, the AF module includes a drive motor and an integrated circuit IC chip, and the IC chip is used to provide a working current to the drive motor when the AF module is in a powered-on state. When the AF module is in a powered-on state and the shooting direction of the camera is vertical, the IC chip provides a first working current to the drive motor. When the AF module is in a powered-on state and the shooting direction of the camera is between vertical and horizontal, the IC chip provides a second working current to the drive motor; the first working current is greater than or equal to the second working current.

[0011] In another possible design of the first aspect, the electronic device captures images through a camera, including: the electronic device determines the current posture of the electronic device; the current posture includes a first posture or a second posture. Afterwards, the electronic device determines the current depth of foreground of the camera of the electronic device in the current posture based on the current posture of the electronic device. Then, according to the current depth of foreground of the camera of the electronic device and the AO acquisition condition corresponding to the current posture, the current working state of the AF module is controlled, and the image is captured through the camera in the current working state of the AF module. Among them, the current working state of the AF module includes a power-on state or a power-off state. The AO acquisition condition corresponding to the current posture includes the condition that the camera captures an image that meets the quality requirements in the current posture. For example, in the current posture, the camera captures an image whose clarity meets the AO mode requirements of the electronic device.

[0012] In this design, the electronic device determines whether the current depth of field of the camera can capture an image that meets the quality requirements based on the AO acquisition conditions, and controls the current working state of the AF module. In this way, the quality of the AO image captured by the camera under the current depth of field can be predicted based on the current depth of field.

[0013] In another possible design of the first aspect, the above-mentioned controlling the current working state of the AF module according to the current depth of field and the AO acquisition condition of the camera of the electronic device may include: if the current depth of field meets the AO acquisition condition corresponding to the current posture, the electronic device controls the current working state of the AF module to be a power-off state. If the current depth of field does not meet the AO acquisition condition corresponding to the current posture, the electronic device controls the current working state of the AF module to be a power-on state.

[0014] In another possible design of the first aspect, the camera further includes a lens, and the determining of the current depth of field of the camera of the electronic device in the current posture based on the current posture of the electronic device includes: the electronic device obtains the current shooting direction of the camera based on the current posture. Then, the electronic device calculates the component of the gravity of the lens in the current shooting direction based on the mass of the lens. Then, the electronic device obtains the current depth of field of the camera based on the component of the gravity of the lens in the current shooting direction.

[0015] In this design, the electronic device can accurately calculate the current depth of field of the camera based on gravity and the current posture of the lens.

[0016] In another possible design of the first aspect, the current working state of the AF module is a power-on state, and images are captured through a camera in the current working state of the AF module, including: the electronic device adjusts the depth of field of the camera through the AF module, and captures images through the adjusted camera; the depth of field of the adjusted camera satisfies the AO capture conditions corresponding to the current posture.

[0017] In another possible design of the first aspect, the adjusting the depth of field of the camera through the AF module includes: obtaining a compensated image distance according to a difference between the current depth of field and the initial depth of field, or obtaining a compensated image distance according to a difference between the current depth of field and the initial image distance. Afterwards, the electronic device adjusts the image distance of the camera to the compensated image distance through the AF module to adjust the depth of field of the camera.

[0018] In this design, the electronic device adjusts the image distance of the camera to the compensated image distance through the AF module. Since the image distance and the depth of field are related, the electronic device also adjusts the depth of field through the AF module so that the adjusted depth of field meets the requirements of the AO acquisition conditions. Then, the electronic device can acquire images that meet the quality requirements through the camera.

[0019] In another possible design of the first aspect, the AO acquisition conditions corresponding to the above-mentioned current posture specifically include: the current posture includes a first posture, the first posture includes a vertical shooting direction of the camera, and the AO acquisition conditions corresponding to the first posture specifically include: the first preset distance is within the current depth of view, or the first preset range is within the current depth of view. The current posture includes the first posture, the first posture includes a shooting direction of the camera between vertical and horizontal, and the AO acquisition conditions corresponding to the first posture specifically include: the second preset distance is within the current depth of view, or the second preset range is within the current depth of view. The current posture includes a second posture, the second posture includes a horizontal shooting direction of the camera, and the AO acquisition conditions corresponding to the second posture specifically include: the initial depth of field of the camera is within the current depth of view, or the initial image distance of the camera is within the current depth of view.

[0020] Among them, the initial depth of field of the above-mentioned camera within the current depth of view can be understood as the minimum value in the initial depth of field is greater than or equal to the minimum value in the current depth of view, and the maximum value in the initial depth of field is less than or equal to the maximum value in the current depth of view. The initial image distance of the above-mentioned camera within the current depth of view can be understood as the initial image distance is greater than or equal to the minimum value in the current depth of view, and the initial image distance is less than or equal to the maximum value in the current depth of view.

[0021] In this design, the electronic device can accurately obtain the image distance at which the camera can clearly capture the image, that is, the compensated image distance, through the difference between the current depth of field and the initial depth of field, or through the difference between the current depth of field and the initial image distance. In this way, the electronic device can simply and quickly adjust the image distance of the camera to adjust the depth of field of the camera so that the adjusted depth of field of the camera meets the AO acquisition conditions. Then, the adjusted camera can capture images that meet the requirements of the AO mode.

[0022] In another possible design of the first aspect, the electronic device determines the current posture of the electronic device, including: determining the posture of the electronic device through an acceleration sensor of the electronic device and / or a gyroscope of the electronic device.

[0023] In this design, the current posture of the electronic device can be accurately determined by the acceleration sensor and / or the gyroscope electronic device.

[0024] In a second aspect, the present application provides an electronic device, comprising: a memory, one or more processors, and a Bluetooth module; the memory is coupled to the processor; wherein the memory stores computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method provided by the above-mentioned first aspect and any possible design of the first aspect.

[0025] In a third aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method provided by the first aspect and any possible design of the first aspect.

[0026] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method provided by the first aspect and any possible design of the first aspect.

[0027] Among them, the technical effects brought about by any design method in the second to fourth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of a scenario in which an AO function is enabled in an electronic device provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the principle of a camera focusing process provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of a use scenario of an air recognition method provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the structure of a camera of an electronic device provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of a flow chart of a method for remote identification provided in an embodiment of the present application;

[0035] Figure 8 A schematic diagram of a user graphical interface of an electronic device provided in an embodiment of the present application;

[0036] Fig. 9A A schematic diagram of a Cartesian coordinate system established with the center of an electronic device as the origin provided in an embodiment of the present application;

[0037] Fig. 9B A schematic diagram of different postures of an electronic device provided in an embodiment of the present application;

[0038] Fig.10A schematic diagram of the depth of field of a camera provided in an embodiment of the present application;

[0039] Fig.11 A schematic diagram of a flow chart of another method for remote identification provided in an embodiment of the present application;

[0040] Fig.12 A schematic diagram of the principle of a remote recognition method provided in an embodiment of the present application;

[0041] Fig.13 A schematic diagram of different postures of another electronic device provided in an embodiment of the present application;

[0042] Fig.14A A schematic diagram of a scenario in which a user uses the AO function of an electronic device provided in an embodiment of the present application;

[0043] Fig. 14B A schematic diagram of another scenario in which a user uses the AO function of an electronic device provided in an embodiment of the present application;

[0044] Fig. 14C A schematic diagram of another scenario in which a user uses the AO function of an electronic device provided in an embodiment of the present application;

[0045] Fig.14D A schematic diagram of another scenario in which a user uses the AO function of an electronic device provided in an embodiment of the present application;

[0046] Fig.15 A schematic diagram of the hardware structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the embodiments of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0048] Meanwhile, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0049] At present, mobile phones and other electronic devices can use the camera real-time online (always on, AO) function. Electronic devices that use the AO function will have their cameras (such as the front camera and / or the rear camera) in a normally open state, allowing functions such as air gestures to manipulate the screen, air gestures to take screenshots, and smart screen-off display (always on display, AOD) to be completed more quickly. Understandably, the AO function can also be called AO mode, "smart perception", "air recognition" function, etc.

[0050] For example, see Figure 1In part A of the figure, the electronic device 100 is in the screen-on state, the AO function of the electronic device 100 is turned on, and the front camera of the electronic device 100 can be in the normally-on state and collect images in real time; in this way, the electronic device 100 can realize functions such as air gesture control of the screen, air gesture screenshot, staring to keep the screen on, staring to reduce the volume, etc. through the collected images. Figure 1 In part B, the electronic device 100 is in a screen-off state, the AO function of the electronic device 100 is turned on, the front camera of the electronic device 100 is always on, and collects images in real time; in this way, the electronic device 100 can realize functions such as intelligent AOD through the collected images.

[0051] When the AO function is turned on in an electronic device, the camera of the electronic device is in a normally-on state, and the camera collects images in real time. In order to ensure that the camera collects clear images, the auto focus (AF) module will be in a working state (power-on state) to adjust the positional relationship between the camera lens and the image sensor (such as the distance between the two) to achieve the focus of the camera. If the AF module is always in a working state (power-on state), the AF module will continue to consume the power of the electronic device, resulting in a relatively high power consumption (or power consumption) of the electronic device, affecting the battery life of the electronic device, and further affecting the user experience.

[0052] In some embodiments, for an electronic device with an AO function enabled, the AF module of the camera can adjust the positional relationship between the lens and the image sensor through a phase detection focusing method or a contrast detection focusing method to achieve camera focus.

[0053] In these solutions, the AF module will continuously adjust the positional relationship between the camera lens and the image sensor, that is, the image distance of the camera, based on the relevant information of the image collected by the camera (such as phase, contrast). Since the image distance of the camera will affect the focus of the camera, the image distance of the camera can be adjusted to achieve the focus of the camera.

[0054] For example, see Figure 2, the dotted line in the figure represents the propagation path of light during the focusing process of the camera. During the focusing process, the AF module will control the movement of the lens to change the distance between the lens and the image sensor, such as: moving to the farthest from the image sensor, or moving to the closest to the image sensor, to change the convergence point of light passing through the lens on the image sensor. During the movement of the lens, the image sensor will capture images in real time and record the contrast of the image to obtain a contrast curve 201. Afterwards, the AF module adjusts the distance between the lens and the image sensor to the distance corresponding to the peak of the contrast in the contrast curve 201. In this way, the AF module achieves the focus of the camera through the contrast detection focusing method.

[0055] from Figure 2 It can be seen from the corresponding description that in the process of the AF module realizing camera focusing through the contrast detection focusing method, since the AF module needs to continuously consume electrical energy to drive the lens displacement and change the distance between the lens and the image sensor, this will cause the AF module's focusing process to have relatively high power consumption, affecting the battery life of the electronic device and the user experience.

[0056] In view of this, an embodiment of the present application provides an air recognition method, in which: in the process of the camera of the electronic device capturing images, the working state of the AF module is different when the electronic device is in different postures and / or the shooting direction of the camera, that is, the working state of the AF module is adjusted according to the posture of the electronic device and / or the shooting direction of the camera. For example: when the electronic device captures the first AO image through the camera of the electronic device, the electronic device is in a first posture, the shooting direction of the camera is vertical, and the AF module is in a power-on state. Next, when the electronic device captures the second AO image through the camera, the electronic device is in a second posture, the shooting direction of the camera is horizontal, and the AF module is in a power-off state. Then, the electronic device performs the AO operation corresponding to the captured image based on the captured AO image (such as the first AO image and / or the second AO image), such as lighting the screen, taking a screenshot, and so on.

[0057] In this air recognition method, since the AF module of the electronic device in the second posture is in a power-off state, that is, the AF module will not work and will not consume the power of the electronic device. This can reduce the power consumption of the AF module during the air recognition process, thereby increasing the battery life of the electronic device and improving the user experience.

[0058] The first posture includes that the shooting direction of the camera is vertical, or the shooting direction of the camera is between vertical and horizontal; the second posture includes that the shooting direction of the camera is horizontal. The shooting direction of the camera can be understood as the direction in which the image sensor of the camera points to the lens.

[0059] It is understandable that in the second posture, the shooting direction of the electronic device is horizontal, and the gravity of the camera lens will not affect the image distance of the camera, and will not affect the depth of field of the camera. Then, in the second posture, if the AF module of the electronic device is powered off, the depth of field of the camera will be the initial depth of field, and the camera can collect images that meet the requirements of air recognition.

[0060] The air recognition method provided in the embodiment of the present application can be applied to the process in which a user uses an electronic device with an AO function enabled. Figure 3 When a user uses an electronic device with the AO function turned on, the electronic device can capture images through a camera and recognize the captured images so that the user can use the AO function of the electronic device 100. The electronic device 100 can be a mobile phone, a tablet computer, a wearable device, a smart screen, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and other electronic devices with cameras; it can also be a vehicle-mounted computer, a vehicle-mounted computer, and other vehicle-mounted devices with cameras; it can also be some smart watches, smart bracelets, and other IoT devices with cameras. The embodiments of the present application do not impose any restrictions on the product form of the electronic device.

[0061] Next, the hardware structure and software architecture of the electronic device 100 are briefly introduced.

[0062] For example, Figure 4 A schematic diagram of the hardware structure of the electronic device 100 is shown.

[0063] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0064] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0065] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0067] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0068] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. For example, the electronic device 100 may include a front camera and / or a rear camera.

[0069] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0070] For example, the electronic device can obtain the heading angle, roll angle and pitch angle of the electronic device through the gyro sensor 180B to determine the posture of the electronic device. Fig. 9B Corresponding related introduction.

[0071] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0072] For example, the electronic device can detect the direction of gravity of the electronic device through the acceleration sensor 180E to determine the posture of the electronic device. Fig. 9B Corresponding related introduction.

[0073] In some embodiments, the structure of the camera 193 of the electronic device 100 can be seen in Figure 5As shown. The camera 193 may include: a lens 193A, an image sensor 193B, a drive motor 193C, an integrated circuit (IC) chip 193D, etc. Among them, the lens 193A can also be called an optical lens, which can include a fixed focus lens, a zoom lens, a wide-angle lens, etc. The lens 193A is mainly used to collect the reflected light of the illuminated object and focus it on the image sensor 193B. The image sensor (sensor) 193B can also be called a photosensitive element and can be a CCD, a CMOS phototransistor, etc. The drive motor 193C controls the lens 193A to move within a certain range to adjust the positional relationship (such as relative distance) between the lens 193A and the image sensor 193B, so that the object distance of the camera 193 can be adjusted to achieve the focus of the camera. Exemplarily, the drive motor 193C can control the lens to move away from / close to the image sensor by applying a push / pull force to the lens. The IC chip 193D is used to control the drive motor 193C; for example, the received digital signal type control command is converted into an analog signal type control instruction (voltage, current, waveform, etc.) that can directly control the drive motor 193C. It can be understood that in some embodiments, the drive motor 193C and the IC chip 193D can be independently deployed in the AF module, or integrated into a functional module and deployed in the AF module, wherein the functional module can have the functions of the drive motor 193C and the IC chip 193D.

[0074] In some embodiments, the camera may further include an elastic device, such as a spring, which is used to provide tension or thrust to the lens. When the drive motor is not working and the camera is not affected by gravity, the distance between the lens and the image sensor is kept fixed by the elastic device and fixed to the initial image distance. Among them, the depth of field of the camera at the initial image distance is the initial depth of field, and the object distance is the initial object distance. The camera at the initial depth of field can capture images that meet the AO function requirements of the electronic device.

[0075] In the present application, when the AF module is powered on, the components in the AF module are in a working state, such as the drive motor and IC chip 193D in the AF module are in a working state. At this time, the IC chip can provide current / voltage to the drive motor so that the drive motor controls the movement of the lens to adjust the positional relationship between the lens 193A and the image sensor 193B to achieve focusing. When the AF module is powered off, the components in the AF module are in a non-working state or a power-off state, such as the drive motor and IC chip 193D in the AF module are in a non-working state or a power-off state. At this time, the IC chip and the drive motor do not work and will not control the movement of the lens to achieve focusing. Among them, the electronic device 100 can control the on and off of the power supply of the AF module to realize the power on or off of the AF module, and control the relative distance between the lens 193A and the image sensor 193B through the AF module.

[0076] Among them, the AF module is powered on, which can also be referred to as the AF module being in a powered-on state; it can be understood that the IC chip 193D in the AF module is in a working state; the AF module is powered off, which can also be referred to as the AF module being in a powered-off state, which can be understood as the IC chip 193D in the AF module being in a non-working state, such as an off state or a stand-by state. It can be understood that the current of the IC chip 193D in the non-working state can be 0 or a very small current, which is much smaller than the current of the IC chip 193D in the working state. Specifically, the current of the IC chip 193D in the non-working state can be designed according to actual usage requirements, and the embodiments of the present application do not impose any restrictions on this.

[0077] It is understandable that in actual applications, the camera 193 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. For example, the AF module may be independent of the camera 193, etc. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware. The specific settings may be made according to actual usage requirements, and the embodiments of the present application are not limited to this.

[0078] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0079] For example, see Figure 6 The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, framework layer, hardware abstraction layer and driver layer.

[0080] The application layer can include a series of application packages. Figure 6 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0081] Among them, the application package may also include a smart perception application; it is understandable that the above-mentioned smart perception application supports various services (functions). For example, the services supported by the smart perception application may include an air gesture service, and may also include a smart code recognition service, a gaze-unextinguishing screen service, a gaze-lowering volume service, a smart horizontal and vertical screen service, an auxiliary photo service, and a smart AOD service. These services may be collectively referred to as smart perception services.

[0082] The framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. In some embodiments, the framework layer may also be referred to as the application framework layer, the system middle platform framework layer, and the like.

[0083] Exemplarily, the framework layer may include an input framework middle station, a smart perception middle station, etc. Among them, the input framework middle station can also be called an input management service (input manager server, IMS), and the upper-layer application can obtain the input events sent by the lower layer through IMS. The smart perception middle station can send images collected from the lower-layer camera module to the upper-layer smart perception application, etc.

[0084] Hardware abstraction layer is an abstract structure between the hardware and the upper layer. It is used to provide a unified interface to the upper layer. It makes it unnecessary for the upper layer application to know how the lower layer hardware works, thus shielding the implementation details of the lower layer.

[0085] The hardware abstraction layer can provide a standard interface to display the device hardware functions to the higher-level application framework layer. The hardware abstraction layer contains multiple library modules, each of which is an interface implemented for a specific type of hardware component. The library modules can include audio modules, Bluetooth modules, camera modules, sensors modules, key modules, etc. The sensor modules can include, for example, accelerometer modules, magnetic field modules, orientation modules, gyroscope modules, ambient light sensors, pressure sensors, temperature sensors, and proximity sensors. When the application framework layer requires access to the device hardware, the system will load the corresponding library module for the hardware component. Manufacturers can define interfaces in the hardware abstraction layer.

[0086] The driver layer is a layer between hardware and software. Exemplarily, the driver layer may include screen driver, camera driver, sensor driver, etc.

[0087] Optionally, a hardware layer may be included below the driver layer. The hardware layer may include sensor devices (such as the above Figure 4 Sensor 180 in the embodiment) and camera (as described above Figure 4 Camera 193 in the image).

[0088] The following briefly introduces the AO function on electronic devices and the implementation process of the AO function in combination with the software and hardware of the electronic device.

[0089] After the AO function is turned on in an electronic device, the camera of the electronic device will always be in working state. The camera will continue to collect images and send the collected images to the upper layer. For example, the camera sends the collected images to the smart perception center through the camera driver and the camera module. After that, the smart perception center sends the image to the smart perception application. The smart perception application processes the image, such as identifying gestures in the image, performing face recognition on the image, and so on. And perform different functions or services based on the recognition results, such as air gesture control screen service, air gesture screenshot service, staring without turning off the screen service, staring to reduce the volume service, and so on.

[0090] It is understandable that the smart perception application can identify the user's trigger operation from the collected image, and perform different functions based on different trigger operations. In other words, the smart perception application can obtain the user's air operation based on the collected image recognition, and perform the preset response operation corresponding to the air operation based on the air operation. The above-mentioned air operations include but are not limited to air grabbing gestures, air up / down sliding gestures, air pressing gestures, or the user looking at the phone, etc. The preset response operations include but are not limited to screenshots, page turning, answering calls, lighting up the screen, etc.

[0091] For example: air grabbing gesture → screenshot, air up / down sliding gesture → page turning, air pressing gesture → answering a call, user looking at the phone → screen turning on, etc. Taking the air grabbing gesture as an example, the user changes from an extended palm state to a clenched fist state, and the control method corresponding to the gesture is preset to take a screenshot of the display interface of the electronic device. When the electronic device is in the screen-on state, the electronic device will automatically perform a screenshot operation when the air grabbing gesture is detected through the camera. Among them, the air grabbing gesture can also be called an air screenshot gesture, the user clenched his fist operation, and so on.

[0092] It should be noted that the AO function on the above electronic device and the implementation process of the AO function are only examples. It is understandable that in actual application, the design can be made according to actual use needs, and this application does not impose any restrictions on this.

[0093] The following will Figure 3 In the usage scenario shown, the electronic device is a mobile phone, and the mobile phone captures images through the front camera as an example to introduce the air recognition method provided in the embodiment of the present application.

[0094] Exemplarily, the air recognition method can be divided into 1) AO image acquisition process: acquiring images through the camera, 2) AO image processing process: identifying whether there is an air operation of the user in the acquired AO image, and 3) AO operation execution process: when the user's air operation is recognized based on the acquired image, executing the preset response operation corresponding to the air operation. Among them, the purpose of the AO image acquisition process is to acquire an AO image that meets the quality requirements, such as acquiring a clear AO image. The AO image can be understood as an image that includes the user's trigger operation. The purpose of the AO image processing process is to identify the trigger operation from the acquired AO image, such as the above-mentioned air grabbing gesture, air up / down gesture, air pressing gesture, etc. The purpose of the AO operation execution process is to execute the AO operation corresponding to the trigger operation, such as executing the screenshot operation corresponding to the air grabbing gesture, executing the page turning operation corresponding to the air up / down gesture, and so on.

[0095] It can be understood that in specific applications, the mobile phone can also capture images through the rear camera, or through the front and rear cameras together. The embodiments of the present application do not impose any restrictions on this.

[0096] For example, see Figure 7 In the air-space identification method provided in the embodiment of the present application, the process of AO image acquisition may include steps S701-S705. In some embodiments, before step S701, the air-space identification method provided in the embodiment of the present application may also include step S700.

[0097] S700. The mobile phone has the real-time online (AO) function turned on.

[0098] Among them, real-time online functions may include but are not limited to air gesture functions and air screenshot functions.

[0099] Exemplarily, the user may enable the real-time online function through a mobile phone-related graphical user interface, such as a settings interface.

[0100] For example, see Figure 8 , the mobile phone displays the desktop 800, and the user can click the setting icon 801 in the desktop 800. In response to the user clicking the setting icon 801, the mobile phone displays the setting menu interface 810. Next, the user clicks the smart perception setting option 811 in the setting menu interface 810. In response to the user clicking the smart perception setting option 811, the mobile phone displays the smart perception menu 820. The smart perception menu 820 may include a switch 821 corresponding to "smart perception". Then, the user clicks the switch 821. In response to the user clicking the switch 821, the mobile phone displays the smart perception related option switches in the smart perception menu 820. For example, the "gesture screenshot" option switch 831 and the "gesture operation screen" option switch 832. The user can turn on the function corresponding to the option switch through the option switch.

[0101] As another example, the user can also use the voice recognition function of the mobile phone to activate the AO function of the mobile phone.

[0102] For example, a user can use the wake-up word of the mobile phone to wake up the voice recognition function of the mobile phone. Then, through a voice command, such as "turn on the AO function", the AO function of the mobile phone is turned on.

[0103] It is understandable that in some other embodiments, the AO function on the mobile phone is set when the mobile phone leaves the factory and is in a normally open state. That is, the user does not need to actively trigger it. After the mobile phone is turned on, the AO function of the mobile phone is automatically turned on, and the user can use the AO function on the mobile phone.

[0104] S701. The mobile phone obtains the current posture of the mobile phone.

[0105] Among them, the posture of the mobile phone can be used to represent the relative position of the mobile phone with respect to the ground plane in different situations.

[0106] For example, see Fig. 9A , Fig. 9A A schematic diagram of a Cartesian coordinate system established with the center of the mobile phone as the origin is shown. For the convenience of the following description, the Cartesian coordinate system established with the center of the mobile phone as the origin is referred to as the coordinate system below. The X-axis of the mobile phone is the direction from the bottom of the mobile phone to the top of the mobile phone, the Z-axis of the mobile phone is the direction from the back of the mobile phone to the front of the mobile phone, and the Y-axis of the mobile phone is the direction from the left side of the mobile phone to the right side of the mobile phone. Among them, the angle of rotation of the mobile phone around the X-axis can be called the roll angle, the angle of rotation of the mobile phone around the Y-axis can be called the pitch angle, and the angle of the mobile phone around the Z-axis can be called the heading angle. It can be understood that in Fig. 9A In the coordinate system shown, the shooting direction of the front camera of the mobile phone is the positive direction of the Z axis, and the shooting direction of the rear camera of the mobile phone is the negative direction of the Z axis. The shooting direction of the camera can be understood as the direction in which the image sensor of the camera points to the lens; for example, the center point of the image sensor points to the direction of the optical center point of the lens.

[0107] For example, see Fig. 9B , Fig. 9BDifferent postures of the mobile phone are shown. Such as the first posture 901, the second posture 902, the third posture 903, the fourth posture 904 and the fifth posture 905, etc. Among them, in the first posture 901, the coordinate system of the mobile phone is a coordinate system composed of XYZ, the X axis is parallel to the longitude direction, and the positive direction of the X axis is the same as the positive direction of the longitude direction; the Y axis is parallel to the latitude direction, and the positive direction of the Y axis is the same as the positive direction of the latitude direction; the Z axis is parallel to the height direction, and the positive direction of the Z axis is the same as the positive direction of the height direction. In the second posture 902, the coordinate system of the mobile phone is a coordinate system composed of X1-Y1-Z1, the X1 axis is parallel to the longitude direction, and the positive direction of the X1 axis is opposite to the positive direction of the longitude direction; the Y1 axis is parallel to the latitude direction, and the positive direction of the Y1 axis is opposite to the positive direction of the latitude direction; the Z1 axis is parallel to the height direction, and the positive direction of the Z1 axis is the same as the positive direction of the height direction. In the third posture 903, the coordinate system of the mobile phone is a coordinate system composed of X2-Y2-Z2, the X2 axis is parallel to the latitude direction, and the positive direction of the X2 axis is the same as the positive direction of the latitude direction; the Y2 axis is parallel to the longitude direction, and the positive direction of the Y2 axis is opposite to the positive direction of the longitude direction; the Z2 axis is parallel to the height direction, and the positive direction of the Z2 axis is the same as the positive direction of the height direction. In the fourth posture 904, the coordinate system of the mobile phone is a coordinate system composed of X3-Y3-Z3, the X3 axis is parallel to the longitude direction, and the positive direction of the X3 axis is the same as the positive direction of the longitude direction; the Y3 axis is parallel to the latitude direction, and the positive direction of the Y3 axis is opposite to the positive direction of the latitude direction; the Z3 axis is parallel to the height direction, and the positive direction of the Z3 axis is opposite to the positive direction of the height direction. In the 5th posture 905, the coordinate system of the mobile phone is a coordinate system composed of X4-Y4-Z4, the X4 axis is parallel to the latitude direction, and the positive direction of the X4 axis is opposite to the positive direction of the latitude direction; the Y4 axis is parallel to the longitude direction, and the positive direction of the Y4 axis is opposite to the positive direction of the longitude direction; the Z4 axis is parallel to the height direction, and the positive direction of the Z4 axis is opposite to the positive direction of the height direction.

[0108] It can be understood that the above-mentioned first posture 901, second posture 902, third posture 903, fourth posture 904 and fifth posture 905 are only examples. In actual applications, the mobile phone may also have other postures, and the embodiments of the present application do not impose any limitations on this.

[0109] In the following embodiments of the present application, Fig. 9A Taking the coordinate system shown as an example, the technical solution provided by this application is introduced. It should be pointed out that Fig. 9A The coordinate system shown is only an example; specifically, it can be designed according to actual usage requirements, and this application will not go into details here.

[0110] In some embodiments, the mobile phone can collect corresponding data through its own sensors and obtain the current posture of the mobile phone based on the collected data. In one possible design, the mobile phone can detect the direction of gravity acceleration through the built-in acceleration sensor of the mobile phone and obtain the current posture of the mobile phone based on the detected direction of gravity acceleration.

[0111] For example, the mobile phone may be preset with an initial posture, as described above Fig. 9B The first posture 901 in the figure. After that, the mobile phone can detect the direction of gravity acceleration through the acceleration sensor to determine the current coordinate system of the mobile phone. Next, the mobile phone can determine the current posture of the mobile phone based on the difference between the current coordinate system of the mobile phone and the initial posture coordinate system, such as the current posture is Fig. 9B The second posture 902, the third posture 903, the fourth posture 904 or the fifth posture 905.

[0112] In another possible design, the mobile phone may be preset with an initial posture, as described above. Fig. 9B The first posture 901 in the figure. In the initial posture, the heading angle, roll angle and pitch angle of the mobile phone are all 0 degrees. After that, the mobile phone detects the changes of the heading angle, roll angle and pitch angle through the gyroscope. Next, the mobile phone obtains the difference between the current coordinate system of the mobile phone and the initial posture coordinate system based on the changes of the heading angle, roll angle and pitch angle, and determines the current posture of the mobile phone based on the difference. For example, the current posture is Fig. 9B The second posture 902, the third posture 903, the fourth posture 904 or the fifth posture 905.

[0113] It is understandable that in some other possible designs, the mobile phone can also obtain the current posture of the mobile phone by combining the gyroscope and the acceleration sensor. For example, the mobile phone first determines the current posture of the mobile phone through the gyroscope. After that, the mobile phone corrects the current posture of the mobile phone determined by the gyroscope through the acceleration sensor. This can improve the accuracy of the calculation of the mobile phone posture.

[0114] In some embodiments, the mobile phone may periodically acquire the current posture of the mobile phone. For example, the mobile phone may periodically acquire corresponding data through its own sensor and acquire the current posture of the mobile phone based on the acquired data.

[0115] Exemplarily, the mobile phone can start a timer, and if the timer is greater than or equal to the preset time duration, the mobile phone obtains the current posture of the mobile phone through the sensor. The preset time duration can be 3 seconds, 5 seconds, 10 seconds, etc. It can be understood that since the operation of the mobile phone's own sensor also consumes the power of the mobile phone; therefore, by periodically obtaining the current posture of the mobile phone, the power consumption of the electronic device can be further reduced, the battery life of the electronic device can be increased, and the user experience can be improved.

[0116] S702. The mobile phone calculates the depth of field of the camera in the current posture.

[0117] Among them, the depth of field (DOF) refers to the distance before and after the focus of the camera that can present a clear image. Objects within the depth of field can be clearly captured by the camera. In other words, the objects within a certain distance range within the camera's field of view can be clearly captured by the camera. On the contrary, objects outside the depth of field cannot be clearly captured by the camera.

[0118] For example, see Fig.10 , assuming that the depth of field of the mobile phone camera is [15CM.60CM], object A is 80cm away from the mobile phone, object B is 50cm away from the mobile phone, and person C is 20cm away from the mobile phone. Since object A is outside the depth of field of the mobile phone, it cannot be clearly captured by the mobile phone camera; therefore, in the picture taken by the mobile phone, object A is blurred and the clarity is very low. Since person C and object B are within the depth of field of the mobile phone, they can be clearly captured by the mobile phone camera; therefore, in the picture taken by the mobile phone, person C and object B are clear and the clarity is relatively high.

[0119] Since the camera is deployed in a fixed position on the phone, after obtaining the current posture of the phone, the phone can obtain the shooting direction of the camera through the current posture. For example, assuming that the shooting direction of the front camera is the same as the positive direction of the Z axis of the phone, the phone can obtain the shooting direction of the front camera based on the current posture. For example, the shooting direction is horizontal, the shooting direction is vertical, the angle between the shooting direction and the horizontal direction is N degrees, etc. Among them, 90≥N≥0.

[0120] Next, the mobile phone can calculate the component of the gravity of the lens in the shooting direction based on the shooting direction of the front camera, that is, the influence of the gravity of the lens on the camera. It can be understood that the existence of the self-weight of the lens will affect the process of the AF module controlling the movement of the lens, and / or when the AF module does not control the movement of the lens, the distance between the lens and the image sensor will be affected by the self-weight of the lens. Therefore, by calculating the component of the gravity of the lens in the shooting direction in the current posture, the influence of the gravity of the lens on the lens is characterized. And in the subsequent steps, the mobile phone uses the AF module to offset the influence of the self-weight of the lens on the camera.

[0121] For example, see Fig.11 , step S702 may include steps S7020-S7022.

[0122] S7020. The mobile phone calculates the component of the gravity of the lens in the shooting direction in the current posture.

[0123] For example, see Fig.12 , the mobile phone can calculate the gravity on the lens through the self-weight of the lens. Then, the component of gravity in the shooting direction is calculated through the angle between the Z axis and the vertical direction. It can be understood that the self-weight of the lens or the weight of the lens is fixed and is configured when the lens leaves the factory. In some embodiments, the mobile phone can obtain the self-weight of the lens through the parameter file of the camera.

[0124] S7021. The mobile phone calculates the image distance of the camera based on the component of the gravity of the lens in the shooting direction.

[0125] It is understandable that due to the influence of the lens' own weight, the distance between the lens and the image sensor will change, so the image distance of the camera can be calculated by the component of the lens' gravity in the shooting direction.

[0126] For example, after the mobile phone obtains the component of the gravity of the lens in the shooting direction, the mobile phone can calculate the image distance of the camera based on the component of the gravity of the lens in the shooting direction and the friction of the lens, and / or the component of the gravity of the lens in the shooting direction and the elastic modulus of the elastic device.

[0127] S7022. The mobile phone calculates the depth of field of the camera in the current posture based on the image distance of the camera.

[0128] After obtaining the image distance of the camera, the mobile phone can obtain the object distance of the camera based on the Gaussian imaging formula or Newton's formula.

[0129] Next, the mobile phone can calculate the depth of field of the camera in the current posture according to the following expression 1 or expression 2.

[0130]

[0131]

[0132] Among them, △L represents the depth of field of the camera, δ represents the diameter of the confusion circle, f represents the focal length of the lens, F represents the aperture of the lens, and L represents the object distance of the camera. The diameter of the confusion circle is a parameter related to the resolution of the camera, and the aperture and focal length of the lens are both physical parameters related to the lens, which can be obtained from the parameter file of the camera or pre-configured.

[0133] It is understandable that in some embodiments, the depth of field may be a range value, for example, it may be calculated by the object distance and the foreground depth of field and the back depth of field, such as [L+ΔL1, L+ΔL2], where ΔL1 represents the foreground depth of field and ΔL2 represents the back depth of field.

[0134] Illustratively, the foreground depth ΔL1 can be calculated by Expression 3 or Expression 4 below.

[0135]

[0136]

[0137] Illustratively, the depth of field ΔL2 can be calculated by Expression 5 or Expression 6 below.

[0138]

[0139]

[0140] It should be pointed out that in other solutions, there may be other ways to calculate the depth of field of the camera, and the embodiments of the present application do not impose any limitations on this.

[0141] Assume that the depth of field of the camera is calculated to be [20.50], and the unit of the depth of field can be centimeter (CM). Among them, 20 can be the front depth of the camera, and 50 can be the back depth of the camera. The depth of field is [20.50], which can be understood as objects within the viewing angle of the camera and within the range of [20.50] from the camera can be clearly photographed by the camera. In other words, the objects within the range of [20.50] photographed by the camera will be clear.

[0142] S703. The mobile phone determines whether the depth of field meets the AO acquisition conditions under the current posture; if so, the depth of field of the mobile phone is not adjusted and step S705 is directly executed; if not, steps S704 and S705 are executed.

[0143] Among them, the AO acquisition condition can be used to characterize whether the mobile phone can acquire a clear AO image. If the depth of field of the mobile phone meets the AO acquisition condition, it means that the mobile phone can acquire a clear AO image. If the depth of field of the mobile phone does not meet the AO acquisition condition, it means that the mobile phone cannot acquire a clear AO image. The AO image can be understood as an image used to trigger the AO function of the mobile phone; for example, the AO image can include a gesture image, a face image, and the like.

[0144] It is understandable that when a user uses the AO function of a mobile phone, the distance between the user and the mobile phone is mostly fixed within a certain distance range, so the AO acquisition conditions can be preset based on the distance range, so that the AO acquisition conditions can be used to determine whether a clear AO image can be acquired. If a clear AO image cannot be acquired, that is, the AO acquisition conditions are not met, it is necessary to adjust the positional relationship between the lens and the image sensor in the mobile phone camera; if a clear AO image can be acquired, that is, the AO acquisition conditions are met, there is no need to adjust the positional relationship between the lens and the image sensor in the mobile phone camera. In this way, the number of times the mobile phone drive motor adjusts the distance between the lens and the image sensor can be reduced, which means that the working time of the mobile phone's AF module is reduced, which can reduce the power consumption of the mobile phone and improve the battery life of the mobile phone.

[0145] In some embodiments, the AO acquisition conditions of the mobile phone in different postures may be the same.

[0146] Exemplarily, the AO acquisition condition may include a preset AO distance within the depth of field of the camera, wherein the AO distance may be related to the initial depth of field of the camera and / or the initial image distance of the camera.

[0147] For example, the preset AO distance may be the initial depth of field of the camera, or the preset AO distance may be the median of the initial depth of field, or the preset AO distance may be the average of the median of the initial depth of field and the initial image distance, or the preset AO distance may be the initial image distance, etc. Assume that the above preset AO distance may be the initial image distance, which is 40CM. In other words, if 40CM is within the depth of field of the mobile phone, the AO acquisition condition is met, otherwise it is not met.

[0148] In another exemplary embodiment, the AO acquisition condition may include a preset AO range within the depth of field of the camera, wherein the AO range may be related to the initial depth of field of the camera and / or the initial image distance of the camera.

[0149] For example, the AO range may be the initial depth of field of the camera, or the AO range may be a range formed by the median value of the initial depth of field and the initial image distance, and so on.

[0150] Assume that the initial depth of field is [25.45]. If the depth of field of the mobile phone includes [25.45], that is, the initial depth of field [25.45] is within the depth of field of the mobile phone, then the AO acquisition condition is met, otherwise it is not met.

[0151] In other embodiments, the AO acquisition conditions of the mobile phone in different postures may be different.

[0152] It is understandable that, considering that in different postures, when a user uses the AO function of a mobile phone, the distance between the user and the mobile phone may be different, different AO acquisition conditions can be set for different postures, so that the mobile phone can acquire clear AO images in different postures.

[0153] Exemplarily, the acquisition condition of the mobile phone in a vertical posture may be a vertical AO acquisition condition, the acquisition condition of the mobile phone in a horizontal posture may be a horizontal AO acquisition condition, and the acquisition condition of the mobile phone in other postures may be other AO acquisition conditions.

[0154] For example, see Fig.13 The vertical posture of the mobile phone can be understood as the shooting direction of the mobile phone camera is horizontal; the horizontal posture of the mobile phone can be understood as the shooting direction of the mobile phone camera is vertical; other postures of the mobile phone can be understood as the posture of the hand between the horizontal posture and the vertical posture.

[0155] The vertical AO acquisition condition may include a preset AO range within the depth of field of the camera, or a preset AO distance within the depth of field of the camera. For the relevant contents of the AO range and AO distance, please refer to the above introduction, which will not be repeated here.

[0156] Other AO acquisition conditions may include a preset specified range A within the depth of field of the camera, or a preset specified distance A within the depth of field of the camera.

[0157] The horizontal AO acquisition condition may include a preset specified range B being within the depth of field of the camera, or a preset specified distance B being within the depth of field of the camera.

[0158] Among them, the specified range A can also be called the second preset range, the specified range B can be called the first preset range, the specified distance A can be called the second preset distance, and the specified distance B can be called the first preset distance. Exemplarily, the specified range A is [20.35], [20.45], [18.35], etc., and the specified range B can be [20.35], [20.45], [15.35], etc.; the specified distance A can be 20, 24, 30, 40, etc., and the specified distance A can be 20, 24, 30, 45, etc. The specified range A can be the same as or different from the specified range B, and the specified distance A can be the same as or different from the specified distance B. Specifically, this can be designed according to actual usage needs, and the embodiments of the present application do not limit this.

[0159] It is understandable that when a user uses a mobile phone in different postures, the distance between the user and the mobile phone will be different. For example, the distance between the user and the mobile phone will become smaller as the state of the mobile phone changes (e.g., the sequence of vertical state-other state-horizontal state). Therefore, the above-mentioned specified distance A can be less than or equal to the above-mentioned AO distance, and the above-mentioned specified distance B can be less than or equal to the above-mentioned specified distance A.

[0160] Exemplarily, the collection conditions of the mobile phone in different postures may be as shown in the following Table 1.

[0161] Table 1

[0162] AO acquisition conditions Mobile phone posture scope distance Horizontal AO acquisition conditions Horizontal attitude Specified range B = [20.40] Specified distance B = 24 Other AO acquisition conditions Other postures Specified range A = [20.40] Specified distance A = 24 Vertical AO acquisition conditions Vertical posture AO range = [25.45] AO distance = 40

[0163] Among them, the initial depth of field of the camera can be [25.45], and the initial image distance can be 40.

[0164] S704. The mobile phone adjusts the depth of field of the camera so that the adjusted depth of field meets the AO acquisition condition in the current posture.

[0165] In some embodiments, the mobile phone can adjust the depth of field of the camera by adjusting the image distance of the camera (that is, the distance between the lens and the image sensor). Alternatively, the mobile phone can also adjust the depth of field of the camera by adjusting the aperture of the camera. And, it can be understood that in other embodiments, the mobile phone can adjust the depth of field of the camera by combining the adjustment of the image distance of the camera and the adjustment of the aperture of the camera, and the embodiments of the present application do not impose any restrictions on this.

[0166] Below, the technical solution provided in the embodiment of the present application will be introduced by taking adjusting the depth of field of the camera by adjusting the image distance of the camera as an example.

[0167] Assume that the AO acquisition conditions of the mobile phone in different postures are the same, and the preset AO range is within the depth of field of the camera. The mobile phone can control the movement of the lens through the drive motor in the AF module; in this way, the distance between the camera and the image sensor can be changed, and the image distance of the camera is adjusted, that is, the depth of field of the camera is adjusted. Among them, the drive motor can apply different push / pull forces to the lens to control the movement of the lens. The mobile phone (processor) calculates the force that the drive motor of the AF module needs to compensate for the component of the gravity in the shooting direction (hereinafter referred to as the compensation force). After that, the mobile phone generates the code corresponding to the compensation force. Next, the mobile phone sends the code to the IC chip of the AF module. Then, based on the code, the IC chip of the AF module outputs the current specified by the code to the drive motor. In this way, the drive motor can apply a compensation force to the lens to compensate for the influence of the gravity of the lens on the depth of field of the camera (such as offsetting the influence of the gravity of the lens on the depth of field of the camera). Then, the depth of field of the camera after applying the compensation force is the initial depth of field, and the image distance is the initial image distance.

[0168] Exemplarily, regarding the above compensation force, the mobile phone can calculate the image distance to be compensated according to the difference between the AO range (that is, the initial depth of field) and the current depth of field of the mobile phone through the above expression 1 or expression 2. Then, the compensation force is calculated according to the image distance to be compensated and the elastic modulus of the elastic device.

[0169] Based on this, the mobile phone offsets the influence of gravity on the camera through the AF module, so that the adjusted depth of field can meet the above-mentioned AO acquisition condition of the AO distance, allowing the mobile phone to clearly shoot objects within the initial depth of field. At the same time, the mobile phone only needs to adjust the distance between the lens and the image sensor once through the AF module to complete the focus of the camera. Compared with some solutions that require multiple adjustments to the distance between the lens and the image sensor, the method provided by the embodiment of the present application can effectively reduce the energy consumption of the mobile phone and improve the battery life of the mobile phone.

[0170] Assume that the AO acquisition conditions of the mobile phone are different in different postures, as shown in Table 1 above. The mobile phone can calculate the image distance that needs to be compensated by the above expression 1 or expression 2. Then, the compensation force is calculated according to the image distance that needs to be compensated. It can be understood that if the compensation force is applied to the lens, the adjusted depth of field will be the initial depth of field, such as [25.45]. If the mobile phone is in other postures or horizontal postures, the depth of field adjusted by applying the compensation force still does not meet the AO acquisition conditions. Therefore, the mobile phone needs to apply an adjustment force to the lens on the basis of the compensation force, so that the depth of field of the camera after the adjustment force and the compensation force are applied meets the AO acquisition conditions.

[0171] Regarding the calculation process of the adjustment force, the mobile phone can first calculate the difference between the initial depth of field and the depth of field in the current posture, and then calculate the image distance that needs to be adjusted through the above expression 1 or expression 2. After that, the adjustment force is calculated according to the image distance that needs to be adjusted and the elastic modulus of the elastic device. After that, the mobile phone generates a code corresponding to the combined force of the above adjustment force and the above compensation force. Next, the mobile phone sends the code to the IC chip of the AF module. Then, the IC chip of the AF module outputs the current specified by the code to the drive motor based on the code. In this way, the drive motor can apply an adjustment force to the lens so that the depth of field of the adjusted camera meets the depth of field of the AO acquisition condition in the current posture. In this way, the mobile phone can collect a clear AO image through the camera. At the same time, the mobile phone only needs to change the distance between the lens and the image sensor once, or does not change the distance between the lens and the image sensor, to complete the focus of the camera, which can effectively reduce the energy consumption of the mobile phone and improve the battery life of the mobile phone.

[0172] S705. The mobile phone collects the AO image through the camera.

[0173] In some embodiments, after executing step S705, the mobile phone may execute step S701 again. In this way, by executing steps S701-S703 or steps S701-S704 multiple times, the mobile phone can continuously adjust the depth of field of the mobile phone according to the posture of the mobile phone. Then, when executing step S705, the mobile phone can collect a clear AO image through the camera. As a result, the mobile phone can subsequently perform the AO image processing process and the AO operation execution process through the clear AO image.

[0174] The following will introduce in detail the air recognition method provided in the embodiment of the present application in combination with the user's usage scenarios of the AO function of the electronic device.

[0175] After the user turns on the AO function of the mobile phone, the camera of the mobile phone is in the normally-on state, and the electronic device collects AO images through the camera. The normally-on state of the mobile phone's camera can be understood as the camera is always on; or it can be understood as the camera is in a sleep state, and is turned on periodically (every 1 second, every 3 seconds), etc. Afterwards, when the user's air operation is recognized based on the collected image, the electronic device performs a preset response operation corresponding to the air operation. When the camera collects images, if the electronic device is in a first posture, the AF module is in a powered-on state; if the electronic device is in a second posture, the AF module is in a powered-off state. The posture of the electronic device is used to characterize the relative position relationship between the electronic device and the ground plane. Among them, the above-mentioned first posture includes that the shooting direction of the camera is vertical, or the shooting direction of the camera is between vertical and horizontal; the second posture includes that the shooting direction of the camera of the electronic device is horizontal. For example, the above-mentioned first posture may include Fig.13The horizontal posture or other posture in the second posture may include Fig.13 Vertical posture in.

[0176] Among them, the process of the user turning on the AO function of the mobile phone can refer to the relevant description of the above step S700, which will not be repeated here. The mobile phone can perform image recognition on the collected image through a pre-trained image recognition model to obtain the user's air operation. And execute the preset response operation corresponding to the air operation. For example, the user's gaze at the mobile phone can correspond to the mobile phone lighting up the screen, the above-mentioned user's fist operation can correspond to the mobile phone screenshot, and so on. The specific design can be based on actual usage needs, and the embodiment of the present application does not impose any restrictions on this. It can be understood that if the user is not looking at the mobile phone, or the user does not make a fist operation towards the mobile phone; the mobile phone may not obtain a recognition result when recognizing the AO image, and the mobile phone does not respond at this time.

[0177] In the above method, the AF module can be powered on only when necessary, and not powered on when not necessary; for example, the AF module of the electronic device in the second posture is in a powered-off state, that is, the AF module will not work, and the AF module will not consume the power of the electronic device. This can reduce the power consumption of the AF module during the air recognition process, thereby increasing the battery life of the electronic device and improving the user experience.

[0178] As a possible implementation method, the above-mentioned electronic device collects AO images through a camera, which may include: the electronic device determines the current posture of the electronic device; the current posture includes a first posture or a second posture. Afterwards, the electronic device determines the current depth of field of the camera of the electronic device in the current posture based on the current posture of the electronic device. Then, according to the current depth of field of the camera of the electronic device and the AO acquisition condition corresponding to the current posture, the current working state of the AF module is controlled, and the image is collected through the camera in the current working state of the AF module. Among them, the current working state of the AF module includes a power-on state or a power-off state. The AO acquisition condition corresponding to the current posture includes the condition that the camera collects an image that meets the quality requirements in the current posture. For example, the camera collects an image whose clarity meets the AO mode requirements of the electronic device in the current posture. A detailed description of this implementation method can be found in the above steps S701-S705, which will not be repeated here.

[0179] For example, see Fig.14A, the mobile phone is in a horizontal posture, and the shooting direction of the mobile phone (front) camera is vertical. At this time, the mobile phone executes the above steps S701-S704. When executing step S703, since the shooting direction of the camera is vertical, the gravity of the lens has an impact on the camera, and the depth of field of the camera does not meet the horizontal AO acquisition condition. The mobile phone executes step S704, calculates the compensation force, generates the code A corresponding to the compensation force, and sends the code A to the AF module. At this time, the AF module will be in a power-on state. The IC chip of the AF module can output the current corresponding to the code A to the drive motor based on the code A. The drive motor applies the above compensation force to the lens, adjusts the distance between the lens and the image sensor, so as to adjust the depth of field of the camera, so that the depth of field of the adjusted camera meets the depth of field of the horizontal AO acquisition condition. Then, the mobile phone collects the first AO image through the camera, and recognizes the first AO image through the above image recognition model to obtain a recognition result. For example, if the user is looking at the mobile phone, the mobile phone controls the display screen to light up.

[0180] For example, see Fig. 14B , the mobile phone is in a vertical posture, and the shooting direction of the mobile phone (front) camera is horizontal. At this time, the mobile phone executes the above steps S701-S704. When executing step S703, since the shooting direction of the camera is horizontal, the gravity of the lens is perpendicular to the shooting direction, and gravity has no effect on the camera; if the AF module does not adjust the distance between the image sensor and the lens, the depth of field of the camera is the initial depth of field. It can be seen from Table 1 that the initial depth of field meets the AO acquisition conditions. In other words, when the mobile phone executes step S703, the mobile phone does not adjust the depth of field of the camera, and the depth of field of the camera will meet the vertical AO acquisition conditions. At this time, due to the previous (such as, Fig.14A In the corresponding solution), the AF module is powered on based on the indication of code A. The mobile phone can instruct the AF module to power off, so that the AF module is powered off and does not work; the IC chip of the AF module will not output current to the drive motor. Afterwards, the mobile phone collects the second AO image through the camera, and recognizes the second AO image through the above-mentioned image recognition model to obtain a recognition result, such as the user making a fist operation, and the mobile phone takes a screenshot.

[0181] Compared with some solutions, the AF module uses a contrast detection focusing method to achieve the focus of the camera to capture a clear second AO image. Fig. 14B In the corresponding scene, the AF module is in a power-off state, that is, the power consumption of the AF module is very small; at the same time, the depth of field of the camera is the initial depth of field, and a clear second AO image can also be captured. It can be seen that the method provided in the embodiment of the present application can reduce the power consumption of the AF module, thereby improving the battery life of the electronic device and improving the user experience.

[0182] For further example, see Fig. 14C , the mobile phone is in other postures, and the shooting direction of the mobile phone (front) camera is between horizontal and vertical. At this time, the mobile phone executes the above steps S701-S704. When executing step S703, since the shooting direction of the camera is vertical, gravity has an impact on the camera, and the depth of field of the camera does not meet other AO acquisition conditions. The mobile phone executes step S704, calculates the adjustment force and the compensation force, and generates a code B corresponding to the combined force of the adjustment force and the compensation force, and sends the code B to the AF module. At this time, the AF module will be in a power-on state. Based on the code B, the IC chip of the AF module can output the current corresponding to the code B to the drive motor. The drive motor applies the above adjustment force and compensation force to the lens, so that the depth of field of the adjusted camera meets the depth of field of the horizontal AO acquisition condition. Then, the mobile phone collects the third AO image through the camera, and recognizes the third AO image through the above image recognition model to obtain a recognition result. For example, if the user is looking at the mobile phone, the mobile phone controls the display screen to light up.

[0183] For example, see Fig.14D , the mobile phone is in a horizontal posture, the shooting direction of the mobile phone (front) camera is vertical, and the AO acquisition conditions can refer to the distance part in the above Table 1. At this time, the mobile phone executes the above steps S701-S704, generates code C, and sends code C to the AF module, so that the AF module of the mobile phone is in a powered-on state. Afterwards, the mobile phone collects the fourth AO image through the camera, and recognizes the fourth AO image through the above image recognition model to obtain a recognition result, such as a user's fist operation, and the mobile phone takes a screenshot.

[0184] It is understandable that in Fig. 14B In the corresponding scenario, the phone is in a vertical position. If before this, the phone has experienced Fig.14A , Fig. 14C or Fig.14D In the corresponding scenario, the mobile phone has executed step S704, that is, the AF module is already powered on. Fig. 14B In the corresponding scenario, the mobile phone executes step S703 to determine whether the depth of field meets the AO acquisition conditions; and at this time the AF module is in a powered-on state. The mobile phone can instruct the AF module to power off, that is, instruct the AF module to change from a powered-on state to a powered-off state. It is understandable that when the mobile phone is in a vertical posture, if the AF module is powered off, the depth of field of the camera will be the initial depth of field, and it will also meet the AO acquisition conditions, and the camera can capture images that meet the AO requirements. In this way, the power consumption of the AF module can be further reduced, and the battery life of the electronic device can be further improved.

[0185] It should be pointed out that Fig.14DIn the corresponding scenario, since the shooting direction of the mobile phone camera is vertical, the influence of gravity on the lens is the greatest; that is, the compensation force obtained by the mobile phone in a horizontal posture after executing step S704 will be greater than or equal to the compensation force obtained by the mobile phone in other postures after executing step S704. Based on this, since the compensation force of the mobile phone in a horizontal posture is the greatest, the current output by the IC chip of the AF module in a horizontal posture to the drive motor will also be the greatest. In other words, the current output by the IC chip of the AF module in a horizontal posture to the drive motor will be greater than the current output by the IC chip of the AF module in other postures to the drive motor.

[0186] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0187] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0188] The present application also provides an electronic device, such as Fig.15 As shown, the electronic device may include one or more processors 2001 , a memory 2002 and a communication interface 2003 .

[0189] The memory 2002 and the communication interface 2003 are coupled to the processor 2001. For example, the memory 2002, the communication interface 2003 and the processor 2001 may be coupled together via a bus 2004.

[0190] The communication interface 2003 is used for data transmission with other devices. The memory 2002 stores computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the processor 2001, the electronic device executes the relevant method steps in the above method embodiment of the present application.

[0191] Among them, the processor 2001 can be a processor or a controller, for example, it can be a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like.

[0192] The bus 2004 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus 2004 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0193] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program code is stored. When the processor executes the computer program code, the electronic device executes the relevant method steps in the method embodiment.

[0194] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the relevant method steps in the above method embodiment.

[0195] Among them, the electronic device, computer-readable storage medium or computer program product provided in this application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0196] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0197] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0200] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0201] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A remote recognition method, characterized in that: The method is applied to an electronic device including a camera, the electronic device starts a real-time online AO ​​mode, and the camera includes an auto-focus AF module; The method comprises: Collecting images through the camera; In the case where an airborne operation of the user is recognized based on the collected image, a preset response operation corresponding to the airborne operation is executed; The collecting of images by the camera comprises: Determine a current posture of the electronic device; the current posture includes a first posture or a second posture; the posture of the electronic device is used to characterize a relative position relationship between the electronic device and a ground plane; Based on the current posture of the electronic device, determining the current depth of field of the camera of the electronic device in the current posture; According to the current depth of field of the camera of the electronic device and the AO acquisition condition corresponding to the current posture, the current working state of the AF module is controlled, and an image is acquired by the camera in the current working state of the AF module; wherein the current working state of the AF module includes a power-on state or a power-off state; the AO acquisition condition corresponding to the current posture includes a condition that the camera acquires an image that meets quality requirements in the current posture; The controlling the current working state of the AF module according to the current depth of field and AO acquisition conditions of the camera of the electronic device includes: If the current depth of field meets the AO acquisition condition corresponding to the current posture, controlling the current working state of the AF module to be a power-off state; If the current depth of field does not meet the AO acquisition condition corresponding to the current posture, the current working state of the AF module is controlled to be a power-on state.

2. The method according to claim 1, characterized in that The first posture includes that the shooting direction of the camera is vertical, or the shooting direction of the camera is between vertical and horizontal; the second posture includes that the shooting direction of the camera is horizontal.

3. The method according to claim 2, characterized in that The AF module includes a driving motor and an integrated circuit IC chip, and the IC chip is used to provide working current to the driving motor when the AF module is in a power-on state; When the AF module is in a powered-on state and the shooting direction of the camera is vertical, the IC chip provides a first working current to the driving motor; When the AF module is in a power-on state and the shooting direction of the camera is between vertical and horizontal, the IC chip provides a second operating current to the driving motor; and the first operating current is greater than or equal to the second operating current.

4. The method according to any one of claims 1 to 3, characterized in that: The camera further includes a lens, and determining the current depth of field of the camera of the electronic device in the current posture based on the current posture of the electronic device includes: Obtaining a current shooting direction of the camera based on the current posture; Based on the mass of the lens, a component of the gravity of the lens in the current shooting direction is calculated; Based on the component of the gravity of the lens in the current shooting direction, the current depth of field of the camera is obtained.

5. The method according to claim 4, characterized in that The current working state of the AF module is a power-on state, and collecting an image through the camera in the current working state of the AF module includes: The depth of field of the camera is adjusted by the AF module, and an image is captured by the adjusted camera; the depth of field of the adjusted camera satisfies the AO acquisition condition corresponding to the current posture.

6. The method according to claim 5, characterized in that The adjusting the depth of field of the camera through the AF module includes: Obtaining a compensated image distance according to a difference between the current depth of field and the initial depth of field, or obtaining a compensated image distance according to a difference between the current depth of field and the initial image distance; The image distance of the camera is adjusted to the compensated image distance through the AF module to adjust the depth of field of the camera.

7. The method according to any one of claims 1 to 3, characterized in that: The AO acquisition conditions corresponding to the current posture specifically include: The current posture includes the first posture, the first posture includes that the shooting direction of the camera is vertical, or the shooting direction of the camera is between vertical and horizontal, and when the first posture is that the shooting direction of the camera is vertical, the AO acquisition condition corresponding to the first posture specifically includes: a first preset distance is within the current depth of view, or a first preset range is within the current depth of view; when the first posture is that the shooting direction of the camera is between vertical and horizontal, the AO acquisition condition corresponding to the first posture specifically includes: a second preset distance is within the current depth of view, or a second preset range is within the current depth of view; The current posture includes the second posture, the second posture includes that the shooting direction of the camera is horizontal, and the AO acquisition conditions corresponding to the second posture specifically include: the initial depth of field of the camera is within the current depth of field, or the initial image distance of the camera is within the current depth of field.

8. The method according to any one of claims 1 to 3, characterized in that: The determining the current posture of the electronic device comprises: Determining the current posture of the electronic device by an acceleration sensor of the electronic device; and / or, The current posture of the electronic device is determined by a gyroscope of the electronic device.

9. An electronic device, characterized in that: The electronic device includes a memory, one or more processors and a camera, the memory is coupled to the processor, and the processor is coupled to the camera; wherein computer program code is stored in the memory, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

Images