Image focusing method and related equipment

Abstract

The present application discloses an image focusing method and related equipment, which is applied to an electronic device, the electronic device includes a first sensor and a camera, the method includes: drawing a first image frame by the first sensor; the starting drawing time of all rows of the first image frame is the same; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to draw all rows of any image frame at the same time; focusing on the subject in the first image frame to obtain the focused first image frame; drawing the first image and focusing on the subject in the first image are completed within a first time period, the first time period is the time interval between two adjacent image frames displayed at the first frame rate; based on the first frame rate, the focused first image frame is displayed. The frame rate can be increased by the present application, thereby increasing the AF scheduling frame rate.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image focusing method and related equipment. Background Art

[0002] When using the camera to shoot, users may experience slow focusing in night scenes and sports scenes. This slow focusing sensation is caused by low frames per second (FPS) and thus low automatic focus (AF) scheduling frame rate.

[0003] In night scenes, due to the dark ambient light and long exposure time, if the FPS is too high, the exposure time cannot be met. Therefore, in night scenes, the FPS is low and the AF scheduling frame rate is low. For example, the frame rate of photos taken under light is 30FPS. At this 30FPS frame rate, the time corresponding to one frame is 0.033 seconds (1 / 30FPS), which means that a maximum of 30 images can be displayed on the display screen per second. In night scenes, 0.033 seconds is not enough to expose the image, so in night scenes, the time corresponding to one frame needs to be greater than 0.033 seconds, such as 0.05 seconds. The frame rate of photos taken in night scenes is 20FPS (1 / 0.05 seconds). Therefore, in night scenes, the FPS cannot be increased. For sports scenes, the moving subject is faster, so a higher FPS is required. Summary of the invention

[0004] The present application provides an image focusing method and related equipment, which can improve the frame rate, thereby improving the AF scheduling frame rate.

[0005] In the first aspect, some embodiments of the present application provide an image focusing method. The method is applied to an electronic device, which includes a first sensor and a camera, and the method includes: drawing a first image frame through the first sensor; the starting drawing time of all rows of the first image frame is the same, and the first image frame is any image frame at a first frame rate; the first frame rate is the frame rate corresponding to the first scene, and the first scene is the scene of the image captured by the camera; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to draw all rows of any image frame at the same time; focus on the subject in the first image frame to obtain a focused first image frame; the drawing of the first image and the focusing of the subject in the first image are completed within a first time period, and the first time period is the time interval between displaying two adjacent image frames at the first frame rate; based on the first frame rate, the focused first image frame is displayed.

[0006] In the above manner, all rows of any image frame are drawn simultaneously by the first sensor, thereby reducing the time for drawing any frame. Since the time for drawing any image frame is reduced, a higher frame rate (FPS) can be set. Since the higher the frame rate, the more image frames are drawn in one second, and autofocus needs to be performed after drawing each image frame, when the frame rate is increased, the AF scheduling frame rate is also increased.

[0007] In another implementation, the first sensor is a depth sensor; focusing on the subject in the first image frame includes: after the first image frame is drawn, focusing on the subject in the first image frame.

[0008] Through the above method, the depth sensor is a sensor that draws all rows of any image frame at the same time. The characteristic of this sensor is that after the image frame is drawn, the image frame is focused. Since the depth sensor only needs a very short time to expose the image frame when drawing (the exposure method adopted by the depth sensor is micro-exposure or no exposure), and the depth sensor takes a long time to draw the image frame, the use of the depth sensor can improve the frame rate and increase the AF scheduling frame rate.

[0009] In another implementation, the method also includes: determining a first scene of the image captured by the camera based on at least one second sensor, the first scene including the brightness of the environment in which the image captured by the camera is located, and the motion state of the subject in the image captured by the camera; determining a first frame rate corresponding to the first scene based on the first scene.

[0010] In the above manner, different first frame rates are determined for different scenes of the image captured by the camera, so that when the scene does not require a high frame rate, the frame rate is reduced to save energy consumption of the electronic device; when the scene requires a high frame rate, the frame rate is increased to ensure clarity during display.

[0011] In another implementation, the second sensor includes an ambient light sensor for detecting the ambient brightness and a subject motion detection module for detecting the motion state of the subject; based on at least one second sensor, determining that the scene of the image captured by the camera is the first scene includes: determining the brightness of the environment in which the image captured by the camera is located based on the ambient light sensor; and determining the motion state of the subject in the image captured by the camera based on the subject motion detection module.

[0012] Through the above method, in night scenes (the brightness of the environment in which the image captured by the camera is located is low) and when the subject moves at high speed (the motion state of the subject in the image captured by the camera is high-speed motion), if the frame rate is low, the problem of blurred display is particularly significant, so the brightness of the environment and the motion state of the subject are detected to avoid blurred display in night scenes and when the subject moves at high speed.

[0013] In another implementation, determining a first frame rate corresponding to the first scene based on the first scene includes: determining the first frame rate corresponding to the first scene based on the brightness of the environment in which the image is located, the motion state of the subject in the image, and a preset scene frame rate mapping table.

[0014] Through the above method, the first frame rate corresponding to the first scene can be quickly determined.

[0015] In another implementation, the method also includes: when detecting through at least one second sensor that the scene of the image captured by the camera changes from the first scene to the second scene, determining a second frame rate corresponding to the second scene based on the second scene; based on the second frame rate, displaying a focused second image frame, the second image frame being any image frame at the second frame rate.

[0016] In the above manner, when a change in the scene captured by the camera is detected, a new frame rate is re-determined, and the new frame rate is different from the frame rate corresponding to the scene before the change. This allows for dynamic adjustment of the frame rate, so that when the scene does not require a high frame rate, the frame rate is reduced to save energy consumption of electronic devices; when the scene requires a high frame rate, the frame rate is increased to ensure clarity during display.

[0017] In another implementation, the brightness of the environment in which the image captured by the camera indicated by the second scene is located is lower than the brightness of the environment in which the image captured by the camera indicated by the first scene is located, and / or the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene; the second frame rate is higher than the first frame rate.

[0018] In night scenes (the brightness of the environment in which the image is captured by the camera is low) and when the subject is in high-speed motion (the subject in the image captured by the camera is in high-speed motion), if the frame rate is low, the problem of blurred display is particularly significant. Therefore, in night scenes and / or scenes with high-speed subject motion, increase the frame rate to avoid blur.

[0019] In a second aspect, the present application provides an image focusing device, which may be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device; wherein the image focusing device may also be a chip system, and the image focusing device may execute the method executed by the electronic device in the first aspect. The functions of the image focusing device may be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. The unit may be software and / or hardware. The operations and beneficial effects performed by the image focusing device may refer to the methods and beneficial effects described in the first aspect above, and the repeated parts will not be repeated.

[0020] In a third aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the image focusing method in any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the image focusing method in any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a chip comprising a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions to execute the image focusing method in any possible implementation of the first aspect above.

[0023] In a sixth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the image focusing method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A is a schematic diagram of a picture captured by a camera provided in an embodiment of the present application;

[0025] Figure 1B is a schematic diagram of a sensor processing an image provided by an embodiment of the present application;

[0026] Figure 1C is a schematic diagram of drawing an image provided by an embodiment of the present application;

[0027] Figure 1D is a schematic diagram of another sensor processing an image provided by an embodiment of the present application;

[0028] Figure 2 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0030] Figure 4 It is a flowchart of an image focusing method based on a software structure provided in an embodiment of the present application;

[0031] Figure 5 is a flow chart of an image focusing method provided in an embodiment of the present application;

[0032] Figure 6 is a flow chart of another image focusing method provided in an embodiment of the present application;

[0033] Fig. 7A is a schematic diagram of a scene change provided in an embodiment of the present application;

[0034] Figure 7B is a schematic diagram of a shooting mode switching provided by an embodiment of the present application;

[0035] Figure 8 is a structural schematic diagram of an image focusing device provided in an embodiment of the present application;

[0036] Fig. 9 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0038] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0039] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0040] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0041] Frame rate: Also known as frame rate, the unit is frames per second (FPS). The frame rate can refer to the number of image frames displayed on the electronic device per second, or the number of image frames processed (exposure, focus, etc.) per second. When the camera application of an electronic device is opened, the screen of the electronic device will display the picture captured by the camera, such as Figure 1A As shown, the picture captured by the camera is displayed on the display screen of the electronic device. When the electronic device is moved, the viewing angle of the camera rotates, so that the picture captured by the camera also changes in the form of animation, or when the object in the picture captured by the camera moves, the picture captured by the camera also changes in the form of animation. The essence of the animation mentioned above is to display multiple frames of static image frames of continuous action on the display screen of the electronic device.

[0042] When electronic devices display at the same frame rate, the number of image frames displayed per second is the same. When users watch these static image frames, they will have a dynamic effect. Generally speaking, a frame rate of 8FPS or above will make users feel that the playback effect is smooth. The standard for movies is 24FPS, and it can be higher.

[0043] AF scheduling frame rate: AF scheduling frame rate refers to the frequency of scheduling autofocus, that is, the number of times autofocus is scheduled per second. The number of times autofocus is scheduled per second can be the same as the number of image frames processed (exposure, focus, etc.) per second. For example, if the electronic device processes 30 image frames, the number of times the electronic device schedules AF per second is 30.

[0044] Different sensors have different AF scheduling processes. The AF scheduling processes of two sensors are introduced below.

[0045] RGB sensor (RGBSensor): When the RGB sensor processes the image frame, it schedules AF during drawing to focus the image. Figure 1B As shown, Figure 1B The portion marked with 101 is the focus frame 101. That is, the RGB sensor can simultaneously draw and focus the image frame. The image frame is drawn in rows from top to bottom. Figure 1B Each line in is a line of the image frame. Figure 1C As shown, Figure 1C As shown on the left side of the figure, draw each row from top to bottom to get the following Figure 1C The image frame shown on the right. Figure 1BThe time of one frame is the interval between the time when the first line of the left image frame is read out and the time when the first line of the right image frame is read out. If the frame rate is 20FPS, the time of one frame is 0.05 seconds, that is, the image is completed within 0.05 seconds. Figure 1B The processing of the left (or right) image frame shown is that AF scheduling is performed every 0.05 seconds.

[0046] The characteristic of this RGBSensor is that the starting drawing time for any row in the image frame is different. This is because the RGBSensor loads many parameters when drawing each row, so the RGBSensor consumes more time when drawing. Compared with the depth sensor introduced below, the RGBSensor consumes more time when drawing.

[0047] Depth Sensor: When processing an image frame, Depth Sensor schedules AF after drawing is completed to focus the image. Figure 1D As shown, Figure 1D The starting drawing time of all rows of the image frame is the same time point. The drawing of the image frame is also drawn from the top at least row by row. Figure 1D Each line in is a line of an image frame. For details, please refer to the above introduction, and this application will not go into details here. Figure 1D The time of one frame is also the interval between the time when the first line in the left image frame is completed to be read out and the time when the first line in the right image frame is completed to be read out.

[0048] The characteristic of this DepthSensor is that the starting drawing time for any row in the image frame is the same. This is because compared with the above-mentioned RGBSensor, this DepthSensor loads fewer parameters, so the time consumed by this DepthSensor during drawing is relatively less.

[0049] In order to improve the frame rate and thus improve the AF scheduling frame rate, the present application provides an image focusing method and related devices. In a specific implementation, the above-mentioned image focusing method can be performed by an electronic device 100. Among them, the electronic device 100 can be an electronic device including a first sensor and a camera. Specifically, the electronic device can be a smart phone, an outward folding screen mobile phone, a tablet computer, a laptop computer, etc., but is not limited thereto.

[0050] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0051] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0052] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0054] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0055] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system. The processor 110 calls the instructions or data stored in the memory, so that the electronic device 100 executes the image focusing method executed by the electronic device in the following method embodiment.

[0056] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0057] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0058] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0059] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0060] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0061] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0062] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0063] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0064] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0065] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0066] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 may be an outward folding screen, that is, a display screen that is folded outward.

[0067] The electronic device 100 can realize the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The camera 193 may include a front camera and a rear camera, the front camera is located in the display area of ​​the screen, and the rear camera is located in the back area of ​​the screen. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs.

[0068] NPU is a neural-network (NN) computing processor. It can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0069] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0070] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as audio data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0071] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0072] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0073] Speaker 170A, also known as "speaker", is used to convert audio electrical signals into sound signals. Receiver 170B, also known as "earpiece", is used to convert audio electrical signals into sound signals. Microphone 170C, also known as "microphone" and "microphone", is used to convert sound signals into electrical signals. Headphone interface 170D is used to connect wired headphones. Pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, pressure sensor 180A can be set on display screen 194. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Air pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes Hall sensor. Acceleration sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Proximity light sensor 180G can include, for example, light emitting diode (LED) and light detector. Ambient light sensor 180L is used to sense ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0074] In addition, an operating system is running on the above components. For example, operating systems such as iOS and Android. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0075] Figure 3It is a software structure block diagram of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiment of the present application, the operating system (taking the Android system, the Android system running on the AP as an example) can be divided into three layers, from top to bottom, namely, the application layer (application, APP), the application framework layer (framework, FWK) and the hardware abstraction layer (hardware abstraction layer, HAL).

[0076] The application layer may include a series of application packages. Figure 3 As shown, the application package may include applications such as a camera. In the embodiment of the present application, the camera refers to a camera application. The camera application may include a display interface module (for displaying an interface related to the camera application), a dynamic frame rate adjustment module, a camera interface module (which may be called CameraApi2Module), etc.

[0077] The display interface module is a module in the camera application that is used to display the images captured by the camera. It is a medium interface for interaction and information exchange between the camera application and the user. The camera interface source code is parsed and rendered on the electronic device and finally presented as content that the user can recognize. The display interface module can also display the shooting button, etc.

[0078] CameraApi2Module is a module in the camera application, which is used to manage and control the camera application and provide the interface of the camera application.

[0079] The dynamic frame rate adjustment module is used to determine the frame rates corresponding to different scenes.

[0080] The application framework layer provides application developers with an application programming interface (API) framework and various services and management tools for accessing core functions, including interface management, data access, application layer messaging, application package management, phone management, location management, etc. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a camera service (CameraService) and the like.

[0081] CameraService is responsible for the startup process scheduling of the camera application, the creation and management of processes, the creation and management of windows, etc.

[0082] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. Figure 3 As shown, the hardware abstraction layer includes an RGB sensor, a first sensor, and a second sensor. The RGB sensor can capture image frames (which can be understood as image frames captured by a camera, and the RGB sensor is a part of the camera); the first sensor is used to draw and focus the image; the second sensor is used to sense the camera environment and interact with the dynamic frame rate adjustment module.

[0083] Based on the above software structure, the present application embodiment provides a flowchart of an image focusing method based on a software structure. Figure 4 As shown, the image focusing method includes the following steps S401 to S408. Among them:

[0084] S401. The second sensor sends the brightness of the environment in which the image captured by the camera is located and the motion state of the subject in the image captured by the camera to the dynamic frame rate adjustment module.

[0085] The second sensor may include multiple modules, such as an ambient light sensor for detecting ambient brightness, and a module for detecting the motion state of the subject, etc. It should be noted that the second sensor may also include other sensors, which are not limited in the present application.

[0086] S402: The dynamic frame rate adjustment module determines a first frame rate corresponding to a first scene based on the brightness of the environment in which the image is located and the motion state of the subject in the image.

[0087] Among them, the first scene is determined based on the brightness of the environment in which the image captured by the camera sent by the second sensor is located and the motion state of the subject in the image captured by the camera. In addition to determining the frame rate based on the brightness of the environment in which the image is located and the motion state of the subject, the dynamic frame rate adjustment module can also be determined based on other parameters, such as the subject focus difference, etc., which is not limited in this application. The subject focus difference refers to: after focusing on a certain image frame, the focus determined is The deviation from the actual position of the subject.

[0088] The dynamic frame rate adjustment module can be a first frame rate determined based on a preset mapping, or a first frame rate calculated based on the motion state of the subject in the brightness image of the environment the image is in. The method for determining the first frame rate can be found below, and this application will not be repeated here.

[0089] S403: The dynamic frame rate adjustment module sends the first frame rate to the RGB sensor.

[0090] Among them, the RGB sensor can be understood as a sensor in a camera of an electronic device, and the camera captured image in this application can be an image captured by the RGB sensor in the camera. For the convenience of description, this application uses the camera to capture the image for description. The RGB sensor can be referred to in the description in the previous text, and this application will not be repeated here.

[0091] S404: The dynamic frame rate adjustment module sends the first frame rate to the first sensor.

[0092] The first sensor may be a depth sensor, which may be used to draw and focus the image frame.

[0093] S405 . The RGB sensor sends image frames to the first sensor at a first frame rate.

[0094] The first frame rate is used to indicate the number of image frames sent by the RGB sensor to the first sensor per second. For example, if the first frame rate is 30 FPS, the RGB sensor sends 30 image frames to the first sensor per second.

[0095] S406: The first sensor draws and focuses the image frame.

[0096] The first sensor draws the image frame, which may be the first sensor exposing the image frame. The first sensor focuses the image frame, which may be the first sensor scheduling AF to focus the image frame. Specifically, the first sensor scheduling AF to determine the focus frame of the subject in the image frame to achieve focusing on the image frame.

[0097] In a possible implementation, the first sensor is a depth sensor. The process of the depth sensor drawing and focusing the image frame can be seen in Figure 1D shown.

[0098] The time for the first sensor to draw and focus on the image frame does not exceed 1 / the first frame rate. For example, if the first frame rate is 30 FPS, the time for the first sensor to draw and focus on the image frame does not exceed 1 / 30 FPS, that is, 0.033 seconds.

[0099] S407: The first sensor sends image frames to the display interface module at a first frame rate.

[0100] The first frame rate is used to indicate the number of image frames sent by the first sensor to the display interface module per second. For example, if the first frame rate is 30 FPS, the first sensor sends 30 image frames to the display interface module per second. The image frames sent are image frames after focusing.

[0101] S408: The display interface module displays the image frames based on the first frame rate.

[0102] The first frame rate is used to indicate the number of image frames displayed by the display interface module per second. For example, if the first frame rate is 30 FPS, the display interface module displays 30 image frames per second. The image frames displayed by the display interface module are image frames after focusing.

[0103] Based on the above, an image focusing method provided by an embodiment of the present application is further described in detail below. Figure 5 As shown, the image focusing method includes the following steps S501 to S503. Figure 5 The execution subject of the method shown can be the electronic device mentioned above. Or Figure 5 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 5 The electronic device is used as an example to illustrate the method. The electronic device includes a first sensor and a camera.

[0104] S501, the electronic device draws the first image frame through the first sensor; the starting drawing time of all rows of the first image frame is the same, and the first image frame is any image frame under the first frame rate; the first frame rate is the frame rate corresponding to the first scene, and the first scene is the scene of the image captured by the camera; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to simultaneously draw all rows of any image frame.

[0105] The drawing includes but is not limited to exposure. The same start drawing time for all rows of the first image frame may be the same start exposure time for all rows of the first image frame. The same end drawing time for all rows of the first image frame may also be the same. For example Figure 1D As shown, the Figure 1D The drawing time in is the time for drawing a certain image frame, and the drawing time of all image frames under the first frame rate is the same. The first frame rate is the number of images processed by the electronic device per second, and the image processing includes drawing, reading out, and mobilizing AF to focus on the image frame. The reading out is to read out each line of the image frame one by one after the drawing is completed, and the reading out can be to read out each line of the graphic frame to the storage area. The image frame is an image processed by the electronic device under one frame.

[0106] In a possible embodiment, the first sensor may be a depth sensor. Based on the characteristics of the depth sensor, after the first image frame is drawn, the subject in the first image frame is focused.

[0107] Optionally, the first sensor may be a sensor other than a depth sensor, such as another sensor capable of global exposure, and the global exposure sensor may start exposing all rows of the image frame at the same time and simultaneously terminate the exposure of all rows of the image frame. Alternatively, the first sensor focuses on the subject in the first image frame after the first image frame is drawn and all rows corresponding to the first image frame are read out to the storage area.

[0108] Optionally, the scene of the image captured by the camera can be the scene of the image captured by the RGB sensor in the camera. Different scenes correspond to different frame rates. The first scene may include but is not limited to the brightness of the environment in which the image captured by the camera is located and the motion state of the subject in the image captured by the camera.

[0109] In the above method, the depth sensor is a sensor that draws all lines of any image frame at the same time. The characteristic of this sensor is that the image frame is focused after the image frame is drawn. Since the time required for the depth sensor to draw the image frame is small, the use of the depth sensor can increase the frame rate and improve the AF scheduling frame rate.

[0110] The first frame rate may be a fixed frame rate, for example, the first frame rate is fixed to 30 FPS, or the frame rate may be a frame rate determined based on different scenarios, for example, after the user clicks to enter the camera application, the electronic device automatically determines the first frame rate based on the current scenario. The method for determining the first frame rate is further described below.

[0111] Method 1: The electronic device automatically determines the first frame rate based on the current scene.

[0112] In a possible embodiment, the electronic device determines a first scene of an image captured by a camera based on at least one second sensor, the first scene including the brightness of an environment in which the image captured by the camera is located, and the motion state of a subject in the image captured by the camera; and determines a first frame rate corresponding to the first scene based on the first scene.

[0113] The first scene includes the brightness of the environment in which the image captured by the camera is located, and the motion state of the subject in the image captured by the camera. The first scene can be the following scenes: scene A: high brightness-subject still scene, scene B: medium brightness-subject still scene, scene C: dim-subject still scene, scene D: high brightness-subject slow motion scene, scene E: high brightness-subject fast motion scene, etc. This application will not be elaborated here.

[0114] In the above manner, different first frame rates are determined for different scenes of the image captured by the camera, so that when the scene does not require a high frame rate, the frame rate is reduced to save energy consumption of the electronic device; when the scene requires a high frame rate, the frame rate is increased to ensure clarity during display.

[0115] In a possible embodiment, the second sensor includes an ambient light sensor for detecting ambient brightness and a subject motion detection module for detecting the motion state of the subject; based on at least one second sensor, determining that the scene of the image captured by the camera is the first scene includes: the electronic device determines the brightness of the environment in which the image captured by the camera is located based on the ambient light sensor; based on the subject motion detection module, determining the motion state of the subject in the image captured by the camera.

[0116] Among them, the ambient light sensor is a sensor that can sense the intensity of ambient light. The ambient light sensor can convert the intensity of ambient light into an electrical signal output, thereby realizing the detection of ambient light. The working principle of the ambient light sensor is based on photoelectric response, that is, when light shines on the surface of an object, it excites the electrons on the surface of the object, causing them to separate from atoms to form their own electrons. When the self-owned electrons recombine with the atoms on the surface of the object, they release energy and generate electrical signals. The motion state module of the detection subject can be based on infrared work, for example, infrared rays are emitted from a mobile phone to the subject at a first time point, and infrared rays are sent to the subject again at a second time point. Based on the time interval between the first time point and the second time point, and the distance between the subjects at the two time points, the motion state of the subject is determined.

[0117] The brightness of the environment in which the camera captures the image determined by the ambient light sensor can be divided into the following three types: high brightness, medium brightness and dim. It should be noted that the brightness of the environment can be divided into more than the above three brightnesses, which are only for example and are not limited in this application.

[0118] Through the above method, in night scenes (the brightness of the environment in which the image captured by the camera is located is low) and when the subject moves at high speed (the motion state of the subject in the image captured by the camera is high-speed motion), if the frame rate is low, the problem of blurred display is particularly significant, so the brightness of the environment and the motion state of the subject are detected to avoid blurred display in night scenes and when the subject moves at high speed.

[0119] In a possible embodiment, determining a first frame rate corresponding to the first scene based on the first scene includes: determining the first frame rate corresponding to the first scene based on the brightness of the environment in which the image is located, the motion state of the subject in the image, and a preset scene frame rate mapping table.

[0120] The preset scene frame rate mapping table includes the brightness of the environment in which the image is located and the motion state of the subject in the image, as well as different frame rates corresponding to the brightness of the environment in which different images are located and the motion state of the subject in different images.

[0121] Exemplarily, the scene frame rate mapping table is shown in the following table:

[0122] The brightness of the image's environment The motion of the subject in the image Frame rate Highlight Still or slow motion Frame rateA Highlight High-speed movement Frame rate B Zhongliang Still or slow motion Frame rate C Zhongliang High-speed movement Frame rate D Dark Still or slow motion Frame rate Dark High-speed movement Frame rate

[0123] If the ambient light sensor determines that the brightness of the image captured by the camera is dim, and the subject motion detection module determines that the motion state of the subject in the image captured by the camera is high-speed motion, the first frame rate is frame rate F. Similarly, if the brightness of the image captured by the camera is high, and the subject motion detection module determines that the motion state of the subject in the image captured by the camera is stationary or slow motion, the first frame rate is frame rate A.

[0124] Since usually when the motion state of the subject in the image is high-speed motion and / or the brightness of the environment in which the image is located is dim, a higher frame rate is required to ensure clear display; while when the motion state of the subject in the image is stationary or slow motion and / or the brightness of the environment in which the image is located is high, a very high frame rate is not required to ensure clear display. However, in order to save energy consumption of electronic devices, a lower frame rate is determined when the motion state of the subject in the image is stationary or slow motion and / or the brightness of the environment in which the image is located is high. That is to say, the frame rate A in the above example is less than the frame rate F, for example, the frame rate A is 20FPS and the frame rate B is 30FPS.

[0125] Method 2: The electronic device determines the first frame rate based on the shooting mode selected by the user.

[0126] Among them, the shooting mode is the shooting mode selected by the user through the display screen of the electronic device, and the shooting mode includes but is not limited to the following shooting modes: night scene shooting mode, portrait shooting mode, large aperture shooting mode, photo mode, video recording mode and professional mode.

[0127] In a possible embodiment, the electronic device determines the first frame rate based on the shooting mode selected by the user. Specifically, the electronic device determines the first frame rate based on the shooting mode selected by the user and a mapping relationship between the shooting mode and the frame rate of the shooting mode pre-stored in the electronic device.

[0128] Exemplarily, the shooting mode includes the following shooting modes: night scene shooting mode, portrait shooting mode, large aperture shooting mode, photo shooting mode, and the shooting mode frame rate mapping relationship is: night scene shooting mode-30FPS, portrait shooting mode-25FPS, large aperture shooting mode-35FPS, photo shooting mode-20FPS. If the shooting mode selected by the user is the night scene shooting mode, the first frame rate is 30FPS. Similarly, if the shooting mode selected by the user is the large aperture shooting mode, the first frame rate is 35FPS. The same is true for other shooting modes, and this application will not be repeated here.

[0129] S502, the electronic device focuses on the subject in the first image frame to obtain a focused first image frame; drawing the first image and focusing on the subject in the first image are completed within a first time period, and the first time period is the time interval between two adjacent image frames displayed at a first frame rate.

[0130] The subject in the first image may be a face in the image, etc. The first time period is determined based on the first frequency, the first time period = 1 / first frequency, for example, if the first frequency is 30 FPS, the first time period is 0.033 seconds, if the first frequency is 20 FPS, the first time period is 0.05 seconds. The first time period may also be as follows: Figure 1D The frame time shown is the time interval between the time when the first line of the previous image frame is read out and the time when the first line of the next image frame is read out in two adjacent image frames. Alternatively, the first time period is the time interval between the time when each line of the previous image frame starts to be drawn and the time when each line of the next image frame starts to be drawn in two adjacent image frames.

[0131] S503: The electronic device displays a focused first image frame based on a first frame rate.

[0132] The electronic device displays the first image frame on a display screen, and then displays a next image frame on the display screen after a first time period at a first frequency.

[0133] In the above manner, all rows of any image frame are drawn simultaneously by the first sensor, thereby reducing the time for drawing any frame. Since the time for drawing any image frame is reduced, a higher frame rate (FPS) can be set. Since the higher the frame rate, the more image frames are drawn in one second, and autofocus needs to be performed after drawing each image frame, when the frame rate is increased, the AF scheduling frame rate is also increased.

[0134] The frame rate in the above-mentioned can be changed from the first frame rate to the second frame rate according to the change of the scene. That is to say, the frame rate in the present application can be dynamically adjusted. The following is a further introduction to the embodiment of dynamically adjusting the frame rate, as follows Figure 6As shown, Figure 6 A schematic diagram of another image focusing method provided in an embodiment of the present application.

[0135] S601. The electronic device determines a first scene of an image captured by a camera based on at least one second sensor, where the first scene includes brightness of an environment in which the image captured by the camera is located and a motion state of a subject in the image captured by the camera.

[0136] The step S601 can refer to the introduction of the above step S501, and this application will not elaborate on it here.

[0137] S602: The electronic device determines a first frame rate corresponding to the first scene based on the first scene.

[0138] The step S602 can refer to the introduction of the above step S501, and this application will not elaborate on it here.

[0139] S603, the electronic device draws the first image frame through the first sensor; the starting drawing time of all rows of the first image frame is the same, and the first image frame is any image frame under the first frame rate; the first frame rate is the frame rate corresponding to the first scene, and the first scene is the scene of the image captured by the camera; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to simultaneously draw all rows of any image frame.

[0140] The step S603 can refer to the introduction of the above step S501, and this application will not elaborate on it here.

[0141] S604, the electronic device focuses on the subject in the first image frame to obtain a focused first image frame; drawing the first image and focusing on the subject in the first image are completed within a first time period, and the first time period is the time interval between two adjacent image frames displayed at a first frame rate.

[0142] The step S604 can refer to the introduction of the above step S502, and this application will not elaborate on it here.

[0143] S605: The electronic device displays a focused first image frame based on the first frame rate.

[0144] The step S605 can refer to the introduction of the above step S503, and this application will not elaborate on it here.

[0145] S606: When the electronic device detects through at least one second sensor that the scene of the image captured by the camera changes from the first scene to the second scene, determine a second frame rate corresponding to the second scene based on the second scene.

[0146] Among them, determining the second frame rate corresponding to the second scene according to the second scene can refer to the description of determining the first frame rate corresponding to the first scene according to the first scene in the previous text, and this application will not repeat it here. Detecting that the scene of the image captured by the camera changes from the first scene to the second scene through at least one second sensor can be: the second sensor detects a change in the brightness of the environment in which the image captured by the camera is located, or a change in the motion state of the subject in the image captured by the camera.

[0147] In a possible embodiment, the electronic device detects through at least one second sensor that the scene of the image captured by the camera changes from a first scene to a second scene, including: when the second sensor detects that the change in brightness of the environment in which the image captured by the camera is located is greater than a first threshold, or when the second sensor detects that the change in the motion state of the subject in the image captured by the camera is greater than a second threshold, it is considered that the scene of the image captured by the camera changes from the first scene to the second scene.

[0148] The ambient light sensor in the second sensor may detect the change in brightness of the environment in which the image captured by the camera is located, and the subject motion detection module in the second sensor may detect the change in the motion state of the subject in the image captured by the camera. The change in the motion state may be a change in the subject motion speed.

[0149] For example, Fig. 7A As shown, Fig. 7A In the scene on the left side, the light 701 is on, the current scene is scene A (high light - still or slow motion), and the current frame rate is frame rate A. When the light 701 is turned off, the light source in the room becomes weaker, and the current scene is scene F (dim - still or slow motion). The electronic device detects through the second sensor that the current scene changes from scene A to scene F, and the electronic device determines the frame rate F corresponding to scene F based on scene F.

[0150] In a possible embodiment, the brightness of the environment in which the image captured by the camera indicated by the second scene is located is lower than the brightness of the environment in which the image captured by the camera indicated by the first scene is located, and / or the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene; the second frame rate is higher than the first frame rate.

[0151] That is to say, the brightness of the environment in which the image captured by the camera is located is lower than the brightness of the environment in which the image captured by the camera is located as indicated by the first scene, the motion state of the subject in the image captured by the camera indicated by the second scene is the same as the motion state of the subject in the image captured by the camera indicated by the first scene, and the second frame rate is higher than the first frame rate; the brightness of the environment in which the image captured by the camera is located is lower than the brightness of the environment in which the image captured by the camera is located as indicated by the first scene, the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene, and the second frame rate is higher than the first frame rate; the brightness of the environment in which the image captured by the camera is located is the same as the brightness of the environment in which the image captured by the camera is located as indicated by the first scene, the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene, and the second frame rate is higher than the first frame rate.

[0152] In night scenes (the brightness of the environment in which the image is captured by the camera is low) and / or the subject is moving at high speed (the subject in the image captured by the camera is moving at high speed), if the frame rate is low, the problem of blurred display is particularly significant. Therefore, in night scenes and / or scenes with high-speed subject motion, the frame rate is increased to avoid blur.

[0153] On the contrary, the brightness of the environment in which the image captured by the camera indicated by the second scene is located is higher than the brightness of the environment in which the image captured by the camera indicated by the first scene is located, and / or the motion state of the subject in the image captured by the camera indicated by the second scene is lower than the motion state of the subject in the image captured by the camera indicated by the first scene; the second frame rate is lower than the first frame rate.

[0154] When the brightness of the environment in which the image captured by the camera is located is high, and / or the motion state of the subject in the image captured by the camera is stationary or in a slow motion state, the frame rate is reduced, thereby saving energy consumption of the electronic device.

[0155] The above is that the electronic device automatically changes the frame rate after detecting a scene change. In addition, optionally, the electronic device changes the shooting mode in response to a user clicking a mode button displayed on the electronic device; based on the changed shooting mode and a preset mapping table between shooting modes and frame rates, the second frame rate is determined.

[0156] That is to say, after the shooting mode of the electronic device is changed, the frame rate is also changed accordingly. The frame rates corresponding to two different shooting modes can be the same or different. For example, the electronic device includes the following three shooting modes: shooting mode A, shooting mode B, and shooting mode C. The frame rates corresponding to shooting mode A and shooting mode B are different, and the frame rates corresponding to shooting mode A and shooting mode C are the same.

[0157] For example, Figure 7B As shown, the original shooting mode of the electronic device is photo shooting. After the user clicks on the large aperture on the interface of the camera application, the shooting mode of the electronic device is changed from the photo shooting mode to the large aperture mode. The frame rate corresponding to the photo shooting mode is 20FPS. In the mapping table between the shooting mode and the frame rate, the frame rate corresponding to the large aperture mode is 30FPS. Then the electronic device changes the frame rate from 20FPS (first frame rate) to 30FPS (second frame rate).

[0158] S607: The electronic device displays a focused second image frame based on a second frame rate, where the second image frame is any image frame at the second frame rate.

[0159] The step S605 can refer to the introduction of the above step S503, and this application will not elaborate on it here.

[0160] See also Figure 8 , Figure 8 A structural schematic diagram of an image focusing device 800 provided in an embodiment of the present application. Figure 8 The image focusing device shown can be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device. The electronic device includes a display screen, and the display screen includes a main screen and a back screen, and the main screen and the back screen are two display areas of the same display screen; when the electronic device is folded, the main screen and the back screen face opposite directions; when the electronic device is unfolded, the main screen and the back screen face the same direction, and the display area is the entire display screen; the electronic device also includes a front camera and a rear camera, and the front camera and the rear camera face opposite directions, the front camera and the main screen face the same direction, and the rear camera and the back screen face the same direction. Figure 8 The image focusing device shown may include a processing unit 801 and a display unit 802. Wherein:

[0161] The processing unit 801 is used to draw a first image frame through a first sensor; the starting drawing time of all rows of the first image frame is the same, and the first image frame is any image frame at a first frame rate; the first frame rate is a frame rate corresponding to a first scene, and the first scene is a scene of an image captured by a camera; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to draw all rows of any image frame at the same time;

[0162] The processing unit 801 is further configured to focus on the subject in the first image frame to obtain a focused first image frame; drawing the first image and focusing on the subject in the first image are completed within a first time period, and the first time period is a time interval between two adjacent image frames displayed at a first frame rate;

[0163] The display unit 802 is configured to display a focused first image frame based on a first frame rate.

[0164] In a possible implementation, the first sensor is a depth sensor; the processing unit 801 is further configured to focus on a subject in the first image frame after drawing the first image frame.

[0165] In one possible implementation, the processing unit 801 is further used to determine a first scene of an image captured by the camera based on at least one second sensor, the first scene including the brightness of an environment in which the image captured by the camera is located, and the motion state of a subject in the image captured by the camera; the processing unit 801 is further used to determine a first frame rate corresponding to the first scene based on the first scene.

[0166] In one possible implementation, the second sensor includes an ambient light sensor for detecting ambient brightness and a subject motion detection module for detecting the motion state of the subject; the processing unit 801 is also used to determine the brightness of the environment in which the image captured by the camera is located based on the ambient light sensor; and determine the motion state of the subject in the image captured by the camera based on the subject motion detection module.

[0167] In a possible implementation, the processing unit 801 is further configured to determine a first frame rate corresponding to the first scene based on the brightness of the environment in which the image is located, the motion state of the subject in the image, and a preset scene frame rate mapping table.

[0168] In one possible implementation, the processing unit 801 is further used to determine a second frame rate corresponding to the second scene based on the second scene when detecting through at least one second sensor that the scene of the image captured by the camera changes from a first scene to a second scene; the display unit 802 is further used to display a focused second image frame based on the second frame rate, where the second image frame is any image frame at the second frame rate.

[0169] In one possible implementation, the brightness of the environment in which the image captured by the camera indicated by the second scene is located is lower than the brightness of the environment in which the image captured by the camera indicated by the first scene is located, and / or the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene; the second frame rate is higher than the first frame rate.

[0170] For the case where the image focusing device can be a chip or a chip system, see Fig. 9 Schematic diagram of the chip structure shown. Fig. 9The chip 900 shown includes a processor 901 and an interface 902. Optionally, it may also include a memory 903. The number of the processor 901 may be one or more, and the number of the interface 902 may be multiple.

[0171] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0172] The interface 902 is used to receive or output signals;

[0173] The processor 901 is used to execute data processing operations of the electronic device.

[0174] It is understandable that some optional features in the embodiments of the present application may be implemented independently in certain scenarios without relying on other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects. They may also be combined with other features according to needs in certain scenarios. Accordingly, the image focusing device provided in the embodiments of the present application may also realize these features or functions accordingly, which will not be elaborated here.

[0175] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0176] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0177] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0178] The present application also provides a computer program product. When the computer program product is executed on a computer, the computer can implement the functions of any of the above method embodiments.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0180] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image focusing method, characterized in that: The method is applied to an electronic device, the electronic device includes a first sensor and a camera, and the method includes: The first image frame is drawn by the first sensor; the starting drawing time of all rows of the first image frame is the same, and the first image frame is any image frame at a first frame rate; the first frame rate is a frame rate corresponding to a first scene, and the first scene is a scene of an image captured by the camera; the first frame rate is the number of images processed by the electronic device per second; the first sensor is used to draw all rows of any image frame at the same time; After the first image frame is drawn, focusing on a subject in the first image frame to obtain a focused first image frame; drawing the first image and focusing on the subject in the first image are completed within a first time period, and the first time period is a time interval between two adjacent image frames displayed at the first frame rate; Based on the first frame rate, the focused first image frame is displayed.

2. The method according to claim 1, characterized in that: The first sensor is a depth sensor.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determining a first scene of the image captured by the camera based on at least one second sensor, the first scene including brightness of an environment in which the image captured by the camera is located, and a motion state of a subject in the image captured by the camera; A first frame rate corresponding to the first scene is determined based on the first scene.

4. The method according to claim 3, characterized in that The second sensor includes an ambient light sensor for detecting ambient brightness and a subject motion detection module for detecting a motion state of a subject; The step of determining, based on at least one second sensor, that the scene of the image captured by the camera is the first scene includes: Based on the ambient light sensor, determine the brightness of the environment in which the image captured by the camera is located; Based on the subject motion detection module, a motion state of a subject in an image captured by the camera is determined.

5. The method according to claim 4, characterized in that The determining, based on the first scene, a first frame rate corresponding to the first scene includes: A first frame rate corresponding to the first scene is determined based on the brightness of the environment in which the image is located, the motion state of the subject in the image, and a preset scene frame rate mapping table.

6. The method according to claim 3, characterized in that The method further comprises: When detecting, by the at least one second sensor, that the scene of the image captured by the camera is changed from the first scene to a second scene, determining a second frame rate corresponding to the second scene based on the second scene; Based on the second frame rate, the focused second image frame is displayed, where the second image frame is any image frame at the second frame rate.

7. The method according to claim 6, characterized in that The brightness of the environment in which the image captured by the camera indicated by the second scene is located is lower than the brightness of the environment in which the image captured by the camera indicated by the first scene is located, and / or the motion state of the subject in the image captured by the camera indicated by the second scene is higher than the motion state of the subject in the image captured by the camera indicated by the first scene; the second frame rate is higher than the first frame rate.

8. An image focusing device, characterized in that: The device comprises a processor, a memory, an input interface and an output interface, wherein the input interface is used to receive information from other image focusing devices outside the image focusing device, and the output interface is used to output information to other image focusing devices outside the image focusing device, and the processor calls the computer program stored in the memory so that the image focusing device executes the method as described in any one of claims 1 to 7.

9. A chip system, applied to a terminal, characterized in that: The chip system includes at least one processor and an interface, wherein the interface is used to receive instructions and transmit them to the at least one processor; the at least one processor runs the instructions so that the terminal executes the method described in any one of claims 1-7.

Images