Method for determining frame rate, electronic device and readable storage medium
By adjusting the frame rate of the image sensor to match the frame rate of the display screen, the frame drop or lag caused by frame rate mismatch in electronic devices is solved, and the user experience is improved.
Patent Information
- Application Number
- CN202310921129.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-07-24
AI Technical Summary
During the operation of the camera function of the electronic device, the image frame rate generated by the image sensor is prone to have a large difference between the frame rate and the frame rate of the display screen playback, resulting in mismatch of the frame rate, which may lead to image frame discarding or previewing stuttering, affecting the user experience.
By acquiring the first frame rate of the electronic device during the camera function and the second frame rate of the display playing image frame, adjusting the third frame rate of the image sensor generating image frames so that it is close to the frame rate of the display screen to avoid discarding or Stuck problem. The specific method includes determining the third frame rate based on the first frame rate and the second frame rate, and adjusting the frame rate when the image sensor generates an image frame to match the frame rate of the display screen.
It effectively avoids image frame discarding or preview lag during the operation of the camera function of the electronic device, improves the user experience, and ensures the matching and stability of the frame rate.
Smart Images

Figure CN117714897B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of terminals, and in particular to a method for determining a frame rate, an electronic device, and a readable storage medium. Background Art
[0002] When the camera application of an electronic device is used by the user, it involves the process of the image sensor generating image frames and the display screen playing the image frames. The image sensor and the display screen are two hardware resources from different sources in the electronic device. The two have different implementation accuracy when performing operations based on the frame rate used by the electronic device when implementing the camera function. Therefore, in some usage scenarios of the camera application, such as photo preview, video preview, and preview scenes during video recording, there will be a large difference between the frame rate of the image sensor generating image frames and the frame rate of the display screen playing the image frames. When the frame rate of the image sensor generating image frames is high, the image frame discarding problem may occur. When the frame rate of the image sensor generating image frames is low, the display screen will play repeated image frames, which may cause preview freezes and affect the user experience. Summary of the invention
[0003] The present application provides a method for determining a frame rate, an electronic device, and a readable storage medium. The solution provided by the present application can avoid discarding image frames or preview freeze problems as much as possible during the operation of the camera function of the electronic device, thereby improving the user experience.
[0004] In order to achieve the above purpose, this application adopts the following technical solutions:
[0005] In a first aspect, a method for determining a frame rate is provided, which is applied to an electronic device, wherein the electronic device includes a display screen and an image sensor. The method includes: during the operation of a camera function of the electronic device, obtaining a first frame rate and a second frame rate, wherein the first frame rate is a frame rate used by the electronic device when implementing the camera function, the second frame rate is a frame rate at which the display screen plays image frames, and the tearing effect signal frequency corresponding to the second frame rate is within a specification error range of the display screen; determining a third frame rate based on the first frame rate and the second frame rate, wherein the third frame rate is a frame rate at which the image sensor generates image frames; and generating image frames based on the third frame rate.
[0006] Based on the above scheme, the third frame rate of the image frame generated by the image sensor can be adjusted based on the second frame rate of the image frame actually played by the display screen and the first frame rate used by the electronic device when implementing the camera function. The frame rate of the image frame generated by the image sensor can be made as close as possible to the frame rate of the image frame played by the display screen, so as to avoid the problem of discarding image frames or preview jamming during the operation of the camera function of the electronic device, thereby improving the user experience.
[0007] In some possible implementations, determining a third frame rate based on the first frame rate and the second frame rate includes: determining a fourth frame rate based on the first frame rate and the second frame rate, the values of the fourth frame rate and the second frame rate are equal, and when the absolute value of the difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold, determining the fourth frame rate to be the third frame rate.
[0008] Based on the above scheme, if the theoretical frame rate calculated by the first frame rate and the second frame rate, that is, the fourth frame rate, does not exceed the range that the image sensor can adjust, for example, when the absolute value of the difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold, it is determined that the fourth frame rate is the third frame rate, that is, the image sensor generates image frames based on the third frame rate, so that the frame rate of the image frames actually generated by the image sensor can be close to the frame rate of the image frames played on the display screen within the accuracy range of the image sensor, thereby minimizing the problem of discarded image frames or preview freezes during the operation of the camera function of the electronic device, thereby improving the user experience.
[0009] In some possible implementations, determining a third frame rate based on the first frame rate and the second frame rate includes: determining a fourth frame rate based on the first frame rate and the second frame rate, the values of the second frame rate and the fourth frame rate are equal, and when the absolute value of the difference between the fourth frame rate and the first frame rate is greater than a preset threshold; determining the third frame rate based on the first frame rate and the preset threshold.
[0010] Based on the above scheme, if the theoretical frame rate calculated by the first frame rate and the second frame rate, that is, the fourth frame rate, exceeds the range that the image sensor can adjust, for example, when the absolute value of the difference between the fourth frame rate and the first frame rate is greater than the preset threshold, the image sensor cannot adjust the frame rate of the generated image frame to the fourth frame rate, and the frame rate of the image sensor generating the image frame can take the boundary value. For example, the first frame rate is 30HZ, and the calculated fourth frame rate is 30.6HZ. The image sensor can adjust the frame rate to ±0.30HZ based on the first frame rate, that is, the preset threshold is 0.30HZ, and the image sensor adjusts the frame rate to 29.70HZ~30.30HZ. At this time, the third frame rate can be 30HZ+0.30HZ=30.3HZ instead of 30.6HZ. In some possible implementations, obtaining the second frame rate includes: obtaining the refresh rate of the display screen; and obtaining the second frame rate based on the refresh rate.
[0011] Based on the above scheme, the second frame rate obtained can be determined based on the average refresh rate of the display screen over a period of time, avoiding the situation where the second frame rate obtained is the refresh rate of the display screen at a specific moment, resulting in the determined third frame rate only matching the frame rate of the image frames played by the display screen at a specific moment, and cannot be applied to other moments, affecting the user experience.
[0012] In some possible implementations, the software architecture of the electronic device includes an application framework layer, a first process runs in the application framework layer, and obtaining the refresh rate of the display screen includes: the first process requests the refresh rate of the display screen from the display screen; the first process receives the refresh rates of multiple display screens from the display screen.
[0013] In some possible implementations, the software architecture of the electronic device includes an application framework layer and a hardware abstraction layer, a first process runs in the application framework layer, and a second process runs in the hardware abstraction layer, and the second frame rate is obtained based on the refresh rate, including: the first process determines the second frame rate based on the refresh rates of the multiple display screens; the first process indicates the second frame rate to the second process; the second process obtains the second frame rate according to the instruction of the first process.
[0014] In some possible implementations, the third frame rate is determined based on the first frame rate and the second frame rate, including: the second process determines the tearing effect signal frequency corresponding to the second frame rate based on the second frame rate; the second process determines the third frame rate based on the tearing effect signal frequency and the first frame rate, or the second process determines the third frame rate based on the tearing effect signal frequency, the first frame rate and a preset threshold.
[0015] In some possible implementations, the method further includes: the second process indicates a third frame rate to the image sensor; and generating image frames based on the third frame rate includes: the image sensor generates image frames according to the third frame rate indicated by the second process.
[0016] In some possible implementations, the method further includes: the image sensor sending feedback information to the second process, where the feedback information indicates that the image sensor generates the image frame according to the third frame rate.
[0017] In some possible implementations, the method further includes: the second process receiving feedback information; and the second process recording at least one of the following according to the feedback information: the feedback information and the second frame rate.
[0018] Based on the above scheme, when the image sensor feedback generates image frames according to the third frame rate, some key data can be recorded, such as feedback information and the second frame rate, so that the behavior of the electronic device can be explained based on the key data later, or if a problem occurs later, the problem can be located based on the key data analysis.
[0019] In a second aspect, an electronic device is provided, including a display screen and an image sensor, wherein the display screen is used to display image frames generated by the image sensor, and the electronic device is used to execute the steps of processing in the first aspect or any one of the methods in the first aspect.
[0020] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the steps of processing in the first aspect or any one of the methods in the first aspect.
[0021] In a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes the steps of processing in the first aspect or any one of the methods in the first aspect.
[0022] Among them, the beneficial effects of the second to fourth aspects can refer to the first aspect and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application;
[0024] Figure 2 It is a schematic diagram of an application scenario provided by an embodiment of the present application;
[0025] Figure 3 It is a flowchart of implementing a preview function in combination with the software architecture of an electronic device provided in an embodiment of the present application;
[0026] Figure 4 is a flowchart of a method for determining a frame rate provided in an embodiment of the present application;
[0027] Figure 5 is a flowchart of a method for obtaining a second frame rate by a first process provided in an embodiment of the present application;
[0028] Figure 6 is a flowchart of a method for determining a third frame rate by a second process provided in an embodiment of the present application;
[0029] Figure 7 It is a structural block diagram of a device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] The technical solution in this application will be described below in conjunction with the accompanying drawings.
[0031] In the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solution of the embodiment of the present application, in the embodiment of the present application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.
[0032] The method for determining the frame rate provided in the embodiment of the present application can be applied to electronic devices, including but not limited to mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart homes, etc. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.
[0033] The following is a schematic diagram of the hardware structure of the electronic device 100 for implementing the above method, for example, Figure 1 A schematic diagram of the hardware structure of the electronic device 100 is shown.
[0034] refer to Figure 1The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.
[0035] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0036] For example, when the electronic device 100 is a mobile phone or a tablet computer, it may include all the components shown in the figure, or may include only some of the components shown in the figure.
[0037] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0038] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
[0039] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0040] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0041] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0042] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.
[0043] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.
[0044] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0045] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0046] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the electronic device 100 .
[0047] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
[0048] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0049] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).
[0050] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0051] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, Microled, Micro-oled, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
[0052] The electronic device 100 can realize the shooting / video recording function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0053] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element (such as an image sensor) through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.
[0054] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
[0055] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
[0056] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0057] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.
[0058] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170 and the application processor.
[0059] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.
[0060] The sensor module 180 includes, for example, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and the like.
[0061] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
[0062] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
[0063] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0064] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.
[0065] In actual application scenarios, the operating system of the electronic device 100 may include, but is not limited to, Symbian, Android, Windows, MacOS, iOS, Blackberry, HarmonyOS, etc. This application does not impose any limitation on the operating system of the electronic device.
[0066] Figure 2 It is a schematic diagram of an application scenario provided by an embodiment of the present application.
[0067] For example, the user can instruct the phone to open the camera application by touching a specific control on the phone screen, pressing a specific physical button or button combination, inputting a voice, performing an air gesture, etc. In response to receiving the user's instruction to open the camera, the phone starts the camera and displays the shooting interface.
[0068] For example, Figure 2As shown in (1) of the figure, the display screen of the mobile phone displays the current output interface content, which displays multiple applications (applications, Apps). The user can click the "Camera" application icon on the mobile phone desktop to instruct the mobile phone to open the camera application. The mobile phone displays the following Figure 2 The shooting interface shown in (2) in FIG.
[0069] For another example, when the phone is in the locked screen state, the user can also instruct the phone to open the camera application by sliding right (or left) on the phone screen, and the phone can also display the following: Figure 2 The shooting interface shown in (2) in FIG.
[0070] Alternatively, when the phone is in lock screen mode, the user can click the shortcut icon of the "Camera" application on the lock screen interface to instruct the phone to open the camera application. The phone can also display the following Figure 2 The shooting interface shown in (2) in FIG.
[0071] For another example, when the phone is running other applications, the user can also click the corresponding control to open the camera application of the phone to take pictures. For example, when the user is using an instant messaging application (such as WeChat application), the user can also select the control of the camera function to instruct the phone to open the camera application to take pictures and videos.
[0072] like Figure 2 As shown in (2) in FIG. 1 , the camera shooting interface generally includes a viewfinder, a photo control, a video control, and other function controls (such as a portrait function control, a night view function control, or more other controls). The user can turn on the video recording mode by clicking the video recording control, and the phone can display the following information: Figure 2 The recording interface is shown in (3) in the figure. The user can enter the setting interface by clicking "Settings". The phone will display the following Figure 2 The interface shown in (4) in . Figure 2 The video frame rate in the interface shown in (4) can be the default, or the user can select the frame rate by clicking "Video Frame Rate".
[0073] Understandably, Figure 2 The setting interface shown in (4) may also include other controls related to camera settings that are not shown, or the setting interface may include Figure 2 Part of the controls shown in (4) in the figure, Figure 2 The controls shown in (4) are merely exemplary descriptions and are not limited in this application.
[0074] The camera includes functions such as taking photos, recording videos and previewing. The preview function means that after opening the camera application, before clicking the corresponding camera control (such as taking photos, recording videos, panorama, etc.) to start shooting, the viewfinder can display the real-time picture taken by the camera. The picture will change with the movement of the phone.
[0075] Figure 3 It is a flowchart of implementing the preview function in combination with the software architecture of the electronic device provided in the embodiment of the present application.
[0076] For example, Figure 3 As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom, namely, the application (APP) layer, the application framework (FWK) layer, the hardware abstraction (HAL) layer, and the kernel layer.
[0077] The application layer includes multiple surface view controls, including a surface view control for camera preview, which is created or run in the camera process. The FWK layer includes a camera service process, a buffer queue (BBQ) #1, a buffer queue #2, and a first process (such as surfaceflinger). The HAL layer includes a second process (such as cameraprovider), a hardware layer mixer (HWC), a DispSync process, etc. The kernel layer includes an image sensor and a display.
[0078] The image sensor can continuously generate image frames and pass them to the second process camera provider, which passes them to the camera service process. The camera provider can call the image sensor downward, and the CameraProviderManager of the FWK layer is called upward through the binder (communication path / mechanism between processes), and the CameraService can use the CameraProviderManager.
[0079] The camera service process sends the image frames to buffer queue #1 in the form of a queue. The production of the image frames is now completed. The subsequent process is the consumption of the image frames. The image frames in buffer queue #1 will be output in a certain order and will pass through buffer queue #2 and surfaceflinger before reaching the display screen through HWC. Buffer queue #2 is the memory allocated for the drawing buffer of the surface in Surfaceflinger. The first process outputs the surface to HWC for HWC synthesis processing. During the processing, HWC will send a hw vsync signal to DispSync. DispSync generates event #1 or event #2 based on the hw vsync signal. Event #1, such as the sf vsync event, can trigger the Surfaceflinger synthesis layer. Event #2, such as the appvsync event, can trigger the drawing of the camera app. After the first process completes the corresponding transaction, it will release the called processes in sequence.
[0080] The image sensor has a phase-locked loop, and the accuracy of the relevant operation can reach 0.01. For example, if the frame rate set during the operation of the camera function of the electronic device is 30 Hz, the frame rate of the image sensor to generate the image frame can be maintained in the range of 29.70 to 30.30 Hz. The frame rate during the operation of the camera function of the electronic device can be set by default in the camera application, or the user can also set it by Figure 2 The "video frame rate" selection shown in (4) is not limited by this application. There is no phase-locked loop in the display screen, so the accuracy when performing operations is not high, for example, 3%. When the frame rate set during the operation of the camera function of the electronic device is 30 Hz, the frame rate of the image frame played by the display screen is in the range of 29.10 to 30.90 Hz. Therefore, there will be a large difference between the frame rate of the image frame generated by the image sensor and the frame rate of the image frame played by the display screen.
[0081] There are two schemes for the process from image frame generation to display, such as cached frame scheme and non-cached frame scheme. The following introduces the problems that may occur when the frame rate of the image sensor generating the image frame and the frame rate of the display playing the image frame differ greatly under these two schemes.
[0082] Frame cache solution: When the frame rate of the image sensor generating image frames is greater than the frame rate of the display screen playing image frames, the buffer queue #1 of the FWK layer will fill up the memory as the image sensor continues to generate image frames, which will cause buffer accumulation, resulting in tracking problems and delays. After a period of time, it will cause preview and filming frame loss. Filming is the finished product shot after preview. When the frame rate of the image sensor generating image frames is less than the frame rate of the display screen playing image frames, the buffer queue #1 of the FWK layer may be in an empty state, which does not affect the rotation, but there will be a dull feeling, because in order to align the frame rate of the display screen playing image frames with the frame rate of the image sensor generating image frames during the display sending process, the display screen will display repeated image frames, and the film will not lose frames, but there will be preview jamming problems.
[0083] Non-cached frame solution: When the frame rate of the image sensor generating image frames is greater than the frame rate of the display screen playing image frames, frames will be lost before entering the buffer queue #2, and the time intervals for the image frames to enter the first process are uneven, which will aggravate the frame loss during preview and cause preview jamming. When the frame rate of the image sensor generating image frames is less than the frame rate of the display screen playing image frames, similar to the cached frame solution, the display screen will display repeated image frames, and the film will not be lost, but the preview jamming problem will occur.
[0084] In view of this, an embodiment of the present application provides a method for determining a frame rate, so as to avoid discarding image frames or preview freezes as much as possible during the operation of the camera function of the electronic device, thereby improving the user experience.
[0085] The method includes: during the operation of the camera function of an electronic device (including a display screen and an image sensor), obtaining a first frame rate and a second frame rate, the first frame rate being the frame rate used by the electronic device when implementing the camera function, the second frame rate being the frame rate of the display screen playing image frames, and the tearing effect signal frequency corresponding to the second frame rate being within the specification error range of the display screen; determining a third frame rate based on the first frame rate and the second frame rate, the third frame rate being the frame rate of the image sensor generating image frames; and generating image frames based on the third frame rate.
[0086] Exemplarily, determining the third frame rate based on the first frame rate and the second frame rate includes: determining the fourth frame rate based on the first frame rate and the second frame rate, the values of the fourth frame rate and the second frame rate are equal, and when the absolute value of the difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold, determining the fourth frame rate to be the third frame rate.
[0087] Exemplarily, determining the third frame rate based on the first frame rate and the second frame rate includes: determining a fourth frame rate based on the first frame rate and the second frame rate, the second frame rate and the fourth frame rate are equal, and when the absolute value of the difference between the fourth frame rate and the first frame rate is greater than a preset threshold; determining the third frame rate based on the first frame rate and the preset threshold. In a possible implementation, obtaining the second frame rate includes: obtaining a refresh rate of a display screen; and obtaining the second frame rate based on the refresh rate.
[0088] Exemplarily, the software architecture of the electronic device includes an application framework layer, in which a first process runs, and obtaining the refresh rate of the display screen includes: the first process requests the refresh rate of the display screen from the display screen; the first process receives the refresh rates of multiple display screens from the display screen.
[0089] Exemplarily, the software architecture of the electronic device includes an application framework layer and a hardware abstraction layer, a first process runs in the application framework layer, and a second process runs in the hardware abstraction layer, and a second frame rate is obtained based on the refresh rate, including: the first process determines the second frame rate based on the refresh rates of the multiple display screens; the first process indicates the second frame rate to the second process; the second process obtains the second frame rate according to the indication of the first process, or the second process determines the third frame rate based on the tearing effect signal frequency, the first frame rate and a preset threshold.
[0090] A possible implementation method is to determine a third frame rate based on the first frame rate and the second frame rate, including: a second process determines a tearing effect (TE) signal frequency corresponding to the second frame rate based on the second frame rate; and a second process determines the third frame rate based on the TE signal frequency and the first frame rate.
[0091] Optionally, the method further includes: the second process indicating a third frame rate to the image sensor; generating image frames based on the third frame rate, including: the image sensor generating image frames according to the third frame rate indicated by the second process.
[0092] Optionally, the method further includes: the image sensor sending feedback information to the second process, wherein the feedback information indicates that the image sensor generates image frames according to a third frame rate.
[0093] Optionally, the method further includes: the second process receiving feedback information; and the second process recording at least one of the following according to the feedback information: the feedback information and the second frame rate.
[0094] Below Figure 4 Take the above method as an example to describe it in detail. Figure 4The first process takes surfaceflinger as an example, the second process takes camera provider as an example, the application using the camera function takes the camera application as an example, and the camera application can be an Android application package (Android application package, APK).
[0095] like Figure 4 As shown, the method for determining the frame rate can be applied to an electronic device, and the electronic device includes an image sensor and a display screen. The method includes the following steps:
[0096] S401, the user opens the camera application.
[0097] For example, please refer to Figure 2 The relevant description in will not be repeated here.
[0098] S402: The camera application performs streaming.
[0099] Exemplarily, the streaming process starts from the camera application, passes through the framework layer to the hardware abstraction layer.
[0100] S403, the surfaceflinger process requests the display screen for a refresh rate of the display screen.
[0101] Exemplarily, the refresh rate requested by the surfaceflinger process can be a refresh rate of 6 frames, that is, the display screen actually displays a frame rate of 6 frames, or the refresh rate requested by the surfaceflinger process can also be a refresh rate of 12 frames or a refresh rate of other frame numbers. This application does not impose any restrictions on this.
[0102] It can be understood that each time the user opens the camera application or switches the screen, the execution of S403 and S404 can be triggered. Since the refresh rate will change due to the influence of temperature, each time the user opens the camera application or switches the screen, the execution of S403 and S404 is triggered. This can make the second frame rate subsequently determined according to the refresh rate more in line with the actual use of the electronic device, thereby better improving the user experience.
[0103] S404, the display screen sends the refresh rate of the display screen to the surfaceflinger process in response to the request; correspondingly, the surfaceflinger process can receive the refresh rates of multiple display screens from the display screen.
[0104] It can be understood that the embodiment of the present application does not limit the execution order of S403 and S404, and S401 and S402. S403 and S404 can be executed first, or S401 and S402 can be executed first.
[0105] S405: The surfaceflinger process obtains a second frame rate.
[0106] The following method 500 is used to illustrate a method for the first process (taking the surfaceflinger process as an example) to obtain the second frame rate. Figure 5 As shown, the method 500 includes the following steps:
[0107] S501, the surfaceflinger process obtains multiple refresh rates of the display screen, for example, obtains the refresh rate of the display screen displaying 6 frames. The specific acquisition method refers to the description of S403 and S404, which will not be repeated here.
[0108] S502, the surfaceflinger process determines a second frame rate according to a least squares regression line equation.
[0109] Exemplarily, the least squares method is used to regress the linear equation y=ax+b, and the values of a and b are determined according to the refresh rates of the discrete six frames and the equation y=ax+b. If the refresh rates of the six frames obtained are the refresh rates of frame #1, frame #2, frame #3, frame #4, frame #5, and frame #6, respectively, when determining the values of a and b, the discrete points (x1, y1), (x2, y2), (x3, y3), (x4, y4), (x5, y5), and (x6, y6) can be substituted, where x1 represents the time corresponding to frame #1, y1 represents the refresh rate of frame #1, x2 represents the time corresponding to frame #2, y2 represents the refresh rate of frame #2, and so on. After determining the values of a and b, the refresh rate corresponding to a subsequent moment, i.e., the second frame rate, can be determined according to y=ax+b.
[0110] At this point, the surfaceflinger process has acquired the second frame rate.
[0111] S503, the surfaceflinger process determines whether to correct the above equation y=ax+b.
[0112] Exemplarily, each time the camera preview function is used, for example, when the electronic device turns on or off the screen, the user opens the camera application again, or the user switches the screen, the surfaceflinger process determines that the equation needs to be corrected, or in other words, the surfaceflinger process is triggered to correct the equation, that is, S501 and S502 are executed again to obtain a new second frame rate; if the surfaceflinger process determines that the equation does not need to be corrected, for example, other usage scenarios other than turning on or off the electronic device screen, the user opens the camera application again, or the user switches the screen do not require the correction equation, the second frame rate previously determined according to S502 can be directly obtained.
[0113] S406: The surfaceflinger process indicates the second frame rate to the camera provider process through the binder.
[0114] S407, after receiving the instruction from the surfaceflinger process, the camera provider process may determine the second frame rate, and determine the third frame rate in combination with the first frame rate.
[0115] For example, the first frame rate is a “video frame rate” selected by the user in the camera application settings or a default “video frame rate” of the camera application, see Figure 2 in (4).
[0116] S408: After determining the third frame rate, the camera provider process indicates the third frame rate to the image sensor.
[0117] Exemplarily, after receiving the instruction from the camera provider process, the image sensor can determine the third frame rate according to the instruction, and subsequently generate image frames according to the third frame rate.
[0118] S409: The image sensor sends feedback information to the camera provider process, where the feedback information indicates that the image sensor generates image frames according to the third frame rate.
[0119] That is to say, after receiving the instruction from the camera provider process, the image sensor will adjust the frame rate of the image frames generated by itself to the third frame rate according to the instruction. After the adjustment is successful, it will feedback the success of the adjustment to the camera provider process through feedback information. If the adjustment fails, it will feedback the failure of the adjustment to the camera provider process through feedback information.
[0120] S410, after receiving the feedback information, the camera provider process records key data according to the feedback information, where the key data includes at least one of the following: feedback information and a second frame rate.
[0121] The behavior of the electronic device can then be explained based on the key data, or if a problem occurs later, the problem can be located based on key data analysis.
[0122] The following method 600 is used to illustrate a method in which the second process (taking the camera provider process as an example) determines the third frame rate based on the first frame rate and the second frame rate. Figure 6 As shown, the method 600 includes the following steps:
[0123] S601, flow distribution process, can refer to S402, and will not be repeated here.
[0124] S602, the camera provider process obtains the second frame rate, which may be referred to in S406 and will not be described in detail.
[0125] Exemplarily, if the acquisition of the second frame rate fails, S608 is executed to record key data, such as a communication failure flag.
[0126] S603: The camera provider process determines a TE signal frequency corresponding to the second frame rate according to the second frame rate.
[0127] Exemplarily, the second frame rate is 30.5HZ, and the TE signal frequency corresponding to the second frame rate is determined to be 30.5HZ*n. If the specification of the display screen is 120HZ, then n=[120HZ / 30.5HZ]=4, and “[]” indicates rounding, that is, the TE signal frequency corresponding to the second frame rate is 122HZ.
[0128] Optionally, the method 600 further includes:
[0129] The camera provider process determines whether the TE signal frequency determined based on the second frame rate is within the error range of the display specification. If it is within the error range, execute S604. If it is not within the error range, the second frame rate can be recorded, or the TE signal frequency corresponding to the second frame rate can be recorded.
[0130] For example, the display specification is 120HZ, the accuracy is 3%, that is, the error range is 120HZ±120HZ*3%. When the TE signal frequency determined based on the second frame rate is 122HZ, That is to say, the TE signal frequency determined based on the second frame rate is within the error range of the display screen specification.
[0131] S604: The camera provider process determines a fourth frame rate according to the first frame rate and the TE signal frequency.
[0132] For example, the first frame rate is 30 Hz, the TE signal frequency is 122 Hz, and the value of n for 1 / n division is [122 Hz / 30 Hz] = 4. Therefore, combining the first frame rate and the value of n, the fourth frame rate = 122 Hz / n = 122 Hz / 4 = 30.5 Hz.
[0133] It can be understood that the second frame rate is determined based on the refresh rate of the image frames currently actually played by the acquired display screen, and the fourth frame rate is the theoretical frame rate of the subsequent image sensor generated by the camera provider process calculated by the camera provider process. The theoretical frame rate is determined based on the first frame rate and the second frame rate. The value of the fourth frame rate is theoretically equal to the value of the second frame rate, but the embodiments of the present application are not limited to this. The value of the second frame rate and the value of the fourth frame rate can be approximately equal, that is, the difference between the second frame rate and the fourth frame rate is within the error range. The subsequent image sensor will generate image frames at the third frame rate, and the third frame rate is determined based on the fourth frame rate. However, due to the accuracy of the image sensor, the frame rate at which the image sensor actually generates image frames is close to the third frame rate and is not necessarily completely equal to the third frame rate.
[0134] S605: The camera provider process determines whether the difference between the first frame rate and the fourth frame rate is less than or equal to a preset threshold.
[0135] In a possible implementation manner, the camera provider process determines that the absolute value of the difference between the first frame rate and the fourth frame rate is less than or equal to a preset threshold, and then determines that the fourth frame rate is the third frame rate.
[0136] Exemplarily, the preset threshold is the range of the image sensor's ability to adjust the frame rate of the generated image frame. For example, the image sensor can adjust the frame rate of the generated image frame by a maximum of 0.30 Hz, that is, the preset threshold is 0.30 Hz. If the calculated fourth frame rate is 30.20 Hz, the preset threshold is 0.30 Hz, and the first frame rate is 30 Hz, then 30.20-30=0.20, that is, the absolute value of the difference between the first frame rate and the fourth frame rate is ≤ the preset threshold, and therefore the third frame rate is determined to be 30.20 Hz.
[0137] When the absolute value of the difference between the first frame rate and the fourth frame rate is ≤ the preset threshold, execute S606; when the absolute value of the difference between the first frame rate and the fourth frame rate is > the preset threshold, record key data, such as recording the calculated fourth frame rate.
[0138] In a possible implementation, the camera provider process determines that the absolute value of the difference between the first frame rate and the fourth frame rate is greater than a preset threshold, and then determines the third frame rate based on the first frame rate and the preset threshold.
[0139] Exemplarily, the first frame rate is 30 Hz, and the calculated fourth frame rate is 30.60 Hz. The image sensor can adjust the frame rate within the range of ±0.30 Hz based on the first frame rate, that is, the preset threshold is 0.30 Hz, and the image sensor adjusts the frame rate within the range of 29.70 Hz to 30.30 Hz. At this time, the third frame rate can be 30 Hz+0.30 Hz=30.30 Hz instead of 30.60 Hz.
[0140] S606, setting the frame rate of the image sensor generating image frames to the third frame rate, that is, the camera provider process indicates the third frame rate to the image sensor, so that the image sensor subsequently generates image frames according to the third frame rate. For specific indication of the third frame rate, see S408, which will not be repeated here.
[0141] S607, after S606 is executed, if the image sensor adjusts the frame rate to the third frame rate and feeds back to the cameraprovider process that the adjustment is successful, the cameraprovider process can detect the effectiveness result and record key data. For the key data here, please refer to the relevant description in S410 and will not be repeated here.
[0142] Exemplarily, the camera provider process detection effectiveness result includes: the camera provider process obtains the image frame generated by the image sensor, the image frame includes a timestamp, the camera provider process determines the image frame generation time interval based on the timestamp in the image frame continuously generated by the image sensor, and determines whether the time interval matches the third frame rate. For example, the third frame rate is 30HZ, that is, 30 image frames are generated per second. If the timestamp determines that the image frame generation time interval is equal to or approximately equal to 1 / 30 second, it is determined that the time interval matches the third frame rate, and the key data third frame rate and / or feedback information is recorded at this time.
[0143] S608, when executing S602, S603, S605 and S607, key data can be recorded when relevant situations occur. For specific key data to be recorded in which situations, please refer to the contents in the relevant steps and will not be repeated here.
[0144] Based on the solution provided in the embodiment of the present application, the frame rate of the image frames generated by the image sensor can be made as close as possible to the frame rate of the image frames actually played back by the display screen, so as to avoid the problem of discarding image frames or preview freezes during the operation of the camera function of the electronic device, thereby improving the user experience.
[0145] In addition, in addition to the specific implementation of determining the third frame rate exemplified in method 400 or 600, there are other possible implementations, which are not limited in the present application.
[0146] For example, after the surfaceflinger process acquires the second frame rate in step S405, the surfaceflinger process may determine the fourth frame rate based on the second frame rate. For details on how the surfaceflinger process determines the fourth frame rate based on the second frame rate, please refer to the relevant descriptions in S603 and S604.
[0147] Referring to S603, the surfaceflinger process determines whether the TE signal frequency determined based on the second frame rate is within the error range of the display screen specifications. If it is within the error range, the surfaceflinger process performs the operation performed by the camera provider process in S604. If it is not within the error range, the second frame rate can be recorded, or the TE signal frequency corresponding to the second frame rate can be recorded. In this process, the surfaceflinger process can obtain the specifications and accuracy of the display screen from the camera provider process, or the surfaceflinger process can obtain the specifications and accuracy of the display screen from the display screen itself. If the TE signal frequency corresponding to the second frame rate is not within the error range, the surfaceflinger process can record the second frame rate or the TE signal frequency corresponding to the second frame rate by itself, or the surfaceflinger process passes the second frame rate or the TE signal frequency corresponding to the second frame rate to the camera provider process through a binder, and the camera provider process records it.
[0148] After executing S604, the determined fourth frame rate can be passed to the camera provider process through the binder, and the camera provider process completes the judgment of S605, or after executing S604, the surfaceflinger process can execute S605 by itself to make a judgment. The third frame rate can be determined based on the fourth frame rate when the threshold condition is met or not. The method for the surfaceflinger process to determine the third frame rate can refer to the relevant description in S605. The threshold condition is met, that is, the difference between the first frame rate and the fourth frame rate is ≤ the preset threshold. In the case of not meeting the threshold condition, the surfaceflinger process can record the key data, or the surfaceflinger process can pass the key data to the camera provider process through the binder, and the camera provider process records it.
[0149] If the third frame rate is determined by the surfaceflinger process, after the third frame rate is determined, the third frame rate is transmitted to the camera provider process through the binder, and the camera provider process indicates the third frame rate to the image sensor.
[0150] In S607, the surfaceflinger process may detect the effectiveness result, and the basis for the surfaceflinger process to detect the effectiveness result may be that the camera provider process transmits it to the surfaceflinger process through the binder.
[0151] In general, in other possible implementations, the operations performed by the cameraprovider process in method 400, method 500 or method 600 can be performed by the surfaceflinger process, and the data required for the execution by the surfaceflinger process can be transmitted from the camera provider process to the surfaceflinger process through a binder, or the surfaceflinger process obtains it in other ways. Or in method 400, method 500 or method 600, the operations performed by the surfaceflinger process can be performed by the camera provider process, and the data required for the execution by the camera provider process can be transmitted from the surfaceflinger process to the camera provider process through a binder, or the camera provider process obtains it in other ways.
[0152] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.
[0153] Figure 7 This is a structural block diagram of a device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.
[0154] The device 700 is, for example, an electronic device, Figure 7 The device 700 includes: an acquisition module 701, a determination module 702, a generation module 703, and the device 700 also includes an image sensor and a display screen ( Figure 7 not shown).
[0155] During the operation of the camera function of the electronic device, the acquisition module 701 is used to acquire a first frame rate and a second frame rate, the first frame rate is the frame rate used by the electronic device when implementing the camera function, the second frame rate is the frame rate of the display screen playing the image frame, and the tearing effect signal frequency corresponding to the second frame rate is within the specification error range of the display screen; the determination module 702 is used to determine a third frame rate based on the first frame rate and the second frame rate; the generation module 703 is used to generate image frames based on the third frame rate.
[0156] In some possible implementations, the determination module 702 is used to determine a fourth frame rate based on the first frame rate and the second frame rate, and the values of the fourth frame rate and the second frame rate are equal. When the absolute value of the difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold, the determination module 702 is also used to determine that the fourth frame rate is the third frame rate.
[0157] In some possible implementations, the determination module 702 is used to determine the fourth frame rate based on the first frame rate and the second frame rate, the values of the second frame rate and the fourth frame rate are equal, and when the absolute value of the difference between the fourth frame rate and the first frame rate is greater than a preset threshold; the determination module 702 is also used to determine the third frame rate based on the first frame rate and the preset threshold.
[0158] In some possible implementations, the acquisition module 701 is specifically used to acquire a refresh rate of the display screen, and the determination module 702 is further used to acquire a second frame rate based on the refresh rate.
[0159] In some possible implementations, the software architecture of the device 700 includes an application framework layer, a first process runs in the application framework layer, and the acquisition module 701 obtains the refresh rate of the display screen, including: the first process requests the refresh rate of the display screen from the display screen; the first process receives the refresh rates of multiple display screens from the display screen.
[0160] In some possible implementations, the software architecture of the device 700 includes an application framework layer and a hardware abstraction layer, a first process runs in the application framework layer, and a second process runs in the hardware abstraction layer, and the determination module 702 obtains the second frame rate based on the refresh rate, including: the first process determines the second frame rate based on the refresh rates of the multiple display screens; the first process indicates the second frame rate to the second process; the second process obtains the second frame rate according to the instruction of the first process.
[0161] In some possible implementations, the determination module 702 determines the third frame rate based on the first frame rate and the second frame rate, including: the second process determines the TE signal frequency corresponding to the second frame rate based on the second frame rate; the second process determines the third frame rate based on the TE signal frequency and the first frame rate, or the second process determines the third frame rate based on the tearing effect signal frequency, the first frame rate and a preset threshold.
[0162] In some possible implementations, the second process indicates a third frame rate to the image sensor; the generation module 703 generates image frames based on the third frame rate, including: the image sensor generates image frames according to the third frame rate indicated by the second process.
[0163] In some possible implementations, the image sensor is further configured to send feedback information to the second process, where the feedback information indicates that the image sensor generates image frames according to the third frame rate.
[0164] In some possible implementations, the second process is further used to receive feedback information, and record at least one of the following items based on the feedback information: the feedback information and the second frame rate.
[0165] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions, execution steps, interpretation of terms, associations and technical effects brought about can be found in the method embodiment part and will not be repeated here.
[0166] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. For example, a "module" can be a software program, a hardware circuit, or a combination of the two to implement the above-mentioned functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions.
[0167] Therefore, the modules of each example described in the embodiments of the present application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
[0168] In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the above method embodiment, which will not be repeated here.
[0169] It should be understood that the hardware systems, devices and electronic devices in the embodiments of the present application can execute the method for determining the frame rate in the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.
[0170] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium is run on an electronic device, the electronic device executes the method shown above. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) methods. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid state disk (SSD)), etc.
[0171] The embodiment of the present application also provides a computer program product including computer instructions, which, when executed on an electronic device, enables the electronic device to execute the technical solution shown above.
[0172] An embodiment of the present application also provides an electronic device, including a display screen and an image sensor, wherein the display screen is used to display image frames generated by the image sensor, and the electronic device is used to execute the technical solutions shown above, such as method 400 / 500 / 600.
[0173] The devices, electronic devices, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.
[0174] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in each embodiment of the above method may be unnecessary, or some new steps may be added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined solutions also fall within the scope of the embodiments of the present application.
[0175] In the textual description of the embodiments of the present application, the character “ / ” generally indicates that the previously associated objects are in an “or” relationship; in the formulas of the embodiments of the present application, the character “ / ” indicates that the previously associated objects are in a “division” relationship.
[0176] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.
[0177] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.
[0178] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.
[0179] It should also be understood that in the embodiments of the present application, "pre-configuration", "pre-setting" and "pre-definition" can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the electronic device, and the present application does not limit its specific implementation method.
[0180] It should also be understood that in the embodiments of the present application, similar conditional descriptions such as "in the case of...", "if...", "when...", etc. appear, which means that the corresponding operation will be executed after the conditions are met, but the embodiments of the present application do not limit when the corresponding operation is executed, and it does not limit that the corresponding operation will be executed immediately after the conditions are met.
[0181] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.
[0182] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for determining the frame rate, It is characterized in that Applied to an electronic device, the electronic device includes a display screen and an image sensor, the method includes: During the operation of the camera function of the electronic device, a first frame rate and a second frame rate are obtained, wherein the first frame rate is a frame rate used by the electronic device when implementing the camera function, and the second frame rate is a frame rate of the display screen playing image frames, and a tearing effect signal frequency corresponding to the second frame rate is within a specification error range of the display screen; Determining a tearing effect signal frequency corresponding to the second frame rate according to the second frame rate; Determine a value of n based on a ratio of the tearing effect signal frequency to the first frame rate, where n is an integer, and determine a fourth frame rate based on the ratio of the first frame rate to the value of n; determine the fourth frame rate to be the third frame rate when an absolute value of a difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold; determine the third frame rate based on a sum of the first frame rate and the preset threshold when an absolute value of a difference between the fourth frame rate and the first frame rate is greater than a preset threshold, so that the third frame rate is within a range of the frame rate adjusted by the image sensor, and the third frame rate represents a frame rate at which the image sensor generates image frames; Image frames are generated based on the third frame rate.
2. The method according to claim 1, It is characterized in that The acquiring the second frame rate comprises: Obtaining a refresh rate of the display screen; The second frame rate is acquired based on the refresh rate.
3. The method according to claim 2, It is characterized in that The software architecture of the electronic device includes an application framework layer, a first process is running in the application framework layer, and obtaining the refresh rate of the display screen includes: The first process requests the display screen for a refresh rate of the display screen; The first process receives refresh rates of a plurality of the display screens from the display screen.
4. The method according to claim 2 or 3, It is characterized in that The software architecture of the electronic device includes an application framework layer and a hardware abstraction layer, a first process runs in the application framework layer, and a second process runs in the hardware abstraction layer, and the acquiring the second frame rate based on the refresh rate includes: The first process determines the second frame rate based on refresh rates of the plurality of display screens; The first process indicates the second frame rate to the second process; The second process acquires the second frame rate according to an instruction of the first process.
5. The method according to claim 4, It is characterized in that The method further comprises: The second process determines, based on the second frame rate, a tearing effect signal frequency corresponding to the second frame rate; The second process determines the value of n based on the ratio of the tearing effect signal frequency to the first frame rate, where n is an integer, and determines the fourth frame rate according to the ratio of the first frame rate to the value of n; when the absolute value of the difference between the fourth frame rate and the first frame rate is less than or equal to a preset threshold, the fourth frame rate is determined to be the third frame rate; when the absolute value of the difference between the fourth frame rate and the first frame rate is greater than a preset threshold, the third frame rate is determined based on the sum of the first frame rate and the preset threshold, so that the third frame rate is within the range of the image sensor adjusting the frame rate.
6. The method according to claim 5, It is characterized in that The method further comprises: The second process indicates the third frame rate to the image sensor; The generating of image frames based on the third frame rate comprises: The image sensor generates image frames according to the third frame rate indicated by the second process.
7. The method according to claim 6, It is characterized in that The method further comprises: The image sensor sends feedback information to the second process, where the feedback information indicates that the image sensor generates image frames according to the third frame rate.
8. The method according to claim 7, It is characterized in that The method further comprises: The second process receives the feedback information; The second process records at least one of the following according to the feedback information: the feedback information and the second frame rate.
9. An electronic device, It is characterized in that The electronic device comprises a display screen and an image sensor, wherein the display screen is used to display image frames generated by the image sensor, and the electronic device is used to execute the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 8.
11. A computer program product, It is characterized in that The computer program product comprises: a computer program code, and when the computer program code is executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 8.
