A camera request processing method and related apparatus

Abstract

A camera request processing method and related apparatus. In the method, an electronic device can divide a buffer for storing camera requests into two parts, one part for storing normal camera requests and the other part for storing special camera requests. When a camera in the electronic device is started, a plurality of camera requests can be distributed to the buffer for storing normal camera requests. When the electronic device receives a zoom request, the electronic device can directly distribute the zoom request to the buffer for storing special camera requests and issue the zoom request to the HAL. Since the FWK immediately issues the zoom request to the HAL, the HAL can be applied to the generated image frame as soon as possible after receiving the zoom request issued by the FWK, thereby reducing the time for the electronic device to respond to the zoom operation of the user, i.e., improving the followability of the zoom operation of the user and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and in particular to a camera request processing method and related apparatus. Background Technology

[0002] Currently, mobile phones, tablets, and other electronic devices all have camera applications. Users can take photos and videos using these applications. As the photo and video recording functions of camera applications on mobile phones and other electronic devices become more powerful, the number of shooting modes on these devices is also increasing. Users can change the camera's shooting mode in the camera's shooting interface, such as changing the current focal length or adding filters. When the user changes the camera's shooting mode, a preview image frame of the new shooting mode will be displayed in response to the user's operation in the electronic device's shooting interface.

[0003] However, when a user changes the camera shooting mode in the camera shooting interface, the electronic device's response to the user's operation is delayed. For example, when the user zooms in, there will be a delay before the electronic device displays the zoomed image frame in the camera shooting interface, resulting in a poor user experience.

[0004] Therefore, how to reduce the response latency of electronic devices to user changes in shooting mode is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a camera request processing method and related apparatus. Through the camera request processing method provided by this application, electronic devices can reduce the response time to user operations that change shooting modes, thereby improving the user experience.

[0006] Firstly, this application provides a camera request method, which can be applied to an electronic device equipped with a camera module. The method may include: detecting a user's operation of changing camera parameters and issuing a first camera request; determining that the camera parameters in the first camera request are different from the camera parameters in a second camera request, and allocating the first camera request to a first buffer in a first buffer area; dividing a storage area in the electronic device used to store camera requests into a first buffer area and a second buffer area, wherein the first buffer area is used to store the first camera request, and the camera parameters include one or more of zoom ratio, focus parameters, exposure parameters, and filter parameters, and the second camera request is a camera request obtained by the electronic device before obtaining the first camera request; obtaining a first image frame according to the camera parameters in the first camera request; and displaying the first image frame.

[0007] Implementing the method provided in the first aspect, multiple camera requests for acquiring preview images issued when the camera in the electronic device is turned on are stored only in the second buffer, while the first buffer is only allocated to camera requests where camera parameters change. When a user changes the zoom level, focuses, adjusts the exposure, or adds filters in the camera's shooting interface, the electronic device can directly allocate the camera request generated based on that user operation to a special camera request buffer. The application framework layer does not need to wait for a buffer to be cleared before allocating the request to that buffer, nor does it need to wait for the HAL to process the next request before allocating a buffer to the camera request. In this way, the response time of the electronic device to user operations such as changing the zoom level, focusing, adjusting the exposure, or adding filters can be reduced, thereby improving the response efficiency of user operations.

[0008] In one possible implementation, the first buffer includes q buffers and the second buffer includes p buffers. The method may further include determining the values ​​of q and p based on the camera mode of the camera application in the electronic device. In this way, the electronic device can determine different values ​​of p and q based on different modes.

[0009] In one possible implementation, the values ​​of q and p are determined based on the camera mode of the camera application in the electronic device. This includes: when the camera mode is a photo mode, q is determined to be a first value and p to be a second value; when the camera mode is a video mode, q is determined to be a third value and p to be a fourth value, the sum of the third and fourth values ​​is a fifth value, the sum of the first and second values ​​is a sixth value, and the fifth and sixth values ​​are equal. In this way, the electronic device can determine different values ​​of p and q based on different modes. This ensures that the camera can smoothly send camera requests under different camera modes.

[0010] In one possible implementation, before detecting a user's operation to change camera parameters and sending the first camera request, the method may further include: the electronic device detecting an operation to launch a camera application and launching the camera application; the camera application in the electronic device sending multiple camera requests, including a second camera request; if there is an empty buffer in the second buffer, allocating the second camera request to the second buffer, which is an empty buffer in the second buffer; if there is no empty buffer in the second buffer, the electronic device waiting for the second buffer in the second buffer to be cleared before allocating the second camera request to the second buffer. In this way, the second camera request will not be sent to the first buffer. When the electronic device sends the first camera request, the first buffer is empty, and the electronic device can quickly allocate a buffer for the first camera request without waiting for the buffer to be cleared before allocating a buffer.

[0011] In one possible implementation, after determining that the camera parameters in the first camera request are different from those in the second camera request, the method may further include: if there is an empty buffer in the second buffer, allocating the first camera request to a second buffer in the second buffer, which is then empty. This allows for reserving more empty buffers for subsequent camera requests with changing camera parameters.

[0012] In one possible implementation, the user's operation of changing the first camera parameter is equivalent to the user changing the zoom ratio. The method may further include: the camera application displaying a first preview interface, which displays a second image frame, the zoom ratio of which is the first zoom ratio; the camera detecting the user's zoom ratio change operation, displaying a second preview interface, which also displays the first image frame, the zoom ratio of which is the second zoom ratio. In this way, the user can see a preview interface before and after the zoom ratio change on the electronic device.

[0013] In one possible implementation, the electronic device includes a request module and a caching module. The module determines that the camera parameters in a first camera request are different from those in a second camera request, and allocates the first camera request to a first buffer in a first cache area. This can include: the request module determining that the camera parameters in the first camera request are different from those in the second camera request; the request module sending the first camera request to the caching module; and the caching module allocating the first camera request to the first buffer in the first cache area. In this way, the caching module can quickly allocate the first camera request to the first buffer without waiting.

[0014] In one possible implementation, the electronic device may further include a camera processing module. After the caching module allocates the first camera request to a first buffer in a first buffer area, the method may further include: the caching module sending the first camera request to the camera processing module. In this way, the caching module can quickly send the first camera request to the camera processing module.

[0015] Obtaining the first image frame based on the camera parameters in the first camera request can include: the camera processing module sending the camera parameters in the first camera request to the camera module; and the camera module obtaining the first image frame according to the camera parameters in the first camera request. In this way, the electronic device can obtain the first image frame corresponding to the first camera request.

[0016] In one possible implementation, the electronic device further includes a caching module and a camera processing module. After allocating the second camera request to the second buffer, the method may further include: the caching module sending the second camera request to the camera processing module.

[0017] In one possible implementation, the electronic device may further include a camera application, a display module, and a caching module. After the caching module sends the second camera request to the camera processing module, the method may further include: the camera processing module sending the camera parameters in the second camera request to the camera module, and obtaining a second image frame obtained by the camera module according to the camera parameters in the second camera request; the camera processing module sending the second image frame to the display module; the display module displaying the second image frame; and the camera application displaying the second image frame. In this way, the electronic device can obtain and display the second image frame through the respective modules.

[0018] In a second aspect, an electronic device is provided, which may include one or more cameras, a display, one or more processors, and one or more memories; wherein the one or more cameras, the display, and the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by one or more processors, cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0019] Thirdly, an electronic device is provided, which may include one or more functional modules for the methods involved in any possible implementation of the first aspect.

[0020] Fourthly, a chip system is provided for use in an electronic device, the chip system including one or more processors for invoking computer instructions to cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0021] Fifthly, a computationally readable storage medium is provided, including instructions that, when executed on an electronic device, cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0022] In a sixth aspect, a computer program product is provided that, when the program product is run on an electronic device, causes the electronic device to perform the methods involved in any possible implementation of the first aspect. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a user interface provided in an embodiment of this application;

[0024] Figures 2A-2C These are schematic diagrams of a set of user interfaces provided in the embodiments of this application;

[0025] Figure 3 This is a schematic diagram of the hardware and software architecture of an electronic device provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram illustrating the time required for a zoom request to be sent to the HAL layer and the time required for an electronic device to successfully respond to a zoom operation, as provided in an embodiment of this application.

[0027] Figure 5 This is a schematic diagram of the hardware and software architecture of an electronic device provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the software module interaction of a camera request processing method provided in an embodiment of this application;

[0029] Figures 7-9 These are schematic diagrams of a set of user interfaces provided in the embodiments of this application;

[0030] Figure 10 This is a schematic diagram of the camera request processing method provided in an embodiment of this application;

[0031] Figure 11 This is a schematic diagram illustrating the time it takes for an electronic device to send a zoom request to the HAL layer and the time required for the electronic device to successfully respond to the zoom operation, as provided in another embodiment of this application.

[0032] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0034] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0035] Currently, there are increasingly more shooting modes on mobile phones and other electronic devices. Users can change the camera's shooting mode in the camera interface, such as changing the current focal length, or adding filters, exposure, focus, and other camera parameters. In some scenarios, when a user changes camera parameters on their phone, the phone may freeze briefly before displaying a preview image of the new shooting mode. This results in a poor user experience.

[0036] Users can change the camera's focus on their phones. Figure 1 , Figures 2A-2C An exemplary diagram illustrates the interface related to a user changing the camera's focus on a mobile phone.

[0037] For example, the electronic device 100 may display a desktop 101, on which a page containing application icons is displayed. This page includes multiple application icons (e.g., settings application icon, app store application icon, gallery application icon, browser application icon, etc.). Below the multiple application icons, a page indicator 104 is also displayed to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator 104, a tray area 102 is displayed. The tray area 102 includes multiple tray icons, such as a camera application icon 103, a contacts application icon, a phone application icon, and a messaging application icon. The tray area 102 remains displayed when switching pages. In some embodiments, the page may also include multiple application icons and a page indicator 104. The page indicator 104 may not be part of the page and may exist independently. The tray icons are also optional, and this embodiment does not limit this.

[0038] Electronic device 100 can receive input operations (e.g., clicks) from a user on the camera application icon 103, and in response to such input operations, electronic device 100 can display, for example... Figure 2A The shooting interface shown is 200.

[0039] like Figure 2A As shown, the shooting interface 200 may include a display control 203A, a shooting control 203B, a camera switching control 203C, a preview frame 201, a zoom ratio control 202, and controls for one or more shooting modes (e.g., a large aperture shooting mode control 204A, a night scene shooting mode control 204B, a portrait shooting mode control 204C, a photo mode control 204D, a video recording mode control 204E, a multi-lens video recording mode control 204F, and a more mode control 204G).

[0040] Among them, such as Figure 2A As shown, the camera mode control 204D is selected, and the electronic device 100 is in camera mode. The preview box 201 displays a preview image 205 captured by the camera in camera mode. The echo control 203A can be used to trigger the display of captured images or videos. The capture control 203B is used to trigger the saving of images captured by the camera. The camera switching control 203C can be used to switch the camera used by the electronic device 100 to capture images (e.g., switching from a front camera to a rear camera, or vice versa). The zoom ratio control 202 can be used to set the zoom ratio for photos or videos taken by the electronic device 100. The zoom ratio control 202 can display the currently used zoom ratio (e.g., 0.6x), a commonly used zoom ratio 1 (e.g., 1x), which is larger than the currently used zoom ratio, and a commonly used zoom ratio 2 (e.g., 2x), which is larger than the currently used zoom ratio.

[0041] The shooting mode controls can be used to trigger the image processing flow corresponding to that shooting mode. For example, control 204A for the large aperture shooting mode can be used to trigger the camera to use large aperture parameters to capture images. Control 204B for the night scene shooting mode can be used to trigger the increase of brightness and color richness in the captured image. Control 204C for the portrait shooting mode can be used to trigger the electronic device 100 to perform beautification processing on the portrait in the captured image. Control 204D for the shooting mode can be used to trigger the electronic device 100 to capture images using default parameters and process the images captured by the camera using the default image processing flow. Control 204E for the video recording mode can be used to trigger the electronic device 100 to record video using a single camera. Control 204F for the multi-lens recording mode can be used to trigger the electronic device 100 to record video simultaneously using multiple cameras. Control 204G for more modes can be used to trigger the electronic device 100 to display controls for more shooting modes.

[0042] Electronic device 100 can receive input from a user for the zoom ratio control 202 (e.g., clicking the common zoom ratio 1 (e.g., 1x)). In response to this input, the zoom ratio of electronic device 100 can be switched from the currently used zoom ratio to the common zoom ratio 1.

[0043] If the camera application of electronic device 100 sends out a large number of camera requests before electronic device 100 receives user input for zoom ratio control 202, there will be a delay in electronic device 100 processing the zoom ratio change requests. This will cause the preview image of electronic device 100 to lag for a while before displaying the preview image after the zoom ratio has been changed.

[0044] For example, such as Figure 2B As shown, Figure 2B The zoom control 202 in the user interface 210 shown displays the currently used zoom level as 1x. A stutter occurs in the preview box 201, and no preview image is displayed. After the preview box 201 stutters for a period of time, the electronic device 100 can display the user interface 220 showing the zoom level switch from the currently used zoom level to the commonly used zoom level 1.

[0045] like Figure 2C As shown, the user interface 220 of the electronic device 100 may display a preview image 221 obtained after the electronic device 100 switches from the currently used zoom level (e.g., 0.6x) to the commonly used zoom level 1 (e.g., 1x).

[0046] like Figures 2A-2B As shown, when a user zooms in on the camera, there is a delay before the zoomed image frame is displayed on the camera's shooting interface, resulting in a poor user experience.

[0047] Figure 3 This illustration shows a schematic diagram of the hardware and software architecture of a camera service on an electronic device provided in an embodiment of this application.

[0048] like Figure 3 As shown, the layered architecture divides the system into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer, the driver layer, and the hardware layer.

[0049] The application layer can include a series of application packages.

[0050] Application packages can include camera apps, etc.

[0051] The Application Frameworks (FWK) layer provides the application programming interface (API) and programming framework for the application packages in the application layer. The application framework layer includes some predefined functions.

[0052] In some embodiments, the application framework layer may include a camera interface. The camera interface may include an image acquisition interface (e.g., the capture() interface) and a continuous image acquisition interface (e.g., the setRepeatingRequest() interface).

[0053] In some embodiments, the application framework layer may also include a request queue module, which may include camera requests such as Capture request 1 and Capture request 2 issued by the camera application through the capture() interface, as well as continuous capture or preview requests (Repeating Request) issued through the setRepeatingRequest() interface.

[0054] In some embodiments, the application framework layer may further include a request queue processing (RequestThread) module. This request queue processing module may include a WaitForRequest module, a SendRequestsBatch module, and a PrepareHardwareAbstractionLayerRequest (or PrepareHALRequest) module. The WaitForRequest module can be used to keep the camera request (e.g., Capturerequest1) waiting in the request queue module if none of the N buffers allocated to the camera application in the electronic device's buffer are empty. The PrepareHALRequest module can be used to construct the Capturerequest and output buffers of the HAL layer. When there are empty buffers among the N buffers allocated to the camera application, the PrepareHALRequest module can obtain buffers from the AllocateBuffer interface for the camera request sent by the camera application. The Send module can be used to send the CaptureRequest of the HAL layer, which is constructed by the PrepareHALRequest module, to the HAL layer.

[0055] For example, N can be 8, meaning the number of buffers allocated by the electronic device to the camera application for camera requests can be 8. N can also be other values, such as 7 or 9, etc. The embodiments of this application do not limit the value of N. The following description uses the value of N as 8 as an example.

[0056] In some embodiments, the application framework layer may further include an Allocate Buffer interface, which is used to allocate a buffer for camera requests issued by the camera application. For example, the Allocate Buffer interface can specify the address of a buffer for storing the camera request.

[0057] In some embodiments, the application framework layer may further include a display service module, which can be used to acquire image frames uploaded by the HAL layer and display the acquired image frames. When an image frame is successfully displayed, the buffer storing the successfully displayed image frame is cleared. The display service module can notify the Allocate Buffer interface to store the buffer address of the successfully displayed image frame.

[0058] In some embodiments, the application framework layer may also be referred to as the application framework layer.

[0059] The Hardware Abstraction Layer (HAL) is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system.

[0060] The Hardware Abstraction Layer (HAL) can include a camera hardware abstraction layer. This layer can receive camera requests from the application framework layer and send the camera parameters and buffer address from the request to the camera module via the driver layer. It can also call an interface that returns a result (e.g., ProcessCaptureResult) to send the image frames acquired by the camera module according to the camera parameters to the display service in the application framework layer.

[0061] Optionally, the camera hardware abstraction layer may include a camera request waiting module and a camera hardware abstraction processing module. The camera request waiting module and the camera hardware abstraction processing module... Figure 3 Not shown in the image, please refer to the following for information on the camera request waiting module and the camera hardware abstraction processing module. Figure 5 .

[0062] The camera request waiting module can be used to receive camera requests issued by the application framework layer. When the camera hardware abstraction processing module finishes processing a camera request, the camera request waiting module can send a new camera request to the camera hardware abstraction module.

[0063] The camera hardware abstraction processing module can send the camera parameters and the buffer address of the new camera request to the camera module through the driver layer. The camera request processing module can also call the interface that returns the result (e.g., ProcessCaptureResult) to send the image frames acquired by the camera module according to the camera parameters to the display service in the application framework layer.

[0064] The driver layer is the layer between hardware and software. It includes drivers for various hardware components. The driver layer can include camera drivers, image processor drivers, and display drivers, among others. Specifically, the camera driver drives the image sensors of one or more cameras in the camera module (e.g., image sensor 1, image sensor 2, etc.) to acquire images and drives the image signal processor to preprocess the images. The image processor driver drives the graphics processor to process the images. The display driver drives the display.

[0065] The hardware layer may include a camera module, an image signal processor, a display, etc. The camera module may include image sensors from one or more cameras (e.g., image sensor 1, image sensor 2, etc.). Optionally, the camera module may also include a time-of-flight (TOF) sensor, a multispectral sensor, etc. The image signal processor can be used to process the image frames acquired by the camera module. The display can be used to display the image frames sent by the display service module.

[0066] In this embodiment, the request module can be a queue request module. The caching module can be a cache allocation interface module. The camera processing module can be a camera hardware abstraction processing module. The display module can be a display service module.

[0067] The following section, using zoom as an example, describes in detail the processing mechanism of the electronic device 100 based on the camera request generated by the user for changing the focal length, taking the above system architecture as an example:

[0068] ① The electronic device 100 receives the user's operation to open the camera application and sends out multiple camera requests to obtain preview images.

[0069] When electronic device 100 receives an operation from a user to open the camera application (e.g., Figure 1As shown in the diagram, after the user clicks the camera application icon 103, the electronic device 100 launches the camera application. The camera application can issue multiple camera requests to obtain preview images. The first camera request issued by the camera application to obtain preview images can fill the N buffers reserved for camera requests in the cache area.

[0070] Understandably, once the camera application is enabled, it can periodically send requests to the buffer and retrieve the corresponding image frame data based on those requests.

[0071] ② The application framework layer of electronic device 100 places N requests from multiple preview image requests into N buffers in order of their sending time.

[0072] The application framework layer of electronic device 100 can place N requests from multiple preview image requests into N buffers in order of their sending time.

[0073] The application framework layer can sequentially send camera requests from N buffers to the HAL layer. The HAL layer processes the camera requests in the N buffers, informing the camera module of the camera parameters and buffer addresses from each request. The camera module can then acquire image frames according to the camera parameters and store the image frames in the buffer corresponding to the specified buffer address. The HAL layer acquires the image frame and uploads it to the application framework layer, which then displays the image frame. Upon successful display, the image frame is displayed in the preview window of the camera application's shooting interface. After successful display, the buffers containing the image frame and its corresponding camera request can be cleared. The application framework layer can then allocate one of the remaining camera requests from the multiple camera requests to the cleared buffer. The application framework layer will sequentially allocate the remaining camera requests from the multiple camera requests to the cleared buffers among the N buffers.

[0074] It is understandable that the process of the HAL layer handling camera requests can be referred to in steps 3, 4 and 5 below, and will not be repeated here.

[0075] ③ The electronic device 100 receives the user's operation on the camera application and sends a camera request.

[0076] Electronic device 100 can receive user operations on the camera application, such as changing the focal length magnification. Based on this user operation, the camera application in electronic device 100 will issue a camera request. The request queue module in the application framework layer of electronic device 100 can obtain the camera request through the setRepeatingRequest() or capture() interface in the camera interface. When there is an empty buffer among the N buffers reserved for camera requests in the buffer area of ​​electronic device 100, electronic device 100 can send the camera request from the request queue module to the HAL request preparation module in the request queue processing module. After obtaining the buffer address allocated for the camera request by the Allocate Buffer interface, the HAL request preparation module can store the camera request in the buffer. The HAL request preparation module can construct the camera request into a Capturerequest in the HAL layer and send it to the camera request waiting module in the HAL layer through the batch processing request module. When the previous camera request processed by the camera hardware abstraction module has been completed, the camera request waiting module can send the camera request to the camera hardware abstraction processing module. Then, the camera hardware abstraction processing module can send the camera parameters in the new camera request and the buffer address of the camera request to the camera module through the driver layer.

[0077] ④ Electronic device 100 turns on the camera and acquires image frames captured by the camera based on the camera request.

[0078] The camera driver in electronic device 100 can receive camera parameters and the buffer address requested by the camera hardware abstraction processing module. Then, the camera driver can send the camera parameters and the requested buffer address to the camera module. The camera module can acquire image frames according to the camera parameters and store the acquired image frames in the buffer corresponding to the buffer address carried in the camera request.

[0079] In some examples, image frames acquired by the camera module can be transmitted to an image signal processor (ISP). The ISP can preprocess the image frames and then upload them to the HAL layer via the camera driver or the ISP driver. The HAL layer can then upload the image frames to the display service in the application framework layer.

[0080] In some examples, the HAL layer can upload image frames to the camera application through the application framework layer, and the camera application can process the image frame before sending it to the display service.

[0081] ⑤ The electronic device 100 displays image frames.

[0082] The display service can send the image frame to the display, that is, notify the display to display the image frame through the display driver. In other words, the display of electronic device 100 can display the image frame. After the display successfully sends the image frame, it will notify the display service that the image frame has been successfully displayed. It is understandable that after the image frame is sent, the buffer storing the image frame and the buffer used to acquire the image frame in the electronic device's cache will be cleared. The display service can send the address of the cleared buffer to the AllocateBuffer interface. The AllocateBuffer interface can then allocate the address of the cleared buffer to the next camera request.

[0083] For example, such as Figure 3 As shown, when the queue request module issues photo request 1, buffer8 is idle. Therefore, the prepare HAL request module can obtain a buffer from the Allocate Buffer interface for photo request 1. The Allocate Buffer interface can allocate photo request 1 into buffer8. After allocating the buffer, the prepare HAL request module can send photo request 1 to the batch processing request module. Then, the batch processing request module can send photo request 1 to the HAL layer. Photo request 1 can then wait in the HAL layer's request queue for processing.

[0084] When a user changes the zoom level within the camera application's interface, the camera application can issue a photo capture request 2. Since the buffer is full, the request queue module can forward this photo capture request 2 to the waiting request module to wait for an empty buffer. After photo capture request 1 in the prepare HAL request module is forwarded to the HAL layer by the batch request module, the batch request module can notify the waiting request module. Then, the waiting request module can send photo capture request 2 to the prepare HAL request module.

[0085] The HAL layer can first process the requests in buffer1 and return the corresponding image frames to the display service module. The display service module can then display these image frames. Upon successful display, the display service module can return the address of buffer1 to the Allocate Buffer interface. Then, the Allocate Buffer interface can allocate buffer1 to photo request 2. After allocating the buffer, the prepared HAL request can send photo request 2 to the batch processing request module. The batch processing request module can then forward photo request 2 to the HAL layer. Photo request 2 can then wait in the HAL layer's request queue for processing. The HAL layer must process the requests in buffers 2 through 8 in the order they are sent before processing photo request 2.

[0086] According to the camera request processing mechanism described above, if there are spare buffers among the N buffers allocated by electronic device 100 to the camera application for camera requests, the application framework layer of electronic device 100 can continuously send camera requests issued by the camera application to the HAL layer. If there are no spare buffers among the N buffers allocated by electronic device 100 to the camera application for camera requests, the HAL layer needs to successfully process a camera request and return the processing request result (e.g., an image frame obtained according to the camera parameters of the camera request) to the display service of the application framework layer before clearing a buffer.

[0087] When a user opens the camera application, the camera application can send multiple camera requests to acquire preview images. Since the number of buffers used to store these camera requests is limited (e.g., 8), the camera requests sent by the camera application will fill the buffers. When the user clicks the zoom control to switch the current zoom level (e.g., 0.6x) to another zoom level (e.g., 1x), in response to this operation, the camera application of electronic device 100 can send a camera request, which may carry the user's newly selected zoom level (e.g., 1x). Electronic device 100 needs to wait for the HAL to finish processing the camera requests in one buffer and clear that buffer before it can allocate a buffer for the camera request carrying the user's newly selected zoom level and send it to the HAL layer. Thus, if the HAL layer is slow in processing a request, causing a slow frame output, it will not only affect the slow display of the current frame, causing stuttering, but also prevent subsequent camera requests from being sent to the HAL layer in a timely manner, making the stuttering problem increasingly severe. This results in a poor user experience.

[0088] Figure 4An exemplary diagram illustrates the time taken from when electronic device 100 receives a user's operation to change the zoom level, to when electronic device 100 sends the request corresponding to the operation to the HAL layer, and to when electronic device 100 responds to the user's operation and successfully displays the image frame corresponding to the user's operation.

[0089] like Figure 4 As shown in Figure (a), the camera application of electronic device 100 sends an initial camera request (e.g., after being turned on) after being turned on. Figure 4 In request 1) illustrated in (a) of the diagram, the zoom ratio can be 0.6x. Before the user changes the camera's zoom ratio, the camera module of electronic device 100 acquires image frames at a zoom ratio of 0.6x. After the user changes the camera's zoom ratio to 1.0x, the sensors (e.g., touch sensors) in the screen (also called a display, or screen) of electronic device 100 can acquire the user's touch operation. After acquiring the user operation, the camera application of electronic device 100 issues a camera request (e.g., ...) based on the user operation. Figure 4 The example shown is request n(zoom: 1.0 (meaning the zoom ratio is 1.0x)). Taking an electronic device 100 with 8 buffers used to store camera requests as an example, when all 8 buffers store camera requests, the time it takes for a camera request carrying a new zoom ratio issued by the camera application in electronic device 100 to be sent from the application framework layer to the HAL layer is z1. The formula for calculating z1 is as follows:

[0090] z1 = x + y11 + v (Formula 1)

[0091] In Formula 1 above, x represents the time it takes for the screen sensor of electronic device 100 to detect a user operation, and then for the camera application to convert the user operation into a corresponding camera request and send it to the FWK. y11 represents the time it takes for the HAL to process the request in buffer1, obtain the corresponding image frame 401, return the image frame 401 to the FWK, and for the FWK to successfully display the image frame 401. v represents the time it takes for the FWK to allocate a buffer for the zoom request and send the zoom request to the HAL.

[0092] like Figure 4 As shown in Figure (b), the duration of time for the electronic device 100 to receive the user's zoom operation and successfully display the corresponding image frame is T1. The formula for calculating T1 is as follows:

[0093] T1=x+y11+v+y12+y13+y14+y15+y16+y17+y18+y19 (Formula 2)

[0094] In Formula 2 above, y12 represents the time it takes for the HAL to process a request in buffer 2, retrieve the corresponding image frame 402, return image frame 402 to the FWK, and for the FWK to successfully display image frame 402. y13 represents the time it takes for the HAL to process a request in buffer 3, retrieve the corresponding image frame 403, return image frame 403 to the FWK, and for the FWK to successfully display image frame 403. y14 represents the time it takes for the HAL to process a request in buffer 4, retrieve the corresponding image frame 404, return image frame 404 to the FWK, and for the FWK to successfully display image frame 404. y15 represents the time it takes for the HAL to process a request in buffer 5, retrieve the corresponding image frame 405, return image frame 405 to the FWK, and for the FWK to successfully display image frame 405. y16 represents the time it takes for the HAL to successfully display an image frame 406, which corresponds to a request in buffer 6, and to return that image frame 406 to the FWK. y17 represents the time it takes for the HAL to successfully display an image frame 407, which corresponds to a request in buffer 7, and to return that image frame 407 to the FWK. y18 represents the time it takes for the HAL to successfully display an image frame 408, which corresponds to a request in buffer 8, and to return that image frame 408 to the FWK. y19 represents the time it takes for the HAL to successfully display an image frame 409, which corresponds to a zoom request, and to return that image frame 409 to the FWK.

[0095] from Figure 4 As shown in Figures (a) and (b), when the HAL layer processes a request slowly, causing a frame (e.g., image frame 401 corresponding to the request in buffer1) to be displayed slowly, it not only affects the slow display of the current frame, leading to stuttering, but also makes y11 larger. This not only increases the time z1 for the electronic device 100 to receive the zoom operation and send the corresponding zoom request to the HAL, but also increases the total time T1 for the electronic device 100 to respond to the user's zoom operation. Consequently, the camera request generated by the camera application of the electronic device 100 based on the user's zoom operation cannot be sent to the HAL layer in a timely manner, making the stuttering problem increasingly severe. This results in a poor user experience.

[0096] To reduce the latency of camera-related parameters (such as zoom, focus, exposure, and filters) and improve user experience, this application provides a camera request processing method. This method may include: The application framework layer in the electronic device 100 sets P buffers out of N buffers used to store camera requests issued by the camera application as buffers for storing ordinary camera requests, and sets q buffers out of the N buffers as buffers for storing special camera requests. When the camera application of the electronic device 100 is launched, the application framework layer in the electronic device 100 sequentially stores multiple camera requests issued by the camera application into the p buffers. The HAL layer in the electronic device 100 can sequentially send the multiple camera requests stored in the p buffers to the camera module to acquire image frames and return the image frames to the application framework layer for display. When the user's operation on the camera application's shooting interface is received, the camera application of the electronic device 100 sends a first camera request to the application framework layer. When the application framework layer determines that a first parameter in the first camera request has changed, the application framework layer stores the first camera request in buffer 21 of the q buffers. Then, the HAL layer can send the first camera request in buffer21 to the camera module to obtain the corresponding image frame and return it to the application framework layer for display. When the application framework layer determines that the first parameter in the first camera request has not changed, if there are no empty buffers among the p buffers, the application framework layer waits for the HAL layer to process the camera request in buffer11 of the p buffers before storing the first camera request in buffer11 and sending it to the HAL layer. After the HAL layer processes all camera requests before the first camera request, it can send the first camera request to the camera module to obtain the image frame and return it to the application framework layer for display. The first parameter may include focal length, focus, exposure, filters, etc.

[0097] In this application embodiment, compared with the previous camera request, a camera request in which the focal length, focus, exposure, filter and other parameters have not changed can be called a normal camera request; a camera request in which the focal length, focus, exposure, filter and other parameters have changed can be called a special camera request.

[0098] Figure 5 This application provides an exemplary embodiment of the hardware and software architecture for camera services on an electronic device 100.

[0099] like Figure 5As shown, the electronic device 100 system can be divided into five layers, from top to bottom: application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer. The driver layer and hardware layer are not listed below. Figure 5 As shown in the diagram, other modules contained in the driver layer and hardware layer, as well as their connections with other layers, can be found in [reference needed]. Figure 3 The application framework layer's cache allocation interface can divide the cache used to store camera requests into a general camera request storage area and a special camera request storage area.

[0100] For detailed descriptions of the application layer, application framework layer, hardware abstraction layer, driver layer, and hardware layer, please refer to [link to relevant documentation]. Figure 3 The description in the text will not be repeated here.

[0101] See Figure 5 Based on the system architecture of electronic devices, a camera request processing method provided in this application embodiment may include:

[0102] 1. Electronic device 100 receives the user's operation to open the camera application and sends out multiple camera requests R1 to obtain preview images.

[0103] When electronic device 100 receives an operation from a user to open the camera application (e.g., Figure 1 As shown in the diagram, after the user clicks the camera application icon 103, the electronic device 100 launches the camera application. The camera application can issue multiple camera requests for acquiring preview images (e.g., camera request R11, camera request R12, ..., camera request R1m). When the user does not operate the controls for setting focal length, focus, exposure, and filter parameters, the focal length, focus, exposure, and filter parameters carried in the camera requests issued by the camera application will not change. Because the application framework layer sets p of the N buffers used to store camera requests issued by the camera application as buffers for storing ordinary camera requests, and sets q of the N buffers as buffers for storing special camera requests, the multiple camera requests initially issued by the camera application will not fill all N buffers.

[0104] For example, such as Figure 5As shown, the electronic device 100 can have eight buffers for storing camera requests: buffer1, buffer2, buffer3, buffer4, buffer5, buffer6, buffer7, and buffer8. In the application framework layer of the electronic device 100, buffer1, buffer2, buffer3, and buffer4 can be configured to store ordinary camera requests. The application framework layer can also configure buffer5, buffer6, buffer7, and buffer8 to store special camera requests.

[0105] 2. The application framework layer of electronic device 100 places P camera requests from multiple camera requests for obtaining preview images into P buffers in order of their sending time.

[0106] The application framework layer of electronic device 100 can place P requests from multiple camera requests into P buffers in order of their sending time.

[0107] The application framework layer can sequentially send camera requests from P buffers to the HAL layer. The HAL layer processes the camera requests in the P buffers, informing the camera module of the camera parameters and buffer addresses from each request. The camera module can then acquire image frames according to the camera parameters and store the image frames in the buffer corresponding to the specified buffer address. The HAL layer acquires the image frame and uploads it to the application framework layer, which then displays the image frame. Upon successful display, the image frame is displayed in the preview window of the camera application's shooting interface. After successful display, the buffers containing the image frame and its corresponding camera request in the P buffers can be cleared. The application framework layer can then allocate one of the remaining camera requests from multiple camera requests R1 to the cleared buffer. The application framework layer will sequentially allocate the remaining camera requests from multiple camera requests R1 to the cleared buffers in the P buffers.

[0108] For example, such as Figure 5As shown, the application framework can initially distribute four of the multiple camera requests for acquiring preview images into buffer1, buffer2, buffer3, and buffer4 sequentially. In some examples, once the camera application is open, it can continuously send camera requests for acquiring preview images, up to 30 requests per second. After a camera request in buffer1 is processed, the application framework can send another camera request to buffer1. Similarly, after a camera request in buffer2 is processed, the application framework can send another camera request to buffer2. Buffer1, buffer2, buffer3, and buffer4 can sequentially store the multiple camera requests for acquiring preview images sent by the camera application.

[0109] 3. Electronic device 100 receives user operations related to the camera application and sends a camera request R2.

[0110] Electronic device 100 can receive user operations on the camera application, such as changing the focal length magnification. Based on this user operation, the camera application in electronic device 100 will issue a camera request R2. The request queue module in the application framework layer of electronic device 100 can obtain the camera request R2 through the setRepeatingRequest() or capture() interface in the camera interface.

[0111] 4. If the application framework layer of electronic device 100 determines that the first parameter in camera request R2 has changed, then the camera request is allocated to buffer(p+1) of Q buffers; if the application framework layer of electronic device 100 determines that the first parameter in camera request R2 has not changed, then after buffer1 of P buffers is cleared, the camera request R2 is allocated to buffer1.

[0112] The application framework layer of electronic device 100 can determine that a first parameter (e.g., zoom ratio) in camera request R2 has changed. For example, the request queue processing module in the application framework layer can compare camera request R2 with the previous camera request. If the zoom ratio in the previous camera request was 0.6x and the zoom ratio in camera request R2 is 1.0x, then the request queue processing module can confirm that the first parameter in camera request R2 has changed. The allocation buffer interface in the application framework layer can allocate camera request R2 to any buffer in a special camera request buffer, such as buffer5. Since multiple camera requests issued when the camera is turned on are not allocated to buffers 5, 6, 7, and 8, and buffers 5, 6, 7, and 8 are all empty buffers (i.e., no camera requests have been stored yet), camera request R2 can be directly allocated to buffer 5 without waiting for the buffer to be cleared before allocation.

[0113] If the application framework layer of electronic device 100 determines that the first parameter in camera request R2 has not changed, it waits for buffer1 of the P buffers to be cleared before allocating camera request R2 to buffer1. For example, the request queue processing module in the application framework layer can compare camera request R2 with the previous camera request. If the zoom ratio, focus, exposure, filter, and other parameters in the previous camera request are the same as those in camera request R2, the request queue processing module can confirm that the first parameter in camera request R2 has not changed. The allocation buffer interface in the application framework layer can allocate camera request R2 to the buffer in the ordinary camera request buffer. Since multiple camera requests issued when the camera application is activated will fill the buffers in the ordinary camera request buffer (i.e., buffer1, buffer2, buffer3, and buffer4 all store camera requests), it is necessary to wait for one of buffers (buffer1, buffer2, buffer3, and buffer4) to be cleared before allocating camera request R2 to that cleared buffer.

[0114] 5. Electronic device 100 turns on the camera and acquires image frames captured by the camera based on the camera request.

[0115] The camera driver in electronic device 100 can receive camera parameters and the buffer address requested by the camera hardware abstraction processing module. Then, the camera driver can send the camera parameters and the requested buffer address to the camera module. The camera module can acquire image frames according to the camera parameters and store the acquired image frames in the buffer corresponding to the buffer address carried in the camera request.

[0116] In some examples, image frames acquired by the camera module can be transmitted to an image signal processor (ISP). The ISP can preprocess the image frames and then upload them to the HAL layer via the camera driver or the ISP driver. The HAL layer can then upload the image frames to the display service in the application framework layer.

[0117] In some examples, the HAL layer can upload image frames to the camera application through the application framework layer, and the camera application can process the image frame before sending it to the display service.

[0118] 6. Electronic devices 100 display image frames.

[0119] The display service can send the image frame to the display, that is, notify the display to display the image frame through the display driver. In other words, the display of electronic device 100 can display the image frame. After the display successfully sends the image frame, it will notify the display service that the image frame has been successfully displayed. It is understandable that after the image frame is sent, the buffer storing the image frame and the buffer used to acquire the image frame in the electronic device's cache will be cleared. The display service can send the address of the cleared buffer to the AllocateBuffer interface. The AllocateBuffer interface can then allocate the address of the cleared buffer to the next camera request.

[0120] For example, such as Figure 5 As shown, when the queue request module issues photo capture request 1, buffer 4 in the ordinary camera request buffer is idle. Therefore, the prepare HAL request module can obtain a buffer from the Allocate Buffer interface for photo capture request 1. The Allocate Buffer interface can allocate photo capture request 1 into buffer 4. After allocating the buffer, the prepare HAL request module can send photo capture request 1 to the batch processing request module. Then, the batch processing request module can send photo capture request 1 to the HAL layer. Photo capture request 1 can then wait in the HAL layer's request queue for processing.

[0121] When a user can change the zoom level within the camera application's interface, the camera application can issue a photo capture request 2. Since camera requests upon camera activation are only allocated to the regular camera request buffer, the buffer in the special camera request buffer is not full. The queued request module determines that the zoom level in photo capture request 2 is different from that in photo capture request 1. Therefore, the Allocate Buffer interface can allocate buffer 5 from the special camera request buffer to photo capture request 2, and then send it to the HAL layer. Photo capture request 2 can then wait in the HAL layer's request queue for processing. The HAL layer needs to process the requests in buffers 1 through 4 according to the request sending order before processing photo capture request 2.

[0122] When the user doesn't change the shooting mode in the camera's shooting interface, the camera parameters for each preview image are the same, and the user won't experience any stuttering between frames. However, when the user changes the zoom level, focuses, adjusts the exposure, or adds filters in the camera's shooting interface, the current shooting mode is changed, and the camera parameters in the preview images will change accordingly. That is, one or more of the camera parameters such as zoom level, focus, exposure, and filters will differ between the preview images before and after the user's operation. If the electronic device's response time is too long after the user's operation, the user will experience a noticeable stuttering effect when seeing the preview image in the new shooting mode, negatively impacting the user experience.

[0123] Implementing the camera request processing method provided in this application embodiment, multiple camera requests for acquiring preview images issued when the camera is turned on are only stored in a portion of the cache (e.g., Figure 5 In the buffers (buffer1, buffer2, buffer3, buffer4) shown in the diagram, another part of the buffer (e.g., ...) is used for the request buffers of a normal camera. Figure 5The special camera request buffers (buffer5, buffer6, buffer7, and buffer8) shown are only allocated to camera requests whose first parameter changes. When a user changes the zoom level, focuses, adjusts the exposure, or adds filters in the camera's shooting interface, the electronic device 100 can directly allocate the camera request generated based on the user's operation to the special camera request buffer. The application framework layer does not need to wait for a buffer to be cleared before allocating to the buffer, nor does it need to wait for the HAL to process the next request before allocating a buffer to the camera request. In this way, the response time of the electronic device 100 to user operations such as changing the zoom level, focusing, adjusting the exposure, or adding filters can be reduced, thereby improving the response efficiency of user operations.

[0124] In this embodiment, the special camera request buffer can be referred to as the first buffer, and buffer5, buffer6, buffer7, and buffer8 can all be referred to as the first buffer. The ordinary camera request buffer can be referred to as the second buffer. buffer1, buffer2, buffer3, and buffer4 can also be referred to as the second buffer.

[0125] The following is combined with Figure 5 The software framework shown takes, for example, a user changing the current zoom level in the camera application's shooting interface after the camera application is launched, to describe in detail a camera request processing method provided in the embodiments of this application.

[0126] Figure 6 The diagram illustrates the software module interaction of a camera request processing method according to an embodiment of this application.

[0127] like Figure 6 As shown, the software framework of the electronic device 100 may include a camera application, an application framework layer (FWK), and a hardware abstraction layer (HAL). The application framework layer may include a camera interface module 610, a request queue module 620, a request queue processing module 630, an allocation buffer interface module 640, and a display service module 650. The hardware abstraction layer may include a camera hardware abstraction layer (Camera HAL) 660.

[0128] The interaction process between the software modules of the electronic device 100 may include the following steps:

[0129] S601. The camera application detects an operation to launch the camera and launches the camera application.

[0130] Camera apps can detect actions that launch the camera, for example, Figure 1 The user shown can click the camera app icon 103. In response to this user action, the camera app is launched, and the camera app's shooting interface is displayed, for example... Figure 2A The shooting interface 200 is shown in the image.

[0131] S602. The camera application informs the camera mode (photo mode or video mode) to the buffer allocation interface module 640.

[0132] Optionally, the camera application can inform the allocation buffer interface module 640 of the camera mode (photo mode or video mode).

[0133] like Figure 2A As shown, the camera application displays the shooting interface 200 after it is opened, as shown below. Figure 2A As shown, control 204D for normal photo-taking mode is selected, and the camera application is in photo-taking mode. When the camera application is in photo-taking mode, it can inform the allocation buffer interface module 640 that the camera application's camera mode is photo-taking mode.

[0134] like Figure 7 As shown, the electronic device 100 can receive input operations from the user's selection of the recording mode control 204E (e.g., clicking the control 204E), and in response to the input operation, the electronic device 100 can switch from the photo mode to the recording mode.

[0135] like Figure 8 As shown, the electronic device 100 can display a shooting interface 800. The video recording mode control 204E in the shooting interface 800 is selected, and the camera application of the electronic device 100 is in video recording mode. After switching to normal video recording mode, the electronic device 100 can display a preview image 802 obtained in real-time by the camera in normal video recording mode in the preview frame 801, and replace the aforementioned shooting control 203B with a recording start control 203D. After switching to normal video recording mode, the electronic device 100 can display one or more functional controls (e.g., flash control 803A, settings control 803B, etc.) on the shooting interface 800. The flash control 803A can be used to trigger the electronic device 100 to turn the flash on or off. The settings control 803B can be used to set the shooting parameters of the electronic device 100 (e.g., image size, image storage format, etc.).

[0136] When the camera application switches from photo mode to video mode, the camera application can inform the allocation buffer interface module 640 that the camera application's camera mode is video mode.

[0137] For example, a camera application can indicate its camera mode by sending a command to the allocation buffer interface module 640. Specifically, the camera application can send command 1 to the allocation buffer interface module 640 to indicate that the camera mode is photo mode. The camera application can send command 2 to the allocation buffer interface module 640 to indicate that the camera mode is video recording mode. For example, the content of command 1 can be "1" and the content of command 2 can be "0". This embodiment of the application does not limit the specific content of command 1 and command 2.

[0138] S603. The buffer allocation interface module 640, based on the camera mode, sets p buffers out of the N buffers used to store camera requests to store ordinary camera requests, and q buffers to store special camera requests.

[0139] The buffer allocation interface module 640 can, based on the camera mode, set p buffers out of the N buffers used to store camera requests to store ordinary camera requests, and q buffers to store special camera requests. The values ​​of p and q can be different in different camera modes. That is, the values ​​of p and q in the photo capture mode can be different from the values ​​of p and q in the video recording mode.

[0140] For example, taking N as 8, in photo mode, p can be 4 and q can be 4. In video mode, p can be 5 and q can be 3.

[0141] Understandably, in photo mode, the camera application continuously sends requests to obtain the preview data stream (i.e., preview image frames). It only needs to obtain the image frame at the moment the user presses the shutter button (also known as the shutter button). In video mode, after the user clicks the start recording button, the camera application continuously sends two types of requests: one for obtaining the preview data stream and the other for obtaining the video data stream (i.e., video frames). Therefore, in video mode, to ensure smooth operation of the camera application, more buffer space needs to be reserved for regular camera requests. Consequently, the value of p in video mode is greater than its value in photo mode, and the value of q in video mode is less than its value in photo mode.

[0142] Optionally, in one possible implementation, the camera application may not inform the allocation buffer interface module 640 of its camera mode. The allocation buffer interface module 640 can determine the values ​​of p and q based on the type of camera request received by the request queue module 620. When the request queue module 620 receives two types of camera requests—one for acquiring a preview data stream and the other for acquiring a recording data stream—the allocation buffer interface module 640 can determine that the value of p is p1 and the value of q is q1. When the request queue module 620 receives only one type of camera request—a request for acquiring a preview data stream—the allocation buffer interface module 640 can determine that the value of p is p2 and the value of q is q2.

[0143] For example, when N equals 8, p1 equals 5 and q1 equals 3. p2 equals 4 and q2 equals 4. The values ​​of N, p1, q1, p2, and q2 are not limited in the embodiments of this application.

[0144] S604. The camera application sends multiple camera requests (camera request R11, camera request R12, ..., camera request R1m).

[0145] The camera application can send multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) to the camera interface module 610. For example, camera request R11 can be one of the above... Figure 3 or Figure 5 The photo request 1 is shown in the image.

[0146] S605. The camera interface module 610 sends multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) issued by the camera application to the request queue module 620.

[0147] The camera interface module 610 may include a capture() interface and a setRepeatingRequest() interface. When multiple camera requests are issued by the camera application for continuously acquiring preview data streams, the camera interface module 610 can obtain the multiple camera requests issued by the camera application through the setRepeatingRequest() interface.

[0148] The camera interface module 610 can send multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) issued by the camera application to the request queue module 620.

[0149] S606a. The request queue module 620 sends multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) to the request queue processing module 630.

[0150] The request queue module 620 can send multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) to the request queue processing module 630 one by one in the order of request delivery.

[0151] Optionally, before the request queue module 620 sends multiple camera requests to the request queue processing module 630, the request queue module 620 can also determine whether the multiple camera requests are ordinary camera requests or special camera requests. If the request queue module 620 determines that the multiple camera requests are ordinary camera requests, the request queue module 620 can send the multiple camera requests (camera request R11, camera request R12, ..., camera request R1m) to the request queue processing module 630 one by one according to the request sending order.

[0152] Optionally, the request queue module 620 may also skip the judgment and directly assume that the camera request sent by the camera application when it is turned on is a normal camera request.

[0153] S606b. The request queue processing module 630 obtains buffers from the buffer allocation interface module 640 for camera request R11, camera request R12, ..., camera request R1p.

[0154] S606c. Camera requests R1(p+1)... and R1m in the request queue processing module 630 are waiting for buffer allocation.

[0155] The request queue processing module 630 can acquire buffers for multiple camera requests one by one according to the order in which the requests are sent. Since the number of buffers used to store camera requests is limited, the request queue processing module 630 acquires buffers from the buffer allocation interface module 640 for camera requests R11, R12, ..., R1p. Camera requests R1(p+1), ..., R1m in the request queue processing module 630 are waiting for buffer allocation.

[0156] The request queue processing module 630 may include a waiting request module and a preparing HAL request module. The preparing HAL request module can obtain buffers from the buffer allocation interface module 640 for camera requests R11, R12, ..., R1p. Camera requests R1(p+1), ..., R1m can wait for buffer allocation in the waiting request module.

[0157] S607. The buffer allocation interface module 640 allocates camera request R11, camera request R12, ..., camera request R1p into p buffers.

[0158] The buffer allocation interface module 640 can allocate camera requests R11, R12, ..., R1p into p buffers one by one. For example, as... Figure 5 As shown, p is 4, meaning that there are 4 buffers (buffer1, buffer2, buffer3, and buffer4) in the ordinary camera request buffer area. The buffer allocation interface module 640 can allocate camera request R11 to buffer1, camera request R12 to buffer2, camera request R13 to buffer3, and camera request R14 to buffer4.

[0159] Optionally, the buffer allocation interface module 640 can allocate buffers for camera request 11, camera request R12, ..., camera request R1p, and return the address of the buffer used to store each camera request to the request queue processing module 630. The request queue processing module 630 can store camera request 11, camera request R12, ..., camera request R1p into the buffer corresponding to each camera request according to the buffer address.

[0160] S608. The buffer allocation interface module 640 sequentially sends camera request R11, camera request R12, ..., camera request R1p to the camera hardware abstraction layer 660.

[0161] The buffer allocation interface module 640 sequentially sends the camera requests in p buffers to the camera hardware abstraction layer 660.

[0162] In one possible implementation, the buffer allocation interface module 640 can sequentially send p buffer addresses for storing camera requests R11, R12, ..., R1p to the camera hardware abstraction layer 660. The camera hardware abstraction layer 660 can then retrieve the camera requests one by one from the corresponding buffers according to their buffer addresses.

[0163] Optionally, in another possible implementation, the request queue processing module 630 further includes a batch request distribution module. The buffer allocation interface module 640 can allocate buffers for camera requests R11, R12, ..., R1p, and return the address of the buffer used to store each camera request to the request queue processing module 630. Then, the batch request distribution module can distribute the buffer address for each camera request (R11, R12, ..., R1p) to the camera hardware abstraction layer 660. The camera hardware abstraction layer 660 can then retrieve the camera requests one by one from the corresponding buffer according to their buffer addresses.

[0164] For example, such as Figure 5 The diagram shows a typical camera request buffer, which contains four buffers (buffer1, buffer2, buffer3, and buffer4). The buffer allocation interface module 640 can allocate camera request R11 to buffer1, camera request R12 to buffer2, camera request R13 to buffer3, and camera request R14 to buffer4. The batch processing request module can send the addresses of buffer1, buffer2, buffer3, and buffer4 to the camera hardware abstraction layer 660. The camera hardware abstraction layer 660 can retrieve camera request R11 from buffer1 according to the address of buffer1. After processing camera request R11, the camera hardware abstraction layer 660 can retrieve camera request R12 from buffer2 according to the address of buffer2. The camera hardware abstraction layer 660 can sequentially retrieve the camera requests stored in the buffers according to their addresses.

[0165] S609. The camera hardware abstraction layer 660 obtains the image frame 1 corresponding to camera request R11 stored in buffer1 of p buffers through the camera.

[0166] The camera hardware abstraction layer 660 can acquire image frame 1 corresponding to camera request R11 stored in buffer1 of the N1 buffers. Specifically, as shown... Figure 5 As shown, the camera hardware abstraction layer 660 may include a camera request waiting module and a camera hardware abstraction processing module. Camera requests awaiting processing can be queued in the camera request waiting module. The camera hardware abstraction processing module can send the camera parameters from the camera request R11 being processed to the camera module via the camera driver. The camera module can then acquire image frame 1 according to these camera parameters.

[0167] The camera hardware abstraction layer 660 can send the address of buffer1 and camera parameters to the camera module, and the image frame 1 acquired by the camera module can be stored in buffer1. For example, buffer1 can be... Figure 5 The buffer1 shown in the image.

[0168] In one possible implementation, the camera module can send the acquired image frame 1 to the camera hardware abstraction layer 660.

[0169] In another possible implementation, the camera module can notify the camera hardware abstraction layer 660 that image frame 1 has been successfully acquired. Upon receiving the notification, the camera hardware abstraction layer 660 can retrieve image frame 1 from buffer 1.

[0170] S610. The camera hardware abstraction layer 660 returns image frame 1 to the display service module 650.

[0171] The camera hardware abstraction layer 660 can return image frame 1 to the display service module 650.

[0172] In one possible implementation, the camera hardware abstraction layer 660 can directly send the acquired image frame 1 to the display service module 650.

[0173] Alternatively, in another possible implementation, the camera hardware abstraction layer 660 can send the address of the buffer storing image frame 1 to the display service module 650.

[0174] S611. Display service module 650 sends display image frame 1.

[0175] The display service module 650 can send image frame 1 to the display via the display driver. Optionally, the display service module 650 can send the address of buffer 11 corresponding to the image frame 1 to be sent to the display via the display driver, and the display can obtain the image frame from buffer 1 and send it for display.

[0176] For example, image frame 1 can be as follows: Figure 7 The preview image 205 shown in the image can also be... Figure 8 The preview image 802 shown is shown in the image.

[0177] In this embodiment of the application, the buffer corresponding to the image frame can refer to the buffer that stores the image frame.

[0178] S612. The display service module 650 returns the address of buffer11 to the buffer allocation interface module 640.

[0179] Once image frame 1 is successfully displayed, the display can notify the display service module 650 via the display driver. The electronic device 100 can then clear buffer 1. The display service module 650 can then return the address of buffer 1 to the buffer allocation interface module 640.

[0180] S613a. The buffer allocation interface module 640 allocates buffer1 to the camera request R1(p+1).

[0181] The buffer allocation interface module 640 can allocate buffer1 to camera request R1(p+1). The electronic device 100 can process multiple camera requests according to the steps S608-S612 described above.

[0182] S613b. Camera requests R1(p+2)... and R1m in the request queue processing module 630 are waiting for buffer allocation.

[0183] Once camera request R1(p+1) is allocated to the buffer, the remaining camera requests, namely camera request R1(p+2)... and camera request R1m, can continue to wait for buffer allocation.

[0184] S614. The camera application detected an operation that changed the camera's current zoom level.

[0185] Camera apps can detect operations that change the camera's current zoom level, for example, Figure 2A The image shows a user clicking the zoom ratio 1x control in zoom ratio control 202.

[0186] S615. The camera application sends a camera request R2 to the camera interface module 610.

[0187] The camera application can send a camera request R2 to the camera interface module 610.

[0188] S616. The camera interface module 610 sends the camera request R2 to the request queue module 620.

[0189] The request queue module 620 can obtain the camera request R2 through the setRepeatingRequest() or capture() interface in the camera interface module 610.

[0190] S617. Request queue module 620 determines that the zoom ratio in camera request R2 has changed.

[0191] The request queue module 620 can determine that the zoom ratio in camera request R2 has changed. For example, the request queue processing module 620 can compare camera request R2 with the previous camera request. If the zoom ratio in the previous camera request was 0.6x and the zoom ratio in camera request R2 is 1.0x, then the request queue processing module can confirm that the zoom ratio in camera request R2 has changed.

[0192] S618. The request queue module 620 requests the buffer allocation interface module 640 to obtain a buffer for the camera from R2.

[0193] The request queue module 620 can directly request the buffer from the buffer allocation interface module 640 for the camera R2 to obtain the buffer.

[0194] S619. The buffer allocation interface module 640 allocates the camera request R2 to buffer (P+1) in q buffers.

[0195] The buffer allocation interface module 640 can allocate the camera request R2 to buffer(P+1) out of q buffers. For example, buffer(P+1) can be... Figure 5 The buffer shown is buffer5. Buffer (P+1) can also be called the first buffer.

[0196] Optionally, when there is an empty buffer among the p buffers, the buffer allocation interface module 640 can also allocate the camera request R2 to an empty buffer among the p buffers.

[0197] S620. The buffer allocation interface module 640 sequentially sends camera request R2 to the camera hardware abstraction layer 660.

[0198] The buffer allocation interface module 640 can send camera request R2 to the camera hardware abstraction layer 660. Step S620 can be referred to the description in step S608, and will not be repeated here.

[0199] S621. The camera hardware abstraction layer 660 obtains the image frame 2 corresponding to the camera request R2 through the camera.

[0200] The camera hardware abstraction layer 660 can obtain image frame 2 corresponding to camera request R2 through the camera. For example... Figure 5 As shown, the camera hardware abstraction layer 660 may include a camera request waiting module and a camera hardware abstraction processing module. Camera requests to be processed can be queued in the camera request waiting module. That is, the camera hardware abstraction layer processes the camera requests in the previously issued p buffers before processing camera request R2. The camera hardware abstraction processing module can send the camera parameters from the camera request R2 being processed to the camera module through the camera driver. The camera module can then acquire image frame 2 according to these camera parameters.

[0201] Step S621 can be referred to the description in step S609, and will not be repeated here.

[0202] S622. The camera hardware abstraction layer 660 returns image frame 2 to the display service module 650.

[0203] The camera hardware abstraction layer 660 can return image frame 2 to the display service module 650.

[0204] In one possible implementation, the camera hardware abstraction layer 660 can directly send the acquired image frame 2 to the display service module 650.

[0205] Alternatively, in another possible implementation, the camera hardware abstraction layer 660 can send the address of the buffer storing image frame 2 to the display service module 650.

[0206] S623. Display service module 650 sends display image frame 2.

[0207] The display service module 650 can send image frame 2 to the display via the display driver. Optionally, the display service module 650 can send the address of buffer 22 corresponding to the image frame 2 to be sent to the display via the display driver, and the display can obtain the image frame from buffer 22 and send it for display.

[0208] For example, image frame 1 can be as follows: Figure 2C The preview image 221 shown in the image can also be... Figure 9 The preview image 901 shown is shown in the image.

[0209] In this embodiment of the application, camera request R2 and photo request 2 can be referred to as the first camera request, and camera request R11 and photo request 1 can be referred to as the second camera request.

[0210] It is understood that steps S614-S623 described above also apply to user focusing operations, exposure, adding filters, and other operations in the camera application. The request queue module 620 can distinguish zoom parameters, focus parameters, exposure parameters, and filter parameters from the identifiers carried in the camera request. For example, when the camera request carries the identifier android.control.zoomRatio, the request queue module 620 can determine that the identifier android.control.zoomRatio indicates zoom parameters. When the zoomRatio value in the preceding camera request R2 is 0.6x, and the zoomRatio value in camera request R2 is 1.0x, the request queue module 620 can determine that the zoom ratio in camera request R2 has changed.

[0211] When the camera request carries the identifier android.control.aeMode, the request queue module 620 can determine that the identifier android.control.aeMode is used to indicate exposure parameters. When the camera request carries the identifier android.control.afRegions, the request queue module 620 can determine that the identifier android.control.afRegions is used to indicate focus parameters. When the camera request carries the identifier android.control.effectMode, the request queue module 620 can determine that the identifier android.control.effectMode is used to indicate filter parameters.

[0212] Implementing the camera request processing method provided in this application embodiment, multiple camera requests for acquiring preview images issued when the camera is turned on are only stored in a portion of the cache (e.g., Figure 5 as well as Figure 10 Figures (a) and (b) show the common camera request buffers (buffer1, buffer2, buffer3, buffer4), and another part of the buffer (e.g., Figure 5 as well as Figure 10 The special camera request buffers (buffer5, buffer6, buffer7, and buffer8) shown in Figures (a) and (b) are allocated only to camera requests whose first parameter (e.g., zoom level, focus parameter, exposure parameter, filter parameter, etc.) changes. This allows the application framework layer to manage the number of requests when the camera first issues a request, according to the camera mode. For example, in photo mode, the application framework layer can allocate only four buffers to camera requests 1, 2, 3, and 4 for each of the multiple camera requests that acquire preview images. In video mode, the application framework layer can allocate only five buffers to camera requests 1, 2, 3, 4, and 5 for each of the multiple camera requests that acquire preview images. The initial request from the camera application will not fill the buffer queue. When a user changes the zoom level, focuses, adjusts exposure, or adds filters in the camera's shooting interface, the electronic device 100 can directly allocate the camera request generated based on that user action to a special camera request buffer. The application framework layer does not need to wait for a buffer to be cleared before allocating the request, nor does it need to wait for the HAL to process the next request before allocating a buffer for the camera request. For example, as... Figure 10As shown in Figure (b), when the camera APK sends a zoom request, since the initial stream (i.e., the first request) does not fill the bufferQueue, the camera FWK can immediately acquire an empty buffer and send it to the HAL. Because the FWK immediately sends the zoom request to the HAL, the HAL can quickly apply the zoom request to the generated image frame after receiving it, thereby reducing the response time of the electronic device 100 to the user's zoom operation, which improves the responsiveness of the user's zoom operation and enhances the user experience.

[0213] It is understood that the user interface diagrams provided in the embodiments of this application are for reference only. Figure 1 , Figures 2A-2C ,as well as Figures 7-9 The user interface shown may include more or fewer interface elements, and the positions of the interface elements in the user interface may not be limited to those shown in the above figure.

[0214] Figure 11 An example is shown in which the electronic device 100 is configured according to the above. Figure 5 ,as well as Figure 6 The diagram illustrates the time taken for the electronic device 100 to send the corresponding request to the HAL layer from receiving the user's operation to change the zoom ratio, to successfully responding to the user's operation and displaying the corresponding image frame.

[0215] like Figure 11 As shown in Figure (a), the camera application of electronic device 100 sends an initial camera request (e.g., after being turned on) after being turned on. Figure 11 In request 1) shown in Figure (a), the zoom ratio can be 0.6x. Before the user changes the camera's zoom ratio, the camera module of electronic device 100 acquires image frames at a zoom ratio of 0.6x. After the user changes the camera's zoom ratio to 1.0x, the sensors (e.g., touch sensors) in the screen (also called a display or screen) of electronic device 100 can acquire the user's touch operation. After acquiring the user operation, the camera application of electronic device 100 issues a camera request (e.g., ...) based on the user operation. Figure 11The example shown is request n(zoom: 1.0 (indicating a zoom ratio of 1.0x)). Taking an electronic device 100 with 8 buffers in its cache for storing camera requests as an example, when buffers 1-4 are used to store ordinary camera requests and buffers 5-8 are used to store special camera requests, the time it takes for a camera request carrying a new zoom ratio issued by the camera application in electronic device 100 to be sent from the application framework layer to the HAL layer is z2. The formula for calculating z2 is as follows:

[0216] z² = x + v (Formula 3)

[0217] In Formula 3 above, x represents the time it takes for the screen sensor of electronic device 100 to detect a user operation, and then for the camera application to convert that user operation into a corresponding camera request and send it to the FWK. v represents the time it takes for the FWK to allocate a buffer for the zoom request and send the zoom request to the HAL.

[0218] like Figure 11 As shown in Figure (b), the duration for which the electronic device 100 receives the user's zoom operation and successfully displays the corresponding image frame is T2. The formula for calculating T2 is as follows:

[0219] T2=x+v+y21+y22+y23+y24+y25 (Formula 4)

[0220] In Formula 4 above, y21 represents the time it takes for the HAL to process a request in buffer 1, retrieve the corresponding image frame 1101, return image frame 1101 to the FWK, and for the FWK to successfully display image frame 1101. y22 represents the time it takes for the HAL to process a request in buffer 2, retrieve the corresponding image frame 1102, return image frame 1102 to the FWK, and for the FWK to successfully display image frame 1102. y23 represents the time it takes for the HAL to process a request in buffer 3, retrieve the corresponding image frame 1103, return image frame 1103 to the FWK, and for the FWK to successfully display image frame 1103. y24 represents the time it takes for the HAL to process a request in buffer 4, retrieve the corresponding image frame 1104, return image frame 1104 to the FWK, and for the FWK to successfully display image frame 1104. y25 is for HAL to process zoom requests, obtain the image frame 1105 corresponding to the zoom request, return the image frame 1105 to FWK, and the duration for FWK to successfully send the image frame 1105 to display.

[0221] and Figure 4 Compared to T1 shown in the figure, z1 and T1 Figure 11 In this case, z2 is less than z1, and T2 is less than T1. That is, in electronic device 100 according to the above... Figure 5 ,as well as Figure 6 In the case of handling zoom requests, the camera request processing method shown reduces the time from when the electronic device 100 receives the user's operation to change the zoom ratio, to when the electronic device 100 sends the corresponding request to the HAL layer, and to when the electronic device 100 successfully responds to the user's operation and displays the corresponding image frame. Implementing the camera request processing method provided in this application embodiment, when the camera APK sends a zoom request, since the initial stream (i.e., the initial request) does not fill the bufferQueue, the camera FWK can immediately obtain an empty buffer and send it to the HAL. Because the FWK immediately sends the zoom request to the HAL, the HAL can quickly apply the zoom request to the generated image frame, thereby reducing the time for the electronic device 100 to respond to the user's zoom operation, thus improving the responsiveness of the user's zoom operation and enhancing the user experience.

[0222] The following describes an exemplary electronic device 100 provided in an embodiment of this application.

[0223] Figure 12 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application.

[0224] The following detailed description uses electronic device 100 as an example. It should be understood that electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0225] Electronic device 100 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0226] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0227] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0228] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0229] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0230] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0231] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0232] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0233] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0234] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0235] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0236] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0237] The SIM interface can be used to communicate with the SIM card interface 195 to transmit data to or read data from the SIM card.

[0238] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0239] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0240] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0241] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0242] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0243] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0244] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0245] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0246] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0247] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0248] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0249] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0250] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0251] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0252] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0253] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0254] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0255] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0256] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0257] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as facial recognition, fingerprint recognition, mobile payment, etc.). The data storage area may store data created during the use of electronic device 100 (such as facial information template data, fingerprint information templates, etc.). Furthermore, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0258] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0259] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0260] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0261] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0262] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0263] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0264] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0265] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0266] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0267] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0268] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0269] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0270] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0271] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0272] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0273] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0274] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0275] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0276] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0277] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0278] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, and other SIM cards. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication.

[0279] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0280] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0281] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0282] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A camera request processing method, characterized in that, The method is applied to an electronic device equipped with a camera module, and the method includes: detecting a user's operation of changing camera parameters and sending a first camera request; The values ​​of q and p are determined based on the camera mode of the camera application in the electronic device. The storage area in the electronic device used to store camera requests is divided into a first buffer and a second buffer. The first buffer is used to store the first camera request, and the second buffer is used to store the second camera request. The first buffer includes q buffers, and the second buffer includes p buffers. The q buffers include the first buffer. The second camera request is a camera request obtained by the electronic device before obtaining the first camera request. If the camera parameters in the first camera request are different from those in the second camera request, the first camera request is allocated to the first buffer. The camera parameters include one or more of the following: zoom ratio, focus parameters, exposure parameters, and filter parameters. The first image frame is obtained based on the camera parameters in the first camera request; Display the first image frame.

2. The method according to claim 1, characterized in that, Determining the values ​​of q and p based on the camera mode applied in the electronic device includes: When the camera mode applied by the camera is the photo mode, the value of q is determined to be the first value, and the value of p is determined to be the second value; When the camera mode applied by the camera is recording mode, the value of q is determined to be the third value, the value of p is determined to be the fourth value, the sum of the third value and the fourth value is the fifth value, the sum of the first value and the second value is the sixth value, and the fifth value is equal to the sixth value.

3. The method according to claim 2, characterized in that, Before detecting a user's change in camera parameters and issuing the first camera request, the method further includes: The electronic device detects the operation to launch the camera application and launches the camera application; The camera application in the electronic device sends out multiple camera requests, including the second camera request. If there is an empty buffer in the second buffer area, the second camera request will be allocated to the second buffer, which is an empty buffer in the second buffer area. If there is no empty buffer in the second buffer, the electronic device waits for the second buffer in the second buffer to be cleared before allocating the second camera request to the second buffer.

4. The method according to claim 1, characterized in that, After determining that the camera parameters in the first camera request are different from the camera parameters in the second camera request, the method further includes: If there is an empty buffer in the second buffer, the first camera request is allocated to the second buffer in the second buffer, which is an empty buffer.

5. The method according to claim 4, characterized in that, The user's operation of changing the first camera parameters is the user's operation of changing the zoom ratio, and the method further includes: The camera application displays a first preview interface, which shows a second image frame. The zoom ratio corresponding to the second image frame is the first zoom ratio. The camera application detects the user's operation of changing the zoom ratio and displays a second preview interface. The second preview interface displays the first image frame, and the zoom ratio corresponding to the first image frame is the second zoom ratio.

6. The method according to any one of claims 1-5, characterized in that, The electronic device includes a request module and a cache module. The step of determining that the camera parameters in the first camera request are different from the camera parameters in the second camera request, and allocating the first camera request to the first buffer in the first buffer area, includes: The request module determines that the camera parameters in the first camera request are different from the camera parameters in the second camera request; The request module sends the first camera request to the cache module; The caching module allocates the first camera request to the first buffer in the first buffer area.

7. The method according to claim 6, characterized in that, The electronic device further includes a camera processing module. After the caching module allocates the first camera request to a first buffer in a first buffer area, the method further includes: The caching module sends the first camera request to the camera processing module; Obtaining the first image frame based on the camera parameters in the first camera request includes: The camera processing module sends the camera parameters from the first camera request to the camera module. The camera module acquires the first image frame according to the camera parameters in the first camera request.

8. The method according to claim 3, characterized in that, The electronic device further includes a caching module and a camera processing module. After allocating the second camera request to the second buffer, the method further includes: The caching module sends the second camera request to the camera processing module.

9. The method according to claim 8, characterized in that, The electronic device further includes a camera application and a display module. After the caching module sends the second camera request to the camera processing module, the method further includes: The camera processing module sends the camera parameters in the second camera request to the camera module, and obtains the second image frame obtained by the camera module according to the camera parameters in the second camera request; The camera processing module sends the second image frame to the display module; The display module displays the second image frame; The camera application displays the second image frame.

10. An electronic device, characterized in that, The device includes a camera, one or more processors, and one or more memories; wherein the camera, the one or more memories, and the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method as described in any one of claims 1-9 to be performed.

11. A chip system applied to an electronic device, the chip system comprising one or more processors, characterized in that, The processor is used to invoke computer instructions to cause the execution of the method as described in any one of claims 1-9.

12. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, they cause the method as described in any one of claims 1-9 to be performed.

Images