Interface display method and related device

Abstract

The embodiment of the invention provides an interface display method and a related device, and relates to the technical field of terminals. The method comprises the steps that when the electronic equipment obtains a preview image, the preview image does not need to be displayed by using a SurfaceView, and the preview image with relatively high pixels is displayed based on a layer corresponding to a hardware display module. In this way, the interface of the electronic equipment can display the preview image which is high in pixel and clear, and therefore the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an interface display method and related devices. Background Art

[0002] The electronic device may include a camera application, in which the user may use functions such as preview, photo taking, and video recording. However, in some electronic devices, the image displayed on the preview interface of the camera application has low pixels and is not clear enough, which reduces the user experience. Summary of the invention

[0003] The interface display method and related device provided by the embodiment of the present application can display the preview image with a higher pixel based on the layer corresponding to the hardware display module instead of using the surface view SurfaceView when the electronic device obtains the preview image. In this way, the interface of the electronic device can display a preview image with a higher pixel and a clearer resolution, thereby improving the user experience.

[0004] In a first aspect, an interface display method provided by an embodiment of the present application includes:

[0005] The electronic device receives an operation for opening a first application, and the first application includes a camera application; in response to the operation, the electronic device allocates a cache for the first application, and the resolution corresponding to the cache is a first resolution; the electronic device captures a preview image; the electronic device calls a first preset interface to display the preview image, and the resolution of the preview image is a second resolution, and the second resolution is greater than the first resolution. In this way, the electronic device calls the first preset interface to send the preview image to the layer corresponding to the hardware display module for display, and can display a preview image with higher pixels and clearer resolution, thereby improving the user experience.

[0006] In a possible implementation, the electronic device calls the first preset interface to display a preview image, including: the electronic device calls the first preset interface to store the preview image in a first memory; the electronic device displays the preview image on a first layer based on a mapping relationship between the first memory and a target hardware module in the electronic device, the first layer being a layer corresponding to the target hardware module. In this way, the electronic device can display a preview image with higher pixels on the first layer.

[0007] In a possible implementation, the electronic device starts a second application, which includes a video application; the electronic device calls the first preset interface to play the video of the second application in the first layer, and the resolution of the video of the second application is the second resolution. In this way, the electronic device can display videos with higher pixels.

[0008] In a possible implementation, the first application is provided with a target identifier, and the electronic device calls the first preset interface to display the preview image, including: when the target identifier is a preset value, the electronic device calls the first preset interface to display the preview image on the first layer. In this way, the electronic device can determine whether to display the preview image through the layer corresponding to the hardware display module or to display the image on the OSD layer through the surface view SurfaceView of the Android system according to the target identifier.

[0009] In a possible implementation, when the target identifier is a preset value, the electronic device calls the first preset interface and displays the preview image on the first layer, including: when the electronic device determines that the target identifier is a preset value at the hardware abstraction layer, the electronic device calls the first preset interface at the hardware abstraction layer, and displays the preview image on the first layer based on the hardware driver layer. In this way, the electronic device does not need to return the preview image to the application layer through the application framework layer, and the preview image can display a high-definition image based on the first layer.

[0010] In a possible implementation, the electronic device includes: an image acquisition channel interface module, a decoder component, and the electronic device calls a first preset interface to display a preview image on a first layer, including: when the electronic device determines that the target identifier is a preset value in the image acquisition channel interface module of the hardware abstraction layer, the electronic device calls the first preset interface in the decoder component of the hardware abstraction layer, and displays the preview image on the first layer based on the hardware driver layer. In this way, when the target identifier is the preset value, the image acquisition channel interface module does not need to return the preview image to the first application, and the image acquisition channel interface module can display a higher pixel image on the first layer based on the hardware driver layer.

[0011] In a possible implementation, the first application includes a surface view SurfaceView, and before the electronic device allocates a cache for the first application, the method further includes: the electronic device sets a preset size for the surface view SurfaceView; and the electronic device allocates a cache for the first application, including: the electronic device allocates a cache for the surface view SurfaceView based on the preset size. In this way, fewer resources of the electronic device can be occupied, power consumption can be reduced, and the startup speed of the camera application can also be improved.

[0012] In a possible implementation, the electronic device sets a preset size for the surface view SurfaceView at the application layer; the electronic device allocates a cache for the first application, including: the application framework layer of the electronic device allocates a cache for the surface view SurfaceView based on the preset size of the application layer. In this way, the resources of the electronic device can be better managed, thereby reducing power consumption.

[0013] In a possible implementation, the electronic device includes: a camera preview stream module, the electronic device sets a preset size for the surface view SurfaceView at the application layer, including: a first application at the application layer of the electronic device sets a preset size for the surface view SurfaceView; the electronic device allocates a cache for the first application, including: a camera preview stream module at the application framework layer of the electronic device allocates a cache for the surface view SurfaceView based on the preset size. In this way, less resources of the electronic device can be occupied, power consumption can be reduced, and the startup speed of the first application can be improved.

[0014] In one possible implementation, the preset size includes the minimum value of the size values ​​supported by the surface view SurfaceView. In this way, the electronic device sets the width and height of the preview image displayed by the surface view SurfaceView to smaller values, which can occupy less resources of the electronic device, reduce power consumption, and increase the startup speed of the camera application.

[0015] In a possible implementation, the first application includes an activity component, and before the electronic device allocates a cache to the first application, the electronic device further includes: setting the background of the surface view SurfaceView to be transparent; and setting the theme of the activity component of the first application to be transparent. In this way, the display of the layer corresponding to the hardware display module can be avoided.

[0016] In a possible implementation, the electronic device collects the preview image, including: the electronic device calls the second preset interface to collect the preview image from the camera device. In this way, the preview image can be displayed to the user, providing a corresponding preview function.

[0017] In a possible implementation, before the electronic device calls the second preset interface to collect preview images from the camera device, it also includes: the first application calls the third preset interface to pass the information used to indicate the collection of preview images to the application layer; the application layer passes the information used to indicate the collection of preview images to the application framework layer; the application framework layer passes the information used to indicate the collection of preview images to the hardware abstraction layer; the electronic device calls the second preset interface to collect preview images from the camera device, including: the hardware abstraction layer calls the second preset interface based on the information used to indicate the collection of preview images to collect preview images from the camera device of the hardware driver layer. In this way, the hardware abstraction layer can call the second preset interface to collect preview images after obtaining the information used to indicate the collection of preview images.

[0018] In a possible implementation, the first application calls the third preset interface to pass information for indicating the acquisition of preview images to the application layer, including: the first application calls the third preset interface to pass the information for indicating the acquisition of preview images to the acquisition image data module of the application layer; the acquisition image data module passes the information for indicating the acquisition of preview images to the camera device interface module of the application layer; the application layer passes the information for indicating the acquisition of preview images to the application framework layer, including: the camera device interface module passes the information for indicating the acquisition of preview images to the camera device client module of the application framework layer; the application framework layer passes the information for indicating the acquisition of preview images to the hardware abstraction layer, including: the camera device client module passes the information for indicating the acquisition of preview images to the image acquisition path interface module of the hardware abstraction layer; the electronic device calls the second preset interface to acquire preview images from the camera device, including: the image acquisition path interface module calls the second preset interface based on the information for indicating the acquisition of preview images to acquire preview images from the camera device of the hardware driver layer. In this way, the electronic device passes the information for indicating the acquisition of preview images to the hardware abstraction layer through the application layer and the application framework layer, thereby realizing information transmission in the software framework layer from top to bottom.

[0019] In a possible implementation, the preview image captured by the electronic device is stored in the second memory, and the first preset interface is used to store the preview image in the second memory into the first memory. In this way, data copying from the second memory to the first memory can be achieved, thereby achieving high-definition display of the preview image.

[0020] In a possible implementation, the first preset interface includes a flipToDisp interface, and the second resolution is a 4K resolution. In this way, the electronic device can call the first preset interface to achieve high-definition display of the preview image, thereby improving the user experience.

[0021] In a second aspect, an embodiment of the present application provides an interface display device, which may be an electronic device, or a chip or chip system in an electronic device. The device may include a processing unit and a display unit. The processing unit is used to implement any method related to processing performed by the electronic device in the first aspect or any possible implementation of the first aspect. The display unit is used to implement any method related to display performed by the electronic device in the first aspect or any possible implementation of the first aspect. When the device is an electronic device, the processing unit may be a processor. The device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect. When the device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements the method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).

[0022] Exemplarily, the processing unit is used to receive an operation for starting the first application, allocate a cache for the first application, collect a preview image, and specifically call the first preset interface. The display unit is used to display the preview image.

[0023] In a possible implementation, the processing unit is used to call a first preset interface to store the preview image in a first memory. The display unit is used to display the preview image in a first layer.

[0024] In a possible implementation, the processing unit is used to start the second application and to call the first preset interface to play the video of the second application in the first layer.

[0025] In a possible implementation, the processing unit is configured to cause the electronic device to call a first preset interface and display a preview image on a first layer when the target identifier is a preset value.

[0026] In a possible implementation, the processing unit is used to call the first preset interface in the hardware abstraction layer. The display unit is used to display the preview image in the first layer based on the hardware driver layer.

[0027] In a possible implementation, the processing unit is used to determine the target identifier as a preset value in the image acquisition channel interface module of the hardware abstraction layer, and is also used to call the first preset interface in the decoder component of the hardware abstraction layer. The display unit is used to display the preview image in the first layer based on the hardware driver layer.

[0028] In a possible implementation, the processing unit is used to set a preset size for the surface view SurfaceView, and is also used to allocate a cache for the surface view SurfaceView based on the preset size.

[0029] In a possible implementation, the processing unit is used to set a preset size for the surface view SurfaceView at the application layer, and is also used to allocate a cache for the surface view SurfaceView based on the preset size of the application layer.

[0030] In a possible implementation, the processing unit is used to set a preset size for the surface view SurfaceView at the application layer, and is also used to allocate a cache for the surface view SurfaceView based on the preset size in a camera preview stream module at the application framework layer.

[0031] In a possible implementation, the preset size includes a minimum value of the size values ​​supported by the surface view SurfaceView.

[0032] In a possible implementation, the processing unit is used to set the background of the surface view SurfaceView to be transparent, and is also used to set the theme of the activity component of the first application to be transparent.

[0033] In a possible implementation, the processing unit is used to call the second preset interface to acquire a preview image from the camera device.

[0034] In one possible implementation, the processing unit is used to call the third preset interface to pass information indicating the capture of the preview image to the application layer, and is also used to pass information indicating the capture of the preview image to the application framework layer, and is also used to pass information indicating the capture of the preview image to the hardware abstraction layer, and is specifically used to call the second preset interface to capture the preview image from the camera device of the hardware driver layer.

[0035] In one possible implementation, the processing unit is used to call the third preset interface to pass the information used to indicate the capture of the preview image to the image data capture module of the application layer, and is also used to pass the information used to indicate the capture of the preview image to the camera device interface module of the application layer, and is also used to pass the information used to indicate the capture of the preview image to the camera device client module of the application framework layer, and is also used to pass the information used to indicate the capture of the preview image to the image capture path interface module of the hardware abstraction layer, and is specifically used to call the second preset interface to capture the preview image from the camera device of the hardware driver layer.

[0036] In a possible implementation manner, the processing unit is configured to store the preview image in the second memory into the first memory.

[0037] In a possible implementation, the first preset interface includes a flipToDisp interface, and the second resolution is 4K resolution.

[0038] In a third aspect, an embodiment of the present application provides a terminal device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0041] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0042] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0043] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0046] Figure 3A schematic diagram of an interface display of a large-screen device provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a layer framework when a large-screen device displays an image provided by an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a layer framework corresponding to an interface display method provided in an embodiment of the present application;

[0049] Figure 6 An interactive diagram of a software framework for displaying a preview image in a camera application provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of a process for displaying a preview image provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of another process of displaying a preview image provided by an embodiment of the present application;

[0052] Fig. 9 A schematic diagram of an interface display method provided in an embodiment of the present application;

[0053] Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0055] 1. OSD: On-screen display (OSD) technology can be used to display characters, graphics and images on electronic devices.

[0056] 2. Terminology

[0057] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0058] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0059] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0060] 3. Electronic equipment

[0061] The electronic device of the embodiment of the present application may also be a terminal device in any form. For example, the electronic device may include a handheld device with a display function, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, electronic devices in 5G networks or future evolved public land mobile communication networks (PLANs). The electronic devices in the network (PLMN) and the like are not limited to this in the embodiments of the present application.

[0062] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0063] In addition, in the embodiments of the present application, the electronic device may also be an electronic device in an Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0064] The electronic devices in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0065] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0066] For example, Figure 1 A schematic structural diagram of an electronic device is shown.

[0067] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0068] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0069] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0070] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0071] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0072] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc.

[0073] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0074] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor. For example, the method of an embodiment of the present application may be executed.

[0075] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0076] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1. The electronic device may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0077] Figure 2 It is a software structure block diagram of the electronic device of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the hardware abstraction layer, and the kernel layer.

[0078] The application layer can include a series of application packages. Figure 2 As shown, the application package can include applications such as camera, video, music, calendar, phone, game, memo, etc. Applications can include system applications and third-party applications. The application layer can also be called the application layer.

[0079] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0080] like Figure 2 As shown, the application framework layer may include a notification manager, a resource manager, a notification manager, a content provider, a view system, etc. The application framework layer may also include a camera service manager, a camera device client, a camera stream, etc.

[0081] The camera service manager can be used to provide camera management related functions to the application layer.

[0082] The camera device client can be used to implement the interaction between the camera application and the camera service.

[0083] The camera stream can be used to transmit image data. The camera stream can include camera preview stream, camera photo stream, camera video stream, etc.

[0084] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. The notification manager can also be a notification that appears in the system top status bar in the form of an icon or scroll bar text, or a notification that appears on the screen in the form of a dialog window.

[0085] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, touch the screen, drag the screen, capture the screen, etc.

[0086] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0087] Content providers are used to implement data sharing between different applications, allowing one program to access data in another program while ensuring the security of the accessed data.

[0088] The view system can be responsible for the application interface drawing and event processing.

[0089] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0090] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0091] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. For example, in the embodiment of the present application, the virtual machine can be used to perform functions such as the creation of camera streams, the acquisition of image data of camera devices, and the transmission of image data.

[0092] The system library can also be called the Native layer, which can include multiple functional modules, such as media libraries, function libraries, and graphics processing libraries (such as OpenGL ES).

[0093] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0094] The function library provides developers with API interfaces for multiple services, making it easy for developers to quickly integrate and implement various functions.

[0095] The graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing.

[0096] The hardware abstraction layer is an abstract structure between the kernel layer and the Android runtime. The hardware abstraction layer can be a package of the hardware driver, providing a unified interface for the upper-layer application to call. The hardware abstraction layer can include the camera service manager interface, image acquisition channel interface, decoder component, hardware configuration module, etc.

[0097] The camera service manager interface can be used to manage camera devices, such as plugging and unplugging camera devices, turning on or off camera devices, etc.

[0098] The image acquisition path interface can be used to manage the use of the camera device by a single camera application.

[0099] The decoder component can be used to decode the image data or video data collected by the camera device and send it for display.

[0100] The kernel layer is the layer between hardware and software. The kernel layer may include camera driver, hardware display driver, audio driver, battery driver, Bluetooth driver, CPU driver, USB driver, etc.

[0101] It should be noted that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0102] Some electronic devices may include a camera application, in which users can use functions such as preview, take photos, and record videos. Figure 3 As shown, taking the electronic device as a large-screen device 30 as an example, the large-screen device 30 may include a camera 301 on the top, and the camera 301 may be retractable or fixed. The camera 301 may be located at any position in the large-screen device 30, and the embodiment of the present application is not limited thereto.

[0103] The large-screen device 30 can display a preview interface 302 of the camera application, which can include a picture icon 303 , a photo taking icon 304 , a video recording icon 305 , and a setting icon 306 .

[0104] In some implementations, Figure 4The layer framework corresponding to the interface display image of the large-screen device 30 is shown. Among them, the OSD layer 401 can be understood as the layer corresponding to the electronic device system, and the OSD layer 401 can be a layer that fills the screen of the electronic device. The Activity component 402 of the camera application can be a part of the OSD layer 401, that is, the image data in the Activity component 402 can be displayed in the OSD layer 401. The Activity component 402 can fill the OSD layer 401 or not, and can be customized by the camera application. The surface view SurfaceView403 can be a control for displaying an image in the Activity component 402, and the surface view SurfaceView403 can fill the Activity component 402 or not.

[0105] For example, in the above Figure 3 In the large-screen device 30, when the preview interface 302 displays the preview image, the camera application can call the surface view SurfaceView of the Android system to display the preview image. Figure 3 The picture icon 303, the photo icon 304, the video icon 305 and the setting icon 306 can be displayed by some controls in the Activity component, such as the image control, etc. The embodiment of the present application does not limit the specific controls used.

[0106] It can be understood that, since the image data in the Activity component 402 is displayed in the OSD layer 401 , that is, the camera application uses the surface view SurfaceView403 to display the image based on the OSD layer 401 .

[0107] However, due to the poor graphics processing capabilities of the OSD layer of some electronic devices, the pixels of the image displayed by the surface view SurfaceView are low. For example, the surface view SurfaceView can display 1080P pixel images. However, for images with higher pixels, that is, higher-definition images, such as 4K pixel images, the surface view SurfaceView cannot display them. In this way, the pixels of the images displayed by some electronic devices based on the surface view SurfaceView are low and not clear enough, which reduces the user experience.

[0108] In other implementations, electronic devices can display higher-definition film and television images, such as 4K high-definition videos or film and television dramas, based on the layer corresponding to the hardware display module. It is understandable that when displaying images based on the layer corresponding to the hardware display module, the electronic device can call the interface provided by the hardware display module to display the image on the layer corresponding to the hardware display module, without calling the Android system's surface view SurfaceView to display the image on the OSD layer. In this way, the electronic device can display images with higher pixels. Among them, in some implementations, the layer corresponding to the hardware display module can also be called an XC layer.

[0109] In view of this, the interface display method provided in the embodiment of the present application can display a preview image with a higher pixel based on the layer corresponding to the hardware display module instead of using the surface view SurfaceView when the electronic device obtains the preview image. In this way, the interface of the electronic device can display a preview image with higher pixel and clearer resolution, thereby improving the user experience.

[0110] Figure 5 The layer framework corresponding to the image displayed by the interface display method according to the embodiment of the present application is shown. The layer 501 corresponding to the hardware display module can be understood as the layer provided by the electronic device chip, the layer 501 corresponding to the hardware display module can be the layer that fills the screen of the electronic device, and the OSD layer 502 is above the layer 501 corresponding to the hardware display module.

[0111] When the electronic device displays the preview image, the electronic device can call the interface provided by the hardware display module to copy the memory data storing the preview image to the memory corresponding to the hardware display module. Based on the mapping relationship between the hardware display module and the memory, the hardware display module can display the preview image on the layer 501 corresponding to the hardware display module. In this way, since there is no need to display the image on the OSD layer through the surface view SurfaceView of the Android system, the electronic device can display images with higher pixels, thereby improving the user experience.

[0112] Figure 6 A software framework interaction diagram for displaying a preview image in a camera application of an electronic device is shown.

[0113] (1) Application layer

[0114] The camera application in the application layer may include an image preview (PreviewView) module, a configuration (Configurations) module, an interface callback module, etc. Among them, the interface callback module can obtain the interface callback of the application framework layer and return the information in the callback function to the application. The interface callback module may include a camera manager (CameraManager), a camera device interface (CameraDeviceImpl), and an image data acquisition (CameraCaptureSession) module.

[0115] Among them, the image preview (PreviewView) module can be used to display image data in the preview interface. The module can include a screen buffer (Surface), which can be used to save some parameters of the camera, such as Surface width, Surface height, Surface resolution and other parameters.

[0116] The Configurations module can be used to configure related parameters for the screen buffer (Surface).

[0117] The camera manager (CameraManager) can be used to detect, discover and manage camera devices. Among them, camera devices can also be called camera hardware devices. The camera application can create an object mCameraManager of the camera manager (CameraManager), and the camera manager object mCameraManager can call the functions provided in the camera manager (CameraManager). For example, the camera manager object mCameraManager can call the openCamera() function in the camera manager (CameraManager).

[0118] The camera device interface (CameraDeviceImpl) can be used to obtain the object of the camera hardware device, so as to control the camera hardware device. The camera application can create an object mCameraDevice of the camera device interface (CameraDeviceImpl), and the camera device object mCameraDevice can call the functions provided in the camera device interface (CameraDeviceImpl). For example, the camera device object mCameraDevice can call the function of initializing the acquisition of image data (createCaptureSession(P)) in the camera device interface (CameraDeviceImpl).

[0119] The CameraCaptureSession module can be used to send a request to the camera hardware device to obtain an image. The camera application can create an object mCaptureSession of the CameraCaptureSession module, and the mCaptureSession object can call the functions provided in the CameraCaptureSession module. For example, the mCaptureSession object can call the setRepeatingRequest() function or the capture() function in the CameraCaptureSession module. The setRepeatingRequest() function can repeatedly request to obtain image data, and the capture() function can request to obtain image data once.

[0120] (2) Application framework layer.

[0121] The application framework layer may include a camera service manager (CameraProviderManager), a camera device client (CameraDeviceClient), a camera stream (CameraOutputStream), etc. It is understandable that the camera service (cameraService) of the application framework layer may implement the related functions of the camera service manager (CameraProviderManager), the camera device client (CameraDeviceClient), the camera stream (CameraOutputStream), etc.

[0122] Among them, the camera service manager (CameraProviderManager) can provide the application layer with functions related to camera management. For example, the camera manager (CameraManager) can call the connectDevice() function in the camera service manager (CameraProviderManager) to establish a connection with the camera device.

[0123] The camera device client (CameraDeviceClient) can be used for calls between the camera application and the camera service (cameraService). The camera application can create a camera device client (CameraDeviceClient) object mRemoteDevice, and the camera device client object mRemoteDevice can call the functions provided in the camera device client (CameraDeviceClient), for example, the camera device client object mRemoteDevice can call the configure stream (configureStream(P)) function and the set stream request list (setStreamRequestList()) function in the camera device client (CameraDeviceClient).

[0124] The camera stream (CameraOutputStream) can include camera preview stream, camera photo stream, camera video stream, etc.

[0125] (3) Hardware abstraction layer.

[0126] The hardware abstraction layer may include a camera service manager interface (ICameraProvider), an image acquisition channel interface (ICameraDeviceSession), a decoder component (OMX), and the like.

[0127] Among them, the camera service manager interface (ICameraProvider) can be used to manage camera devices, such as plugging and unplugging camera devices, turning on or off camera devices, etc.

[0128] The image acquisition channel interface (ICameraDeviceSession) can be used to manage the use of the camera device by a single camera application. Each image acquisition channel can correspond to the connection creation of a camera device. For example, the image preview of the camera application can correspond to an image acquisition channel. The camera service (cameraService) can create an object mHidlSession of the image acquisition channel interface (ICameraDeviceSession). The camera service (cameraService) can interact with the hardware abstraction layer through the image acquisition channel object mHidlSession.

[0129] The image acquisition channel interface (ICameraDeviceSession) can also provide a camera service callback function (processCaptureResult()) to return the preview image of the camera device acquired by the image acquisition channel interface (ICameraDeviceSession) to the image preview (PreviewView) module of the application layer through the camera preview stream (CameraOutputStream). The preview image can also be called camera data.

[0130] The decoder component (OMX) can be used to decode and display the image data or video data of the camera device collected by the image acquisition channel interface (ICameraDeviceSession).

[0131] (4) Hardware driver layer.

[0132] The hardware driver layer may include camera devices and hardware display modules, etc.

[0133] For example, Figure 7 As shown, when the interface of the camera application needs to display a preview image, the camera application can call the openCamera() function provided by the camera manager (CameraManager) through the camera manager object mCameraManager to obtain relevant parameters of the available camera devices. For example, the abstract object of the camera device CameraDevice can be obtained, through which the camera application can establish a connection with the camera device.

[0134] The camera application can also configure relevant parameters for the screen buffer (Surface) in the image preview (PreviewView) module through the configuration (Configurations) module, where the relevant parameters can include parameters such as the width of the Surface, the height of the Surface, and the resolution of the Surface. After configuring the relevant parameters, the camera application can call the initialize capture image data (createCaptureSession(P)) function through the camera device object mCameraDevice, configure the parameters to the camera device interface (CameraDeviceImpl), and the camera preview stream (CameraOutputStream) of the application framework layer can initialize the cache Buffer for the screen buffer (Surface).

[0135] When the camera application wants to perform image acquisition, it can call the setRepeatingRequest() function or the capture() function through the image data acquisition object mCaptureSession to send the image acquisition command to the image data acquisition (CameraCaptureSession) module. The image data acquisition (CameraCaptureSession) module can pass the image acquisition command to the image acquisition channel interface (ICameraDeviceSession) of the hardware abstraction layer through the camera device interface (CameraDeviceImpl) of the application layer and the camera device client (CameraDeviceClient) of the application framework layer.

[0136] On the one hand, after initializing the cache according to parameters such as resolution, the image acquisition channel interface (ICameraDeviceSession) can return the preview image of the captured camera device to the image preview (PreviewView) module of the application layer through the camera preview stream (CameraOutputStream) of the application framework layer through the camera service callback function (processCaptureResult()). The image preview (PreviewView) module can write the preview image to the screen buffer (Surface) and display it on the interface of the camera application.

[0137] On the other hand, the image acquisition channel interface (ICameraDeviceSession) can return the image acquisition result to the camera device client (CameraDeviceClient) of the application framework layer through the image acquisition channel object mHidlSession, and the camera device client (CameraDeviceClient) can finally return the image acquisition result to the image data acquisition (CameraCaptureSession) module of the application layer through the camera device client object mRemoteDevice. In this way, the camera application can obtain the image acquisition result through the image data acquisition object mCaptureSession, so that the corresponding process can be executed according to the result, for example, the camera application can prompt the user with a failure message, etc. It can be understood that the image acquisition result may include some status data such as the success or failure of the image data return, but may not include a specific preview image.

[0138] However, the surface view SurfaceView used by the camera application cannot display images with higher pixels, such as 4K pixels. The layer corresponding to the hardware display module of the electronic device can support the display of images with higher pixels. Therefore, when the embodiment of the present application obtains the preview image, the surface view SurfaceView may not be used to display the preview image, but the layer corresponding to the hardware display module may be used to display the preview image with higher pixels.

[0139] Possible implementations, such as Figure 8 As shown, when the interface of the camera application needs to display a preview image, the camera application can call the openCamera() function provided by the camera manager (CameraManager) through the camera manager object mCameraManager to obtain relevant parameters of the available camera devices.

[0140] The camera application can also configure relevant parameters for the screen buffer (Surface) in the image preview (PreviewView) module through the configuration (Configurations) module, where the relevant parameters may include the width of the Surface, the height of the Surface, a field identifier for indicating whether to display the preview image through the layer corresponding to the hardware display module, and parameters such as resolution.

[0141] Among them, the field identifier can be customized by the camera application, for example, the field identifier can be named PQ_MODE, which is not limited in the embodiment of the present application.

[0142] The data type of the field identifier can be an integer, a string, a Boolean, etc. For example, when the field identifier is 1, it can indicate that the preview image is displayed through the layer corresponding to the hardware display module, and when the field identifier is 0, it can indicate that the preview image is displayed through the image preview (PreviewView) module. Alternatively, when the field identifier is 0, it can indicate that the preview image is displayed through the layer corresponding to the hardware display module, and when the field identifier is 1, it can indicate that the preview image is displayed through the image preview (PreviewView) module. The specific value of the field identifier is not limited in the embodiment of the present application.

[0143] The resolution can be customized by the camera application. For example, the resolution can be named SCALER_CROP_REGION. The data type of the resolution can be an integer array type or other types. The embodiment of the present application does not limit the name and data type of the resolution.

[0144] After configuring the relevant parameters, the camera application can call the createCaptureSession(P) function to initialize the image data acquisition through the camera device object mCameraDevice, configure the parameters to the camera device interface (CameraDeviceImpl), and the camera preview stream (CameraOutputStream) can initialize the cache Buffer for the screen buffer (Surface).

[0145] Since the embodiment of the present application does not use the surface view SurfaceView to display the preview image, but under the Android mechanism, the preview function of the camera application needs to call the surface view SurfaceView. Therefore, the embodiment of the present application needs to set relevant parameters such as size and transparency for the surface view SurfaceView, so as to achieve adaptation to the surface view SurfaceView under the Android mechanism. In this way, the clear display of the preview image can be achieved while improving the compatibility of the camera application.

[0146] In a possible implementation, when the camera device interface (CameraDeviceImpl) initializes the cache, the width layout_width and height layout_height of the surface view SurfaceView displaying the preview image can be set to smaller values. In a possible implementation, the width layout_width and the height layout_height can both be set to 1 pixel (px). The width layout_width and the height layout_height of the preview image displayed by the surface view SurfaceView can be customized by the camera application, which is not limited in the embodiments of the present application.

[0147] It can be understood that setting the width and height of the preview image displayed by the surface view SurfaceView to smaller values ​​can, on the one hand, occupy fewer resources of the electronic device and reduce power consumption; on the other hand, since the cache Buffer corresponding to the surface view SurfaceView becomes smaller, the startup speed of the camera application can be improved.

[0148] In addition, the camera device interface (CameraDeviceImpl) can also set the transparency alpha of the surface view SurfaceView to transparent, so as not to affect the display of the preview image of the layer corresponding to the hardware display module. In a possible implementation, the transparency alpha of the surface view SurfaceView can be set to 0, so that the surface view SurfaceView can not display specific data.

[0149] In a possible implementation, the embodiment of the present application may also set the theme of the Activity component to transparent, so as not to block the display of the layer corresponding to the hardware display module.

[0150] When the camera application wants to perform image acquisition, it can call the setRepeatingRequest() function or the capture() function through the image data acquisition object mCaptureSession to send the image acquisition command to the image data acquisition (CameraCaptureSession) module. The image data acquisition (CameraCaptureSession) module can pass the image acquisition command to the image acquisition channel interface (ICameraDeviceSession) of the hardware abstraction layer through the camera device interface (CameraDeviceImpl) of the application layer and the camera device client (CameraDeviceClient) of the application framework layer.

[0151] The image acquisition channel interface (ICameraDeviceSession) can obtain the preview image from the camera device of the hardware driver layer through the dequeueV4l2FrameLocked function. After obtaining the preview image, the image acquisition channel interface (ICameraDeviceSession) can call the pushFrameToDecode function and / or the Wrapper_MsVdec_GetNextDispFrame function to pass the preview image to the decoder component (OMX) for decoding.

[0152] After the decoder component (OMX) performs decoding, the image acquisition channel interface (ICameraDeviceSession) can determine whether to display the preview image through the layer corresponding to the hardware display module according to the field identifier used to indicate whether to display the preview image through the layer corresponding to the hardware display module. If it is determined that the preview image is displayed through the layer corresponding to the hardware display module, the decoder component (OMX) can call the flipToDisp function to send the preview image to the layer corresponding to the hardware display module for display. In this way, the electronic device can use the layer corresponding to the hardware display module to display a preview image with higher pixels.

[0153] Since the image acquisition channel interface (ICameraDeviceSession) does not return the preview image to the application framework layer, the application framework layer cannot return the preview image to the camera application. Therefore, the camera application has no data to write to the screen buffer (Surface), that is, the camera application does not display the preview image.

[0154] It is understandable that the method of the embodiment of the present application can be reflected by the keywords in the log. For example, taking the camera service (cameraService) setting the width and height values ​​of the surface view SurfaceView to 1 as an example, the log may include relevant keywords such as width:1, height:1. For example, the log may include: cameraserver:createSurfaceFromGbp:From surface get width:1,height:1,format:1.

[0155] In addition, the method of the embodiment of the present application can also be embodied through keywords in the chip log of the electronic device. For example, when the camera application is previewing an image, the chip log of the electronic device may include a print log of the flipToDisp function call.

[0156] The following specific embodiments are used to describe the method of the present application in detail. The following embodiments may be combined with each other or implemented independently, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0157] Fig. 9 The interface display method of the embodiment of the present application is shown. The method includes:

[0158] S901: An electronic device receives an operation for opening a first application, where the first application includes a camera application.

[0159] In the embodiment of the present application, the first application can be understood as an application with an image preview function, such as a camera application. The first application can be understood as the camera application in the above embodiment.

[0160] The operation of opening the first application may include an operation of a user touching the first application icon on the screen of the electronic device, or an operation of a user inputting voice input to open the first application, or an operation of a user using a remote control to open the first application, which is not limited in the embodiments of the present application.

[0161] S902: In response to the operation, the electronic device allocates a cache for the first application, and the resolution corresponding to the cache is the first resolution.

[0162] In the embodiment of the present application, the reason and process for the electronic device to allocate the cache to the first application can refer to the above Figure 7 The relevant descriptions in the corresponding embodiments are not repeated here.

[0163] The first resolution may be a lower resolution, for example, the first resolution may include a resolution of 1 px. The specific value of the first resolution is not limited in the embodiment of the present application.

[0164] S903: The electronic device collects a preview image.

[0165] In the embodiment of the present application, the process of the electronic device collecting the preview image can refer to the above Figure 7 The relevant descriptions in the corresponding embodiments are not repeated here.

[0166] S904: The electronic device calls the first preset interface to display a preview image, where the resolution of the preview image is a second resolution, which is greater than the first resolution.

[0167] In the embodiment of the present application, the first preset interface can be understood as the above Figure 8 Corresponding to the flipToDisp function in the embodiment, the interface can send the preview image to the layer corresponding to the hardware display module for display. Displaying the preview image can be understood as displaying the preview image on the preview interface of the first application.

[0168] The second resolution may be a higher resolution, for example, the second resolution may include a 4K resolution. The specific value of the second resolution is not limited in the embodiment of the present application.

[0169] The electronic device calls the first preset interface to send the preview image to the layer corresponding to the hardware display module for display, which can display a preview image with higher pixels and clearer resolution, thereby improving the user experience.

[0170] Optional, in Fig. 9 On the basis of the corresponding embodiment, the electronic device calls the first preset interface to display a preview image, which may include: the electronic device calls the first preset interface to store the preview image in a first memory; the electronic device displays the preview image in a first layer based on a mapping relationship between the first memory and the target hardware module in the electronic device, and the first layer is a layer corresponding to the target hardware module.

[0171] In the embodiment of the present application, the target hardware module can be understood as the above Figure 5 The hardware display module in the corresponding embodiment. The first memory can be understood as the memory corresponding to the target hardware module.

[0172] The electronic device displays the preview image on the first layer based on the mapping relationship between the first memory and the target hardware module. In this way, the electronic device can display a preview image with higher pixels and clearer resolution on the first layer, thereby improving the user experience.

[0173] Optional, in Fig. 9 Based on the corresponding embodiment, the method may include: the electronic device starts a second application, the second application includes a video application; the electronic device calls a first preset interface to play a video with a second resolution on a first layer.

[0174] In the embodiment of the present application, the second application can be understood as an application that can play high-definition videos, for example, the second application can play 4K videos.

[0175] Based on the implementation of the electronic device calling the first preset interface to play the video of the second resolution on the first layer, the embodiment of the present application can also display the preview image of the second resolution on the first layer based on the first preset interface, without calling the surface view SurfaceView of the Android system to display the image on the OSD layer. In this way, the electronic device can display images with higher pixels.

[0176] Optional, in Fig. 9 On the basis of the corresponding embodiment, the first application is set with a target identifier, and the electronic device calls the first preset interface to display a preview image, which may include: when the target identifier is a preset value, the electronic device calls the first preset interface to display the preview image on the first layer.

[0177] In the embodiment of the present application, the target identifier can be understood as the above Figure 8 In the corresponding embodiment, the field identifier is used to indicate whether to display the preview image through the layer corresponding to the hardware display module. The specific field identifier and preset value can be referred to Figure 8 The relevant descriptions in the corresponding embodiments are not repeated here.

[0178] It is understandable that the target identifier of different applications can have different values, so that it can be compatible with different interface display situations. The electronic device can determine, based on the target identifier, whether to display the preview image through the layer corresponding to the hardware display module, or to display the image on the OSD layer through the Android system's surface view SurfaceView.

[0179] Optional, in Fig. 9 On the basis of the corresponding embodiment, when the target identifier is a preset value, the electronic device calls the first preset interface and displays the preview image on the first layer, which may include: when the electronic device determines that the target identifier is a preset value at the hardware abstraction layer, the electronic device calls the first preset interface at the hardware abstraction layer, and displays the preview image on the first layer based on the hardware driver layer.

[0180] In the embodiment of the present application, the electronic device calls the first preset interface in the hardware abstraction layer, and displays the preview image in the first layer based on the hardware driver layer. In this way, the electronic device does not need to return the preview image to the application layer through the application framework layer, that is, the application side does not need to display the preview image, so that the preview image can display a high-definition image based on the first layer.

[0181] Optional, in Fig. 9On the basis of the corresponding embodiment, the electronic device includes: an image acquisition path interface module, a decoder component, the electronic device calls a first preset interface, and displays a preview image on a first layer, which may include: when the electronic device determines that the target identifier is a preset value in the image acquisition path interface module of the hardware abstraction layer, the electronic device calls the first preset interface in the decoder component of the hardware abstraction layer, and displays the preview image on the first layer based on the hardware driver layer.

[0182] In the embodiment of the present application, the image acquisition path interface module can be understood as the above Figure 6 The image acquisition channel interface (ICameraDeviceSession) in the corresponding embodiment. The decoder component can be understood as the above Figure 6 The decoder component (OMX) in the corresponding embodiment will not be described in detail.

[0183] The image acquisition channel interface module can determine whether to display the preview image through the first layer based on the target identifier. When the target identifier is a preset value, the image acquisition channel interface module does not need to return the preview image to the first application. In this way, the image acquisition channel interface module can realize the display of a higher pixel image on the first layer based on the hardware driver layer.

[0184] Optional, in Fig. 9 On the basis of the corresponding embodiment, the electronic device includes a surface view SurfaceView. Before the electronic device allocates cache to the first application, it may also include: the electronic device sets a preset size for the surface view SurfaceView; the electronic device allocates cache to the first application, including: the electronic device allocates cache to the surface view SurfaceView based on the preset size.

[0185] In the embodiment of the present application, the preset size can be understood as the width layout_width and height layout_height of the surface view SurfaceView displaying the preview image. For details, please refer to the above Figure 8 The description in the corresponding embodiment will not be repeated here.

[0186] The electronic device can set the width and height of the preview image displayed by the surface view SurfaceView to smaller values. On the one hand, it can occupy fewer resources of the electronic device and reduce power consumption. On the other hand, since the cache Buffer corresponding to the surface view SurfaceView becomes smaller, the startup speed of the camera application can be improved.

[0187] Optional, in Fig. 9On the basis of the corresponding embodiment, the electronic device sets a preset size for the surface view SurfaceView at the application layer; the electronic device allocates cache for the first application, which may include: the application framework layer of the electronic device allocates cache for the surface view SurfaceView based on the preset size of the application layer.

[0188] In an embodiment of the present application, the application layer can set a smaller preset size for the surface view SurfaceView, so that the resources of the electronic device can be better managed, thereby reducing power consumption.

[0189] Optional, in Fig. 9 On the basis of the corresponding embodiment, the electronic device includes: a camera preview stream module, the electronic device sets a preset size for the surface view SurfaceView at the application layer, which may include: a first application at the application layer of the electronic device sets a preset size for the surface view SurfaceView; the electronic device allocates cache for the first application, which may include: a camera preview stream module at the application framework layer of the electronic device allocates cache for the surface view SurfaceView based on the preset size.

[0190] In the embodiment of the present application, the camera preview stream module can be understood as the above Figure 6 The camera preview stream (CameraOutputStream) in the corresponding embodiment will not be described in detail.

[0191] The first application can set a preset size according to the effect presented by the interface preview, and different applications can set different preset sizes for the surface view SurfaceView. In order not to affect the display of the first layer, the first application can set a smaller preset size for the surface view SurfaceView. In this way, less resources of the electronic device can be occupied, power consumption can be reduced, and the startup speed of the first application can be improved.

[0192] Optional, in Fig. 9 Based on the corresponding embodiment, the preset size includes the minimum value of the size values ​​supported by the surface view SurfaceView.

[0193] In the embodiment of the present application, the minimum value of the size values ​​supported by the surface view SurfaceView is not limited in the embodiment of the present application. For example, the minimum value can be set to 1px.

[0194] It can be understood that the preset size can be the second smallest value, the third smallest value, or other smaller values ​​among the size values ​​supported by the surface view SurfaceView, and the embodiment of the present application is not limited thereto.

[0195] The electronic device sets the width and height of the preview image displayed by the surface view SurfaceView to smaller values, which can occupy fewer resources of the electronic device and reduce power consumption; in addition, since the cache Buffer corresponding to the surface view SurfaceView becomes smaller, the startup speed of the camera application can also be improved.

[0196] Optional, in Fig. 9 On the basis of the corresponding embodiment, the first application includes an activity component. Before the electronic device allocates cache for the first application, the method may also include: the electronic device sets the background of the surface view SurfaceView to be transparent; and the electronic device sets the theme of the activity component of the first application to be transparent.

[0197] In the embodiment of the present application, the setting of transparency can refer to the above Figure 8 The description in the corresponding embodiment will not be repeated here. In this way, the display of the layer corresponding to the hardware display module will not be blocked.

[0198] Optional, in Fig. 9 On the basis of the corresponding embodiment, the electronic device collecting the preview image may include: the electronic device calling the second preset interface to collect the preview image from the camera device.

[0199] In the embodiment of the present application, the second preset interface can be understood as the above Figure 8 The dequeueV4l2FrameLocked function in the corresponding embodiment can collect preview images from the camera device of the hardware driver layer.

[0200] After the electronic device calls the second preset interface to capture the preview image, it can display the preview image for the user and provide a corresponding preview function.

[0201] Optional, in Fig. 9 On the basis of the corresponding embodiment, before the electronic device calls the second preset interface to capture a preview image from the camera device, it may also include: the first application calls the third preset interface to pass information used to indicate the capture of the preview image to the application layer; the application layer passes the information used to indicate the capture of the preview image to the application framework layer; the application framework layer passes the information used to indicate the capture of the preview image to the hardware abstraction layer; the electronic device calls the second preset interface to capture the preview image from the camera device, including: the hardware abstraction layer calls the second preset interface based on the information used to indicate the capture of the preview image to capture the preview image from the camera device of the hardware driver layer.

[0202] In the embodiment of the present application, the third preset interface can be understood as the above Figure 7The setRepeatingRequest() function or the capture() function in the corresponding embodiment can be used to instruct the capture of the preview image.

[0203] The electronic device passes the information used to indicate the capture of the preview image to the hardware abstraction layer through the application layer and the application framework layer, thereby realizing information transmission in the software framework layer from top to bottom, so that the hardware abstraction layer can call the second preset interface to capture the preview image after obtaining the information used to indicate the capture of the preview image.

[0204] Optional, in Fig. 9 On the basis of the corresponding embodiments, the first application calls the third preset interface to pass the information used to indicate the capture of the preview image to the application layer, which may include: the first application calls the third preset interface to pass the information used to indicate the capture of the preview image to the image capture data module of the application layer; the image capture data module passes the information used to indicate the capture of the preview image to the camera device interface module of the application layer; the application layer passes the information used to indicate the capture of the preview image to the application framework layer, which may include: the camera device interface module passes the information used to indicate the capture of the preview image to the camera device client module of the application framework layer; the application framework layer passes the information used to indicate the capture of the preview image to the hardware abstraction layer, which may include: the camera device client module passes the information used to indicate the capture of the preview image to the image capture path interface module of the hardware abstraction layer; the electronic device calls the second preset interface to capture the preview image from the camera device of the hardware driver layer based on the information used to indicate the capture of the preview image.

[0205] In the embodiment of the present application, the image data acquisition module can be understood as the above Figure 6 The camera capture session module and the camera device interface module in the corresponding embodiment can be understood as the above Figure 6 The camera device interface (CameraDeviceImpl) in the corresponding embodiment, the camera device client module can be understood as the above Figure 6 The camera device client (CameraDeviceClient) in the corresponding embodiment and the image acquisition path interface module can be understood as the above Figure 6 The image acquisition channel interface (ICameraDeviceSession) in the corresponding embodiment will not be described in detail.

[0206] The electronic device passes the information used to indicate the capture of the preview image to the hardware abstraction layer through the application layer and the application framework layer, thereby realizing the information transmission in the software framework layer from top to bottom, so that the image capture channel interface module can call the second preset interface to capture the preview image after obtaining the information used to indicate the capture of the preview image.

[0207] Optional, in Fig. 9 On the basis of the corresponding embodiment, the preview image collected by the electronic device is stored in the second memory, and the first preset interface is used to store the preview image in the second memory into the first memory.

[0208] In the embodiment of the present application, the second memory can be understood as the memory in which the preview image is stored after being captured by the camera device.

[0209] The first preset interface is used to store the preview image in the second memory in the first memory, so that the electronic device can copy data from the second memory to the first memory by calling the first preset interface in the hardware abstraction layer, thereby realizing high-definition display of the preview image.

[0210] Optional, in Fig. 9 On the basis of the corresponding embodiment, the first preset interface includes a flipToDisp interface, and the second resolution is 4K resolution.

[0211] In the embodiment of the present application, the electronic device calls the first preset interface to achieve high-definition display of the preview image, thereby improving the user experience.

[0212] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0213] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0214] The embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0215] like Fig.10 FIG. 1 is a schematic diagram of a chip structure provided by an embodiment of the present application. The chip 1000 includes one or more (including two) processors 1001 , a communication line 1002 , a communication interface 1003 and a memory 1004 .

[0216] In some implementations, the memory 1004 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0217] The method described in the above embodiment of the present application can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the processor 1001. The above processor 1001 can be a general processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 1001 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0218] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 1004, and the processor 1001 reads the information in the memory 1004 and completes the steps of the above method in combination with its hardware.

[0219] The processor 1001 , the memory 1004 , and the communication interface 1003 may communicate with each other via the communication line 1002 .

[0220] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0221] The embodiment of the present application also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium can include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0222] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0223] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while optical discs reproduce data optically using lasers.

[0224] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. An interface display method, characterized in that: The method comprises: The electronic device receives an operation for opening a first application, where the first application includes a camera application; In response to the operation, the electronic device allocates a cache for the first application, and a resolution corresponding to the cache is a first resolution; The electronic device collects a preview image; The electronic device calls a first preset interface to display the preview image, wherein the resolution of the preview image is a second resolution, and the second resolution is greater than the first resolution.

2. The method according to claim 1, characterized in that: The electronic device calls a first preset interface to display the preview image, including: The electronic device calls the first preset interface to store the preview image in a first memory; The electronic device displays the preview image on a first layer based on a mapping relationship between the first memory and a target hardware module in the electronic device, where the first layer is a layer corresponding to the target hardware module.

3. The method according to claim 2, characterized in that The method comprises: The electronic device starts a second application, where the second application includes a video application; The electronic device calls the first preset interface to play the video of the second application in the first layer, and the resolution of the video of the second application is the second resolution.

4. The method according to any one of claims 1 to 3, characterized in that: The first application is provided with a target identifier, and the electronic device calls a first preset interface to display the preview image, including: When the target identifier is a preset value, the electronic device calls the first preset interface to display the preview image on a first layer.

5. The method according to claim 4, characterized in that When the target identifier is a preset value, the electronic device calls the first preset interface to display the preview image on a first layer, including: When the electronic device determines at the hardware abstraction layer that the target identifier is the preset value, the electronic device calls the first preset interface at the hardware abstraction layer, and displays the preview image at the first layer based on the hardware driver layer.

6. The method according to claim 5, characterized in that The electronic device comprises: an image acquisition channel interface module and a decoder component. The electronic device calls the first preset interface to display the preview image on the first layer, including: When the electronic device determines that the target identifier is the preset value in the image acquisition path interface module of the hardware abstraction layer, the electronic device calls the first preset interface in the decoder component of the hardware abstraction layer, and displays the preview image in the first layer based on the hardware driver layer.

7. The method according to any one of claims 1 to 6, characterized in that: The first application includes a surface view SurfaceView, and before the electronic device allocates a cache for the first application, the electronic device further includes: The electronic device sets a preset size for the surface view SurfaceView; The electronic device allocates a cache for the first application, including: the electronic device allocates a cache for the surface view SurfaceView based on the preset size.

8. The method according to claim 7, characterized in that The electronic device sets the preset size for the surface view SurfaceView at the application layer; The electronic device allocates cache for the first application, including: the application framework layer of the electronic device allocates cache for the surface view SurfaceView based on the preset size of the application layer.

9. The method according to claim 8, characterized in that The electronic device comprises: a camera preview stream module, The electronic device sets the preset size for the surface view SurfaceView at the application layer, including: the first application at the application layer of the electronic device sets the preset size for the surface view SurfaceView; The electronic device allocates a cache for the first application, including: the camera preview stream module of the application framework layer of the electronic device allocates a cache for the surface view SurfaceView based on the preset size.

10. The method according to any one of claims 7 to 9, characterized in that: The preset size includes the minimum value of the size values ​​supported by the surface view SurfaceView.

11. The method according to any one of claims 7 to 10, characterized in that: The first application includes an activity component, and before the electronic device allocates a cache for the first application, the electronic device further includes: The electronic device sets the background of the surface view SurfaceView to be transparent; The electronic device sets a theme of the activity component of the first application to be transparent.

12. The method according to any one of claims 1 to 11, characterized in that: The electronic device collects a preview image, including: The electronic device calls a second preset interface to collect the preview image from a camera device.

13. The method according to any one of claim 12, characterized in that: Before the electronic device calls the second preset interface to collect the preview image from the camera device, the method further includes: The first application calls a third preset interface to pass information for instructing to collect the preview image to the application layer; The application layer transmits the information for instructing to collect the preview image to the application framework layer; The application framework layer transmits the information for instructing to collect the preview image to the hardware abstraction layer; The electronic device calls the second preset interface to collect the preview image from the camera device, including: the hardware abstraction layer calls the second preset interface to collect the preview image from the camera device of the hardware driver layer based on the information for instructing to collect the preview image.

14. The method according to any one of claim 13, characterized in that: The first application calls the third preset interface to pass the information for indicating the acquisition of the preview image to the application layer, including: the first application calls the third preset interface to pass the information for indicating the acquisition of the preview image to the image data acquisition module of the application layer; the image data acquisition module passes the information for indicating the acquisition of the preview image to the camera device interface module of the application layer; The application layer transmits the information for indicating the acquisition of the preview image to the application framework layer, including: the camera device interface module transmits the information for indicating the acquisition of the preview image to the camera device client module of the application framework layer; The application framework layer transmits the information for indicating the acquisition of the preview image to the hardware abstraction layer, including: the camera device client module transmits the information for indicating the acquisition of the preview image to the image acquisition path interface module of the hardware abstraction layer; The electronic device calls the second preset interface to collect the preview image from the camera device, including: the image acquisition channel interface module calls the second preset interface to collect the preview image from the camera device of the hardware driver layer based on the information for instructing to collect the preview image.

15. The method according to any one of claims 2 to 14, characterized in that: The preview image captured by the electronic device is stored in a second memory, and the first preset interface is used to store the preview image in the second memory into the first memory.

16. The method according to any one of claims 1 to 15, characterized in that: The first preset interface includes a flipToDisp interface, and the second resolution is 4K resolution.

17. An electronic device, characterized in that: include: A memory and a processor, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables an electronic device to execute the method according to any one of claims 1 to 16.

Images