A method for displaying a preview image, an electronic device, and a storage medium

Abstract

The present application provides a method for displaying a preview image, an electronic device, and a storage medium, relating to the technical field of image processing. The method is applied to an electronic device, which includes a camera and a first application, and comprises: receiving an operation of the user to call the camera through the first application; in response to the operation of the first application to call the camera, obtaining a square original preview image; processing the original preview image to obtain a first preview image of a target size, and rotating the first preview image to obtain a second preview image; vertically displaying a preview window of the first application, and the preview window displays the second preview image. The obtained original preview image is a square with the width of the target size as the side length. By processing the original preview image, a preview image of the target size can be obtained, thereby realizing the display of the preview image. During the entire display process of the preview image, no blackening or overlapping processing is required, and there is no influence of black blocks on the preview image displayed by the electronic device.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method for displaying a preview image, an electronic device, and a storage medium. Background Art

[0002] With the continuous development of science and technology, more and more electronic devices can meet the multi-window usage needs of users. Electronic devices can provide various multi-window display modes such as split-screen display mode and small-window display mode. The split-screen display mode can apply multiple application pages in multiple display areas on the display screen of the electronic device. For example, the parallel vision mode is a split-screen display mode.

[0003] When the electronic device is running in the split-screen display module or small-window display mode, there is a situation where the electronic device itself is in the landscape screen state, but the displayed application interface is still in the portrait screen display. At this time, when the portrait-screen displayed application program calls the camera, the preview window corresponding to the application program is also in the portrait screen display state. When the preview window displays the preview image obtained by the camera, the effective preview image of the preview image is generally square, and black blocks are introduced at the top and bottom, seriously affecting the display content of the electronic device. Summary of the Invention

[0004] A method for displaying a preview image, an electronic device, and a storage medium provided by this application are used to ensure that there are no black blocks in the preview image displayed by the electronic device.

[0005] To achieve the above object, this application adopts the following technical solutions:

[0006] In a first aspect, this application provides a method for displaying a preview image, which is applied to an electronic device. The electronic device includes a camera and a first application, and the method includes: receiving an operation by the user to call the camera through the first application; in response to the operation of the first application calling the camera, obtaining a square original preview image; wherein, the side length of the square is the width of the target size, and the width of the target size is greater than the height of the target size; processing the original preview image to obtain a first preview image of the target size, and rotating the first preview image to obtain a second preview image; vertically displaying the preview window of the first application, and the preview window displays the second preview image. In this application, the original preview image is a square with the width (the larger side) of the target size as the side length. By processing the original preview size, a preview image that meets the target size can be obtained, thus realizing the normal display of the preview image. During the entire display process of the preview image, no blackening process or overlapping process is required, so there is no black block influence in the preview image displayed by the electronic device.

[0007] In a possible implementation, when the electronic device is in the landscape display state, the width of the original preview image is cropped to the height of the target size to obtain a cropped preview image; the cropped preview image is rotated to obtain a first preview image of the target size. Only by cropping and then rotating the original preview image can a first preview image of the target size be obtained, without the need for blackening and overlay processing. Therefore, there is no influence of black blocks on the preview image finally displayed by the electronic device.

[0008] In a possible implementation, when the electronic device is in the portrait display state, the height of the original preview image is cropped to the height of the target size to obtain a first preview image of the target size. Based on the obtained original preview image, the display of the preview image in the portrait display state of the electronic device is realized, avoiding the short-term black screen phenomenon during the horizontal and vertical screen state switching of the electronic device, and further improving the user experience.

[0009] In a possible implementation, in response to an operation of the first application calling the camera, the layer corresponding to the first application is obtained, and the size of the layer corresponding to the first application is the target size; when it is determined that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions, a square original preview image is obtained. When the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions, the preview image display method provided in this application is executed.

[0010] In a possible implementation, the method for determining that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions includes: determining that the first label of the electronic device is not empty; determining that the first application is in the application whitelist; determining that the stream name of the layer corresponding to the first application is the preview stream or the callback stream, and determining that the width of the target size is less than the maximum height corresponding to the electronic device. Among them, the first label of the electronic device plays a role in product isolation. The first label of the electronic device that supports the preview image display method provided in this application is not empty, and the first label of the electronic device that does not support the preview image display method provided in this application is empty; among them, the application whitelist plays a role in application isolation, and the applications in the application whitelist support the preview image display method provided in this application; among them, the preview image display method provided in this application is applicable to the image display of the preview stream and the callback stream, and moreover, only when the maximum height of the preview image that the electronic device allows to obtain is greater than the width of the target size can a square original preview image with the width of the target size as the side length be obtained.

[0011] In a possible implementation, when it is determined that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions, a first cache queue and a second cache queue are created; the first cache queue includes a first cache, the width of the first cache is the width of the target size, and the height of the first cache is the width of the target size; the second cache queue includes a second cache, the width of the second cache is the width of the target size, and the height of the second cache is the height of the target size; after obtaining the original preview image of the square, the original preview image of the square is stored in the first cache; after processing the original preview image to obtain the first preview image of the target size, the first preview image of the target size is stored in the second cache. The display method of the preview image provided by the present application is implemented through two cache queues. Among them, the first cache in the first cache queue is used to store the original preview image of the square, and the second cache in the second cache queue is used to store the first preview image of the target size.

[0012] In a possible implementation, the first cache storing the original preview image is obtained from the first cache queue, and the original preview image stored in the first cache is copied to a third cache; the original preview image stored in the third cache is processed to obtain the first preview image of the target size. The third cache is the idle cache of the electronic device itself. After copying the original preview image in the first cache to the third cache, the original preview image in the third cache is processed to obtain the first preview image of the target size.

[0013] In a second aspect, the present application provides an electronic device, which includes: a camera, a display screen, a processor, and a memory; wherein, the camera, the display screen, and the memory are coupled to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the method in the first aspect above.

[0014] In a third aspect, the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run, the method in the first aspect above is implemented.

[0015] In a fourth aspect, the present application provides a computer program product, including a computer program or instruction. When the computer program or instruction is executed by a processor, the electronic device is enabled to implement the method in the first aspect above. Description of the Drawings

[0016] Figure 1 It is a schematic diagram of the display direction of the preview window in the parallel vision scene of an electronic device provided by an embodiment of the present application;

[0017] Figure 2Schematic diagram of the display direction of the preview window in the scenario where a non-system application of an electronic device provided in an embodiment of the present application is not compatible with the electronic device;

[0018] Figure 3 Schematic diagram of the display direction of the preview window of a non-system application with floating window display of an electronic device provided in an embodiment of the present application;

[0019] Figure 4 Schematic diagram of the direction of the original preview image output by the hardware framework layer of an electronic device in multiple states provided in an embodiment of the present application;

[0020] Figure 5A Simplified schematic diagram of a method for displaying a preview image in the current technology;

[0021] Figure 5B Schematic diagram of the preview image displayed by the corresponding electronic device in the current technology;

[0022] Figure 6 Schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;

[0023] Figure 7 Schematic diagram of the process flow of a flow distribution process provided in an embodiment of the present application;

[0024] Figure 8 Schematic diagram of the process flow for determining whether a preset condition is met provided in an embodiment of the present application;

[0025] Figure 9 Schematic diagram of the process flow of a method for displaying a preview image provided in an embodiment of the present application;

[0026] Figure 10 Comparison schematic diagram of the display interface of the preview image of the electronic device provided in an embodiment of the present application and the display interface of the preview image in the prior art;

[0027] Figure 11A Schematic diagram of the cache transfer corresponding to a method for displaying a preview image provided in an embodiment of the present application;

[0028] Figure 11B Schematic diagram of the process flow of a method for displaying a preview image based on cache transfer provided in an embodiment of the present application;

[0029] Figure 12 Schematic diagram of the process flow of another method for displaying a preview image provided in an embodiment of the present application;

[0030] Figure 13 Schematic diagram of the process flow of another method for displaying a preview image provided in an embodiment of the present application;

[0031] Figure 14 This is a schematic diagram showing the composition of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0032] In the description of the present application, the terms "first", "second", "third", etc. in the specification, claims and drawings are used to distinguish different objects, rather than to limit a specific order.

[0033] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0034] For the sake of clear and concise description of the following embodiments, a brief introduction to the related art is given first:

[0035] An application, also known as an application program, refers to a computer program for completing a certain or multiple specific tasks. It runs in user mode, can interact with users, and has a visible user interface. Application icons of applications can be displayed on the display screen of the electronic device, and users can click on the application icons to open the corresponding applications.

[0036] Parallel vision is a split-screen display mode within an application, which can divide the display screen of the electronic device into multiple display windows for displaying multiple application interfaces of an application. For example: the display screen of the electronic device can be divided into two left and right display windows, and the first application interface and the second application interface of an application are respectively displayed in the two left and right display windows.

[0037] Surface: Each application program may correspond to one or more graphical interfaces, and a graphical interface may correspond to at least one surface, called surface. Each graphical interface has its position, size and content to be displayed on the screen.

[0038] The buffer is used to cache the display data after rendering each frame of the image to be displayed. The display data is used to indicate the display content of the corresponding layer, that is, the display data of the layer can be the image obtained after rendering the layer.

[0039] Buffer queue: Each layer has a corresponding buffer queue, and the buffer queue is used to store the corresponding buffer. Specifically, after the rendering operation is performed on the layer that has been drawn, after the rendering operation is completed, the rendered layer is cached in the buffer, and then the buffer is put into the buffer queue.

[0040] The following combines the display method of preview images in the current technology to compare and illustrate the advantages of a display method of preview images provided by this application.

[0041] In the current technology, when a non-system application calls the camera of an electronic device in the landscape screen state, the effective preview image in the preview image displayed by the electronic device is generally presented as a square, and there are black blocks above and below the effective preview image. The black blocks introduced in this preview image affect the display content of the electronic device, thereby affecting the user experience.

[0042] Furthermore, in the scenario where a non-system application performs face recognition, when the non-system application performs face recognition based on this preview image with black blocks introduced, it seriously affects the success rate of face recognition. Even due to the generally low success rate of face recognition by non-system applications, there may be a situation of misidentifying face information, posing a serious information security risk. Generally, in this case, when the electronic device is in the landscape screen state and the non-system application displays the preview window in the portrait screen, face recognition detection is not allowed.

[0043] When the electronic device is in the landscape screen state and a non-system application calls the camera, at this time, the preview window displayed by the electronic device is in the portrait screen display state, that is, the display direction of the preview window is in the portrait screen display. This generally occurs in the parallel vision scene, the scene where the non-system application is not compatible with the electronic device (that is, the display direction of the non-system application does not rotate with the rotation of the electronic device), or the floating window display scene. For the convenience of understanding, the following combines Figure 1 、 Figure 2 and Figure 3 to introduce the display directions of the preview windows in the parallel vision scene, the scene where the non-system application is not compatible with the electronic device, and the floating window scene respectively by way of examples. Among them, the electronic device is taken as the tablet 11.

[0044] As Figure 1 shown, when the display interface of the tablet 11 is the display scene of parallel vision, the tablet 11 displays two display windows on the left and right. Among them, the physical direction of the tablet 11 is the landscape screen direction, and the display directions of the two display windows are both in the portrait screen display. For example, the display screen of the tablet 11 simultaneously displays two application windows, namely the application window 111 and the application window 112. At this time, the application window 111 is used to display the message list interface of the communication application, and the application window 112 displays the preview window, which is displayed to the user in response to the operation of the communication application calling the camera of the tablet 11. In the current technology, at this time, the preview image of this preview window (that is, the camera preview image displayed in the application window 112) is affected by the black blocks.

[0045] As Figure 2As shown, the physical orientation of the tablet 11 is the landscape orientation. After a non-system application calls the camera of the tablet 11, the preview window 113 is displayed in the portrait orientation. Since this non-system camera application is not adapted to the tablet 11, that is, the display orientation of this non-system camera application does not rotate with the rotation of the tablet 11. At this time, when the non-system camera application calls the camera of the tablet 11, the preview window 113 is displayed in the portrait orientation on the tablet 11. In the current technology, at this time, the preview image displayed in the preview window 113 carries black blocks.

[0046] As Figure 3 shown, the physical orientation of the tablet 11 is the landscape orientation. After a non-system application calls the camera of the tablet 11, the preview window of this non-system application is displayed in the portrait orientation through the floating window 114. In the current technology, at this time, the preview image displayed through the floating window 114 is affected by black blocks.

[0047] Based on the above Figures 1 - 3 schematic diagram of the interface of the electronic device shown, in the current technology, when the physical orientation of the electronic device is the landscape orientation (the electronic device is in the landscape state) and the display orientation of the preview window displayed by the non-system application is the portrait orientation, the preview image displayed in the preview window is affected by black blocks.

[0048] After research, the inventor found the reason why black blocks appear in the preview image displayed in the preview window. The following will be introduced in detail in combination with Figures 4 - 5B it.

[0049] First, in combination with Figure 4 introduce the display orientation of the original preview image output by the hardware abstraction layer when the electronic device is in different display states. The display orientation of the original preview image output by the hardware abstraction layer is related to the physical orientation of the electronic device and the photosensitive element in the camera. As Figure 4 shown in (a) below, when the electronic device 10 is in the portrait state, the display orientation of the original preview image output by the hardware abstraction layer is 90 degrees counterclockwise different from the direction of the human body 20 seen by the human eye; as Figure 4 shown in (b) below, when the electronic device 10 is in the landscape state, the display orientation of the original preview image output by the hardware abstraction layer is the same as the direction of the human body 20 seen by the human eye.

[0050] When the physical orientation of the electronic device is in the landscape orientation (i.e., the electronic device is in the landscape state), and after a non-system application calls the camera and the display orientation of the preview window shown is in the portrait orientation, after the non-system application receives the preview image, it will rotate the preview image 90 degrees clockwise to obtain the final preview image, and then display the final preview image through the preview window. When the electronic device is in the landscape state, the display orientation of the original preview image output by the hardware abstraction layer is the same as the direction seen by the human eye. If the original preview image is directly sent to the non-system application without any adjustment, and the non-system application rotates the original preview image 90 degrees clockwise, it will cause the direction of the final preview image to be different from the direction seen by the human eye.

[0051] To ensure that when the electronic device is in the landscape state, the display orientation of the preview image shown in the preview window of the non-system application is the same as the direction seen by the human eye, before the non-system application receives the preview image, it is necessary to rotate the original preview image 90 degrees counterclockwise and then send it to the non-system application. In this way, after the non-system application rotates the received preview image 90 degrees clockwise, the final preview image obtained is the same as the direction seen by the human eye.

[0052] The current technology realizes the display process of the preview image based on a cache queue. The sizes of the buffers in a cache queue are the same. Therefore, the sizes of the original preview image and the preview image received by the non-system application are the same, both being the size of the cache. When the size of the original preview image after being rotated 90 degrees counterclockwise is different from the size of the cache, it is necessary to adjust the size of the image after the original preview image is rotated 90 degrees counterclockwise to the cache size in order to correctly display the preview image. In the current technology, the original preview image is cropped and then rotated 90 degrees counterclockwise, and then superimposed on an image of the cache size to obtain a preview image of the cache size, so as to correctly display the preview image. Cropping the original preview image is to be able to superimpose it on an image of the cache size after being rotated counterclockwise.

[0053] For ease of understanding, the following combines Figure 5A and Figure 5B to briefly introduce how to correctly display the preview image in the current technology.

[0054] Such as Figure 5AAs shown, the original preview image 501 with a resolution of 1920*1080 output by the hardware abstraction layer is cropped by height to obtain a preview image 502 with a resolution of 1080*1080 (i.e., a square preview image 502 with the height as the side length). The preview image 502 is rotated counterclockwise by 90 degrees to obtain the rotated preview image 503. At the same time, the entire original preview image 501 is set to black to obtain the preview image 504 after being set to black (with a size of 1920*1080 resolution). The rotated preview image 503 is superimposed on the preview image 504 after being set to black to obtain the preview image 505 to be output to the application layer (with a size of 1920*1080 resolution). The preview image 505 is sent to the non-system application of the application layer, and the preview image 505 is rotated clockwise by 90 degrees to obtain the final preview image 506. The electronic device displays the final preview image 506 to achieve the correct display of the preview image. Among them, cropping the original preview image 501 with a resolution of 1920*1080 to obtain the preview image 502 with a resolution of 1080*1080 is to ensure that after the preview image 502 is rotated counterclockwise by 90 degrees, it can be superimposed on the preview image 504 with a resolution of 1920*1080 while retaining as much display information carried by the preview image as possible. If the preview image 503 is directly superimposed on the original preview image 501, the display information carried by the original preview image 501 will seriously affect the display of the preview image 503. Therefore, the entire original preview image 501 is set to black to obtain the preview image 504 for superimposing the preview image 503, avoiding the influence on the display of the preview image 503. Specifically, at this time, the preview image displayed by the electronic device is as Figure 5B shown, black blocks are introduced above and below the preview image displayed by the electronic device, affecting the display content of the electronic device.

[0055] The present application provides a method for displaying a preview image. In response to an operation of a non-system application calling the camera of an electronic device, an original preview image of a preset square is obtained, and the side length of the preset square is the width of the target size of the preview image to be displayed in the preview window; based on the physical orientation of the electronic device, the display orientation of the preview window, and the target size, the original preview image of the preset square is adjusted to obtain a preview image of the target size. After the preview image of the target size is rotated, it is displayed through the preview window corresponding to the non-system application. Since the side length of the preset square is the width of the target size, that is, the long side in the target size, adjusting the original preview image of the preset square to obtain a preview image of the target size can be understood as adjusting the original preview image with a larger height to a preview image with a smaller height, and the width of the original preview image and the adjusted preview image is the same (i.e., the width remains unchanged). During the entire display process of the preview image, there is no need to perform overlay processing on the target-size image that is completely blacked out. Therefore, there is no black block affecting the preview image displayed on the electronic device.

[0056] Furthermore, since there is no black block affecting the preview image displayed on the electronic device, when performing biometric detections such as face recognition based on the displayed preview image, the accuracy and success rate of the biometric detection are improved to a certain extent.

[0057] Embodiment 1:

[0058] The following combines Figures 6 - 13 to introduce in detail a method for displaying a preview image provided by an embodiment of the present application.

[0059] Before the hardware abstraction layer outputs the original preview image, it is necessary to pre-perform the flow allocation process of the buffer queue. The following combines Figures 6 - 8 to introduce in detail the above-mentioned flow allocation process.

[0060] In some embodiments, the operating system of the electronic device may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device. As shown in Figure 6 the software structure block diagram of the electronic device in the embodiments of the present application.

[0061] The layered architecture divides software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments of the present application, the operating system is divided into three layers, namely, from top to bottom, the application layer (Android Application Package, APK) 61, the application framework layer (Framework, FWK) 62, and the hardware abstraction layer (Hardware Abstraction Layer, HAL) 63.

[0062] Among them, the application layer 61 may include a series of application packages. For example: the first camera application 611.

[0063] The application framework layer 62 provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer 62 includes some predefined functions. For example: the camera interface module 621, the camera management module 622, the utility class module 623, and the camera 3.0 module.

[0064] The hardware abstraction layer 63 may include multiple library modules, such as the camera module 631. The Android system can load the corresponding library modules for the device hardware, thereby achieving the purpose of the application framework layer accessing the device hardware.

[0065] It should be noted that although the embodiments of the present application are described by taking the Android system as an example, the basic principle is equally applicable to electronic devices based on operating systems such as iOS and Windows.

[0066] Based on Figure 6 the layered framework of the operating system of the electronic device shown, combined with Figure 7 and Figure 8 the following will introduce the flow distribution process provided by the embodiments of the present application by way of example.

[0067] S701. The first camera application 611 sends a layer to the application framework layer 62.

[0068] In a possible implementation, when the first camera application 611 detects a user request to open the camera, the first camera application 611 invokes the camera of the electronic device, and the first camera application 611 sends a layer to the application framework layer 62. For example, the first camera application 611 is a "beauty camera application". The user clicks on the icon of the beauty camera application on the electronic device to open the beauty camera application. The electronic device displays the home page interface of the beauty camera application to the user. This home page interface does not include a preview window. The user can request to open the camera by clicking buttons such as "Take Photo" or "Video". The beauty camera application detects the user's request to open the camera. The beauty camera application invokes the camera, and the electronic device will display a preview window of the beauty camera to the user. When the beauty camera application invokes the camera, the beauty camera application sends a layer to the application framework 62. It can be understood that when the first camera application 611 invokes the camera, the first camera application 611 sends a layer to the application framework layer 62.

[0069] Among them, the size of the layer sent by the first camera application 611 to the application framework layer 62 is configured by the first camera application 611 itself. That is, the size of this layer is related to the configuration of the first camera application 611 itself. When different applications invoke the camera, the sizes of the layers sent to the application framework layer 62 may be different. In a possible implementation, for the convenience of detection by the electronic device, generally, the size of the layer is described by the resolution.

[0070] S702. The camera interface module 621 converts the format of the received layer into a local format.

[0071] In a possible implementation, the camera interface module 621 invokes createCaptureSession to convert the layer into a local format. The local format can be OutputConfiguration, which is used to describe the configuration information of an output stream.

[0072] It should be noted that the camera interface module 621 is stored in the application framework layer 62 of the operating system of the electronic device in the form of a source file of a programming language program. For example, CameraDeviceImpl.java is a source file of a Java programming language program. In addition, the programming language program can also be a C++ language, a C language, etc. This application does not make specific limitations.

[0073] S703. The camera interface module 621 detects whether the received layer has been configured.

[0074] In a possible implementation, the camera interface module 621 can store the received layer sent by the application layer 61, and then, based on the stored layer, detect whether the received layer has been configured. If so, it indicates that the flow allocation process corresponding to this layer has been configured and there is no need to continue with the flow allocation; if not, then S704 is performed.

[0075] S704. The camera interface module 621 sends the received layer to the camera management module 622.

[0076] It should be noted that the camera management module 622 is stored in the application framework layer 62 of the operating system of the electronic device in the form of the source file of a programming language program. For example: CameraDeviceClient.cpp, which is the source file of a C++ programming language program.

[0077] In a possible implementation, the camera interface module 621 is stored in the application framework layer 62 of the operating system of the electronic device in the form of CameraDeviceImpl.java, and the camera management module 622 is stored in the application framework layer of the operating system of the electronic device in the form of CameraDeviceClient.cpp. At this time, the camera interface module 621 can call the mRemoteDevice.createStream function to transfer the layer from the java layer to the native layer.

[0078] S705. The camera management module 622 sends the layer to the utility class module 623.

[0079] Among them, the utility class module 623 includes various functions required to obtain the flow allocation data carried by the layer.

[0080] It should be noted that the utility class module 623 is stored in the application framework layer 62 of the operating system of the electronic device in the form of the source file of a programming language program. For example: SessionConfigurationUtils.cpp, which is the source file of a C++ programming language program.

[0081] S706. The utility class module 623 obtains the flow allocation data of this layer.

[0082] Among them, the flow allocation data includes: layer size, stream name (format), data space (dataspace), etc. Among them, the layer size is represented in the format of width * height, and generally is described in the form of resolution, that is, in units of pixels. For example: the layer size is 1920*1080, which means the width of the layer is 1920 pixels and the height of the layer is 1080 pixels.

[0083] S707. The tool class module 623 returns the flow allocation data of the layer to the camera management module 622.

[0084] S708. The camera management module 622 sends the received flow allocation data of the layer to the camera 3.0 module 624.

[0085] It should be noted that the camera 3.0 module 624 is stored in the application program framework layer of the operating system of the electronic device in the form of a source file of a programming language program. For example: Camera3Device.cpp, which is a source file of a C++ programming language program.

[0086] S709. The camera 3.0 module 624 determines whether the preset conditions are met.

[0087] When the camera 3.0 module 624 determines that the preset conditions are met, then S710 is performed.

[0088] When the camera 3.0 module 624 determines that the preset conditions are not met, then S711 is performed.

[0089] S710. The camera 3.0 module 624 modifies the layer size in the flow allocation data of the layer to width * width, and creates a corresponding buffer transfer management object.

[0090] Among them, the layer size in the flow allocation data of the layer received by the camera 3.0 module 624 is width * height, and the layer size is modified from width * height to width * width. For example: the layer size in the flow allocation data received by the camera 3.0 module 624 is 1920 * 1080, and the layer size is modified to 1920 * 1920. After the hardware abstraction layer 63 completes the flow allocation based on the modified layer size, the hardware abstraction layer 63 can output a square original preview image with a resolution of 1920 * 1920, so as to realize the cropping and rotation of the square original preview image based on the size, and the display of the preview image can be realized without blackening and overlapping, avoiding the black blocks in the displayed preview image affecting the display of the preview image.

[0091] Among them, the corresponding buffer transfer management object is Camera3MultiWinOutputStream, which is a subclass of Camera3OutputStream and is a newly defined buffer transfer management object in this application.

[0092] Specifically, the Camera3MultiWinOutputStream internally includes two groups of buffer queues, which are respectively used for the transfer of buffers of two different sizes. Specifically, one group of buffer queues is used for the transfer between the application framework layer 62 and the hardware abstraction layer 63, and the size of this buffer is the modified layer size of the camera 3.0 module 624, that is, the width * width of the layer; one group of buffer queues is used for the transfer between the application layer 61 and the application framework layer 62, and the size of this buffer is the size of the layer, that is, the width * height of the layer.

[0093] S711. The camera 3.0 module 624 keeps the layer size in the received layer allocation data unchanged and creates a corresponding buffer transfer management object.

[0094] It should be noted that when the camera 3.0 module 624 determines that the preset conditions are not met, other allocation processes and preview image display methods are executed. For example, the existing allocation process and preview image display methods in the current technology are executed. The above S711 only describes one step in an existing allocation process. In addition, other allocation processes can be executed, and the present application does not make specific limitations.

[0095] Among them, the corresponding buffer transfer management object is Camera3OutputStream. Camera3OutputStream internally includes a group of buffer queues for buffer transfer, and the size of this buffer is the size of the received layer, that is, the width * height of the layer.

[0096] S712. The camera 3.0 module 624 sends an allocation request to the hardware framework layer 63.

[0097] Among them, the allocation request carries the layer allocation data.

[0098] S713. The hardware framework layer 63 completes the allocation according to the allocation data carried in the received allocation request.

[0099] When the hardware framework layer 63 completes the allocation, the preview image is displayed based on the buffer queue obtained from the allocation. That is, when the hardware abstraction layer 63 completes the allocation, the electronic device realizes the display of the preview image collected by the camera through the allocation data.

[0100] The above is as Figure 7In the flow of distributing current in the electronic device shown, what is different from the current technology is that when S709 determines that the preset conditions are met, S710 is performed; when S709 determines that the preset conditions are not met, S711 is performed. It can be understood that when the preset conditions are met, the electronic device executes the preview image display method provided in the embodiments of the present application; when the preset conditions are not met, the electronic device executes the preview image display method in the current technology (that is, does not execute the preview image display method provided in the embodiments of the present application).

[0101] For ease of understanding, the following will be combined with Figure 8 to introduce in detail how the camera 3.0 module 624 determines whether the preset conditions are met. As Figure 8 shown, the method for determining whether the preset conditions are met includes:

[0102] S801. Obtain a specific private label of the electronic device.

[0103] Among them, the private label (TAG) of the electronic device refers to a specific identifier or label of the electronic device, which is defined by the manufacturer or manufacturer. In the embodiments of the present application, a new private label is pre-defined in the hardware abstraction layer 63 of the electronic device as the basis for determining whether the electronic device can perform the preview image display method provided in the embodiments of the present application, so as to achieve product isolation of the electronic device.

[0104] In a possible implementation manner, a new private label can also be defined in the hardware abstraction layer 63, which can also be called a capability label (tag): "com.hihonor.device.capabilitoes.multiwindow".

[0105] S802. Based on the obtained specific private label, determine whether the specific private label is not empty.

[0106] When the specific private label is not empty, S803 is performed.

[0107] When the specific private label is empty, the preset conditions are not met.

[0108] If the specific private label is not empty, it indicates that the electronic device supports the display method of the preview screen provided in the embodiments of the present application; when the specific private label is empty, it indicates that the electronic device does not support the display method of the preview screen provided in the embodiments of the present application. Product isolation of the electronic device is achieved through the specific private label. In actual use, there are situations where the electronic device does not support the parallel vision scenario, does not include non-system applications that do not adapt to the landscape screen state, or does not support the floating window scenario. For example, a smart watch does not support the parallel vision scenario and the floating window scenario, and all non-system applications that can be downloaded to the smart watch can adapt to the smart watch in various states (non-system applications that do not adapt to the smart watch cannot be downloaded to the smart watch). At this time, the specific private label of the electronic device is empty, avoiding the display method of the preview image provided in the embodiments of the present application from being performed when there is no problem with the display of the preview image on the electronic device, resulting in unnecessary power consumption.

[0109] S803. Determine whether the stream name of the layer sent by the first camera application 611 is 34 or 35.

[0110] Among them, a stream name (format) of 33 represents a photographing stream; a stream name of 34 represents a preview stream; a stream name of 35 represents a callback stream. For example: when the first camera application 611 calls the camera and displays a preview window, and the user does not perform a photographing operation, the electronic device dynamically displays the preview image captured by the camera in real time through the preview window. At this time, the stream name of the layer corresponding to the image is 34 (preview stream); when the first camera application 611 calls the camera and displays a preview window, and the user performs a photographing operation, the electronic device displays the static image obtained by photographing through the camera through the preview window. At this time, the stream name of the layer corresponding to the image is the photographing stream; when the first camera application 611 has a hidden space that requires face recognition to enter, and the user requests to enter the hidden space, the first camera application 611 calls the camera to obtain an image including face information in real time for face recognition and display. At this time, the stream name of the layer corresponding to the image is the callback stream.

[0111] It can be understood that the display method of the preview image provided in the embodiments of the present application is only applicable to scenarios where the stream name is 34 or 35. Of course, the above 34 and 35 are numerical representations of the stream names of the preview stream and the callback stream. In addition, it can also be limited by other numerical values or other names. The present application does not make specific limitations.

[0112] When it is determined that the stream name of the layer sent by the first camera application 611 is 34 or 35, then perform S804.

[0113] When it is determined that the stream name of the layer sent by the first camera application 611 is not 34 or 35, the preset condition is not met.

[0114] S804. Determine whether the package name of the first camera application 611 is in a preset configuration file.

[0115] Among them, the preset configuration file is a configuration file newly created in the hardware framework layer 63 in advance, which is used to record the package names of application programs that are applicable to the preview image display method provided in the embodiments of the present application. Through this preset configuration file, isolation between applications is achieved. Since in daily use, there are applications that are not applicable to the preview image display method provided in the embodiments of the present application. For example, video conferencing applications. Such video conferencing applications may have long video conferences and require real-time display of preview images for a long time. If the preview image display method provided in the actual example of the present application is used, it will cause relatively high power consumption. To ensure the performance of the electronic device, generally, the package names of video conferencing applications are not set in the preset configuration file to isolate video conferencing applications.

[0116] When the package name of the first camera application 611 is in the preset configuration file, proceed to S805.

[0117] When the package name of the first camera application 611 is not in the preset configuration file, the preset condition is not met.

[0118] In a possible implementation, after determining that the package name of the first camera application 611 is in the preset configuration file, it further includes determining whether the first camera application 611 is a system camera application. When the first camera application 611 is not a system camera application, then proceed to S805; when the first camera application 611 is a system camera application, the preset condition is not met. For example, it can be determined whether the first camera application 611 is a system camera application based on whether the package name of the first camera application 611 is the same as the package name of the system camera application.

[0119] Since the preset configuration file is configured manually by programming, to avoid errors in the manually configured configuration file that affect the overall judgment of whether the preset condition is met, the determination of whether the first camera application 611 is a system camera application is added, adding an extra layer of insurance for the judgment of whether the preset condition is met, and further ensuring the accuracy of the judgment of whether the preset condition is met.

[0120] S805. Determine whether the width of the layer sent by the first camera application 611 is greater than the height.

[0121] When the width of the layer sent by the first camera application 611 is equal to the height, it indicates that the preview image requested by the first camera application 611 is a square preview image, and there is no black block in the finally displayed preview image. For example, in the current technology, the hardware abstraction layer 63 outputs an original preview image with a size of 1080*1080. Only rotation is performed on the original preview image with a size of 1080*1080 to obtain a rotated preview image with a size of 1080*1080. The size of the rotated preview image is the same as the size of the original preview image, that is, the same as the size of the preview image that the first camera application 611 needs to receive, and no cropping or blackening processing is required. Therefore, there is no black block affecting the finally displayed preview image. It can be seen that for the case where the width of the layer is equal to the height, the preview image displayed by the electronic device is not affected by black blocks, so there is no need to perform the preview image display method provided in the embodiments of the present application.

[0122] When the width of the layer sent by the first camera application 611 is less than the height, there are other unavoidable problems in the portrait display of the first camera application 611 at this time, so it does not involve the preview image display method provided in the embodiments of the present application.

[0123] It should be noted that the size of the preview image requested by the first camera application 611 (as well as other applications) from the hardware abstraction layer 63 is generally such that the width is greater than the height.

[0124] When the width of the layer sent by the first camera application 611 is greater than the height, then S806 is performed.

[0125] When the width of the layer sent by the first camera application 611 is not greater than the height, the preset condition is not satisfied.

[0126] S806. Determine whether the width of the layer sent by the first application camera 611 is less than the maximum height of the electronic device.

[0127] There is a tag (TAG) in the hardware abstraction layer 61 of the electronic device that stores the maximum size of the image (layer) allowed to be output by the hardware abstraction layer 61 of the electronic device. The maximum size of the image allowed to be output by the hardware abstraction layer 61 of the electronic device stored in this tag is: maximum width * maximum height. It should be noted that it is stored in this tag in the form of width * height and the unit is generally in the form of resolution. Further, it is necessary to ensure that the unit of the width of the layer sent by the first application camera 611 is the same as the unit of the maximum height.

[0128] In a possible implementation manner, the maximum size of the image allowed to be output by the hardware abstraction layer 61 of the electronic device is stored in a tag named "activeArraySize".

[0129] When the width of the layer sent by the first application camera 611 is not less than the maximum height of the electronic device, it indicates that the hardware abstraction layer 61 of the electronic device does not support outputting a preview image with a size of "width of the layer * width of the layer", so the preset condition is not met. For example, the maximum size of the image allowed to be output by the hardware abstraction layer 61 of the electronic device is 4000 * 3000 resolution, while the size of the layer sent by the first camera application 611 is 3500 * 2000 resolution, and the maximum height is 3000 pixel points < the width of the layer is 3500 pixel points, so the hardware abstraction layer 61 of the electronic device does not support outputting an image with a resolution of 3500 * 3500, and the preset condition is not met.

[0130] When the width of the layer sent by the first camera application 611 is less than the maximum height of the electronic device, it is determined that the preset condition is met.

[0131] From Figure 8 As can be seen from the flowchart of determining whether the preset condition is met shown, there are 5 preset conditions, which are: a specific private label is not empty, the stream name of the layer is 34 or 35, the package name of the first camera application 611 is in the preset configuration file, the width of the layer is greater than the height of the layer, and the width of the layer is less than the preset maximum height of the electronic device. When all of the above 5 conditions are met, it is determined that the preset condition is met; as long as one condition is not met, the preset condition is not met.

[0132] It should be noted that the judgment order of the above S803 - S806 can be adjusted, and it can be executed sequentially or synchronously. This application does not make specific limitations.

[0133] As described above in combination with Figures 6 - 8 The flow matching process has been introduced in detail. When the preset condition is met, after the hardware abstraction layer 61 completes the flow matching according to the flow matching data carried in the received flow matching request (the layer size in the flow matching data is width of the layer * width of the layer), the preview image display method provided in the embodiments of this application is performed. Below in combination with Figures 9 - 12 , the preview image display method provided in the embodiments of this application will be introduced in detail.

[0134] First of all, it should be noted that Figure 9 In the preview image display method shown, the electronic device is in the landscape screen state, and the display direction of the preview window of the first camera application 611 is the portrait screen direction. Exemplarily, the target size of the layer output during the flow matching of the first camera application 611 is 1920 * 1080 resolution, and the size in the flow matching data carried in the flow matching request received by the hardware abstraction layer 63 is 1920 * 1920 resolution.

[0135] As Figure 9 shown, the preview image display method provided in the embodiments of this application includes:

[0136] S901. The hardware abstraction layer 63 outputs the original preview image of the preset square and sends it to the application framework layer 62.

[0137] Among them, the preset square is a square with the width of the layer requested by the first camera application 611 as the side length. That is, the size of the preset square is the target width * the target width. Among them, the target size is the target width * the target height.

[0138] For example: The hardware abstraction layer 63 outputs the original preview image with a resolution of 1920 * 1920 and sends it to the application framework layer 72.

[0139] S902. The application framework layer 62 crops the original preview image based on the target size.

[0140] In a possible implementation, starting from the center of the original preview image, the width of the original preview image is cropped to the target height. For example: Starting from the center of the original preview image with a resolution of 1920 * 1920, the original preview image with a resolution of 1920 * 1920 is cropped to a preview image with a resolution of 1080 * 1920.

[0141] S903. The application framework layer 62 rotates the cropped preview image.

[0142] In a possible implementation, taking the center of the cropped preview image as the rotation center, it is rotated counterclockwise by 90 degrees to obtain the preview image of the target size. For example: Taking the center of the cropped preview image with a resolution of 1080 * 1920 as the rotation center, it is rotated counterclockwise by 90 degrees to obtain the preview image of the target size of 1920 * 1080 resolution.

[0143] S904. The application framework layer 62 sends the preview image of the target size to the application layer 61.

[0144] The application framework layer 62 sending the preview image of the target size to the application layer 61 can also be understood as the application framework layer 62 sending the preview image of the target size for display.

[0145] S905. After rotating the preview image of the target size, the application layer 61 displays it through the preview window.

[0146] Since the first camera application 611 is currently still in the portrait display, after receiving the preview image of the target size, it is rotated and then displayed through the preview window.

[0147] In a possible implementation, the application layer 61 rotates the preview image with the center of the preview image of the target size as the rotation center by 90 degrees clockwise, and displays the preview image after being rotated 90 degrees clockwise. For example: with the center of the preview image of the target size of 1920*1080 resolution as the rotation center, rotate 90 degrees clockwise to obtain a preview image with a resolution of 1080*1920, and display the preview image with a resolution of 1080*1920 through the preview window.

[0148] During the execution of the preview image display method provided by the embodiments of the present application, only the original preview image is cropped and rotated, and then sent to the application layer 61. The application layer 61 rotates the received preview image and then displays it, so as to realize the correct display of the preview image. During the entire display process of the preview image, there is no need to perform blacking and overlapping processing, that is, there is no need to perform blacking processing on the original preview image of the target size, and there is no need to overlap the preview image after cropping and rotation with the preview image after blacking processing. Therefore, there is no black block influence on the preview image displayed by the electronic device.

[0149] It should be noted that Figure 9 In the preview image display method executed by the electronic device shown, for the convenience of description, the hardware abstraction layer, application framework layer, and application layer in the operating system of the electronic device are used as the execution subjects for introduction. In fact Figure 9 each step shown is executed by the functional modules in the hardware abstraction layer, application framework layer, and application layer.

[0150] Next, a comparison is made with the preview image displayed by the first camera application 611 of the electronic device shown Figure 10 Among them Figure 10 (a) in and Figure 10 (b) in are the displays of the same electronic device, the same non-system application, at the same moment, at the same angle, when the electronic device is in the landscape screen state and the display direction of the preview window of the first camera application 611 is the portrait screen direction.

[0151] As Figure 10 (a) in shows, the preview image 1001 displayed in the preview window of the electronic device introduces black block influence; as Figure 10 (b) in shows, the preview image 1002 displayed in the preview window of the electronic device does not introduce black block influence. Further, as Figure 10 shown, the sizes of the preview image 1001 and the preview image 1002 are the same. The preview image 1001 has black block influence, while the preview image 1002 does not have black block influence. Therefore, the effective preview image of the preview image 1001 is smaller than the effective preview image of the preview image 1002, that is, the preview image 1002 carries more effective data.

[0152] In summary, for the method for displaying a preview image provided in an embodiment of the present application, a functional module in the hardware abstraction layer outputs an original preview image to a functional module in the application framework layer. The original preview image is a positive-direction image with a side length equal to the target width. The functional module in the application framework layer crops and rotates the original preview image based on the target size to obtain a preview image with the target size, and sends the preview image with the target size to the functional module in the application layer. After rotating the preview image with the target size, the functional module in the application layer displays it through a preview window. During the execution of the entire method for displaying a preview image, there is no need to perform a blackening process on the original preview image and an overlay operation with the preview image after the blackening process. In this way, the preview image displayed on the electronic device is not affected by black blocks, improving the user experience. Further, since the preview image displayed on the electronic device is not affected by black blocks, when performing biometric detections such as face recognition based on the displayed preview image, there is no influence from black blocks, improving the success rate of biometric detections such as face recognition.

[0153] Further, compared with the prior art, the preview image displayed in the embodiment of the present application can carry more valid data, further improving the success rate of biometric detections such as face recognition.

[0154] The above Figure 9 The method for displaying a preview image shown is implemented based on two buffer queues created during the flow distribution process, and the management of the two buffer queues is achieved by newly defining a buffer flow management object in the present application. Among them, the two buffer queues are a first buffer queue and a second buffer queue, as specifically shown in Figure 11A the simple schematic diagram of buffer flow shown.

[0155] Among them, the first buffer queue is used for buffer flow between the application framework layer 62 and the hardware abstraction layer 63. The size of this buffer is the width of the target size * the width of the target size, which can also be referred to as the target width * the target width.

[0156] Among them, the second buffer queue is used for buffer flow between the application layer 61 and the application framework layer 62. The size of this buffer is the target size, that is, the width of the target size * the height of the target size, which can be referred to as the target width * the target width.

[0157] Next, in conjunction with Figure 11B a detailed introduction will be given on how to implement the method for displaying a preview image as shown in Figure 11A when the electronic device is in the landscape screen state and the display direction of the preview window of the first camera application 611 is the portrait screen direction, based on the two buffer queues as shown in Figure 9 the method for displaying a preview image as shown in

[0158] As Figure 11B shown, the method for displaying a preview image provided by an embodiment of the present application includes:

[0159] S1101. The hardware abstraction layer 63 applies for an idle first buffer from the first buffer queue.

[0160] For the convenience of description, the buffers in the first buffer queue are referred to as first buffers, and the buffers in the second buffer queue are referred to as second buffers.

[0161] S1102. After the hardware abstraction layer 63 stores the original preview image of the preset square in the idle first buffer, it adds the first buffer storing the original preview image to the first buffer queue.

[0162] The size of the first buffer storing the original preview image is the target width * the target width.

[0163] S1103. The application framework layer 62 obtains the first buffer storing the original preview image from the first buffer queue.

[0164] S1104. The application framework layer 62 copies the original preview image stored in the first buffer to the idle third buffer of itself, and returns the idle first buffer to the first buffer queue.

[0165] Among them, after copying the original preview image stored in the first buffer to the idle third buffer of the application framework layer 62 itself, the first buffer is emptied to be an idle first buffer, and the idle first buffer is returned to the first buffer queue to facilitate the buffer transfer between the application framework layer 62 and the hardware abstraction layer 63.

[0166] S1105. The application framework layer 62 crops and rotates the third buffer storing the original preview image.

[0167] Based on the target size, the original preview image stored in the third buffer is cropped and rotated so that the third buffer stores the preview image of the target size.

[0168] S1106. The application framework layer 62 applies for an idle second buffer from the second buffer queue.

[0169] S1107. The application framework layer 62 copies the preview image of the target size stored in the third buffer to the idle second buffer, and adds the second buffer storing the preview image of the target size to the second buffer queue.

[0170] Among them, after copying the preview image of the target size stored in the third buffer to the idle second buffer, the third buffer is emptied.

[0171] S1108. The application layer 61 obtains the second cache of the preview image storing the target size from the second cache queue.

[0172] S1109. The application layer 61 rotates and displays the preview image of the target size stored in the second cache, and returns the idle second cache after the display is completed to the second cache queue.

[0173] After the application layer 61 rotates and displays the preview image of the target size stored in the second cache, the second cache is cleared, and the idle second cache is returned to the second cache queue to facilitate the buffer transfer between the application framework layer 62 and the application layer 61.

[0174] As Figure 11B shown, the buffer transfer of the preset square between the application framework layer 62 and the hardware abstraction layer 63 and the buffer transfer of the target size between the application layer 61 and the application framework layer 62 are respectively implemented through two cache queues, so as to implement the preview image display method provided in the embodiment of the present application.

[0175] Furthermore, when the electronic device is in the portrait state and the display direction of the preview window of the first camera application 611 is the portrait direction, in the prior art, after the first camera application 611 receives the original preview image of the target size output by the hardware abstraction layer 63, the first camera application 611 rotates the original preview image clockwise by 90 degrees to obtain the final preview image, and displays the preview image through the preview window. At this time, the preview image displayed by the electronic device does not introduce black blocks and does not affect the display of the preview image.

[0176] It can be seen that when the electronic device is in the vertical screen state and the display direction of the preview window of the first camera application 611 is the vertical screen direction, if the existing technology is used for the display method of the preview image, the normal display of the preview image can be achieved, and there is no black block affecting the preview image. At this time, the original preview image output by the hardware abstraction layer 63 in the existing technology is the target size (width of the target size * height of the target size). When the electronic device is in the horizontal screen state and the display direction of the preview window of the first camera application 611 is the vertical screen direction, the display method of the preview image provided by the embodiments of the present application needs to be used to achieve the normal display of the preview image, and there is no black block affecting the preview image. At this time, the size of the original preview image output by the hardware framework layer 63 is the width of the target size * the width of the target size. It can be seen that the sizes of the original preview images output by the hardware framework layer 63 in the two scenarios are different in the above cases. When the electronic device switches between the vertical and horizontal screen states, it is necessary to disconnect the camera and re - perform stream allocation, which may cause a brief black screen phenomenon in the display of the preview image during the switching process of the electronic device.

[0177] In order to avoid the brief black screen phenomenon in the display of the preview image during the vertical - horizontal screen switching process of the electronic device, for the scenario where the electronic device is in the vertical screen state and the display direction of the preview window of the first camera application 611 is the vertical screen direction, the embodiments of the present application provide another display method for the preview image, including: the hardware framework layer 63 outputs a preset square original preview image to the application framework layer 62, the application framework layer 62 crops the original preview image to obtain a preview image of the target size, and sends it to the application layer 61. The application layer 61 rotates the received preview image and then displays it through the preview window.

[0178] For the convenience of understanding, the following combines Figure 12 , and details another display method for the preview image provided by the embodiments of the present application in the scenario where the electronic device is in the vertical screen state and the display direction of the preview window of the first camera application 611 is the vertical screen direction. Exemplarily, when the target size of the layer output by the first camera application 611 during stream allocation is 1920 * 1080 resolution, and the size in the stream allocation data carried in the stream allocation request received by the hardware abstraction layer 61 is 1920 * 1920 resolution.

[0179] As Figure 12 shown, the display method for the preview image provided by the embodiments of the present application includes:

[0180] S1201. The hardware abstraction layer 63 outputs a preset square original preview image and sends it to the application framework layer 62.

[0181] Among them, the preset square is a square with the target width of the layer requested by the first camera application 611 as the side length.

[0182] S1202. The application framework layer 62 crops the original preview image based on the target size.

[0183] In a possible implementation, starting from the center of the original preview image, the height of the original preview image is cropped to the height of the target size to obtain a preview image of the target size. For example: starting from the center of the original preview image with a resolution of 1920*1920, the original preview image with a resolution of 1920*1920 is cropped to a preview image with a resolution of 1920*1080.

[0184] S1203. The application framework layer 62 sends the preview image of the target size to the application layer 61.

[0185] The application framework layer 62 sending the preview image of the target size to the application layer 61 can also be understood as the application framework layer 62 presenting the preview image of the target size.

[0186] S1204. After rotating the preview image of the target size, the application layer 61 displays it through the preview window.

[0187] Since the first camera application 611 is currently still in the portrait display, after receiving the preview image of the target size, it is rotated and then displayed through the preview window.

[0188] In a possible implementation, the application layer 61 rotates 90 degrees clockwise with the center of the preview image of the target size as the rotation center, and displays the preview image after rotating 90 degrees clockwise. For example: with the center of the preview image with a resolution of 1920*1080 of the target size as the rotation center, it is rotated 90 degrees clockwise to obtain a preview image with a resolution of 1080*1920, and this preview image with a resolution of 1080*1920 is displayed through the preview window.

[0189] From Figure 9 and Figure 12 It can be seen from the preview image display method shown that the sizes of the original preview images output by the hardware framework layer in the two scenarios are the same (i.e., a square with the target width as the side length), and there is no need to reconfigure the stream, avoiding the short-term black screen phenomenon during the horizontal and vertical screen state switching process of the electronic device, and further improving the user experience.

[0190] It should be noted that when the electronic device is in the vertical screen state and the display direction of the preview window of the first camera application 611 is the vertical screen direction, based on the two sets of buffer queues shown in Figure 11A as shown in Figure 12 shown, the specific implementation process is similar to the process shown in Figure 11B Specifically,Figure 11B Different from S1105 shown, the application framework layer 62 can crop the third cache storing the original preview image.

[0191] It should be noted that the preview image display methods described in the embodiments of this application are all described by taking a non-system application calling the camera as an example. During actual use, when a system application calls the camera, the electronic device can also execute the preview image display method provided in the embodiments of this application, which is not specifically limited.

[0192] Embodiment 2:

[0193] Next, in combination with Figure 13 , a preview image display method provided in the embodiments of this application will be introduced in detail. This method is applied to an electronic device, and the electronic device includes a camera and a first application.

[0194] S1301. Receive an operation in which the user calls the camera through the first application.

[0195] In a possible implementation manner, the user can click the capture button of the first application to enter the interface including the preview window of the first application. The click of the capture button of the first application is an operation in which the user calls the camera through the first application.

[0196] S1302. In response to the operation of the first application calling the camera, obtain a square original preview image.

[0197] Among them, the side length of the square is the width of the target size, and the width of the target size is greater than the height of the target size.

[0198] In a possible implementation manner, in response to the operation of the first application calling the camera, obtain the layer corresponding to the first application, and the size of the layer corresponding to the first application is the target size; when it is determined that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions, obtain a square original preview image.

[0199] Further, the method for determining that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions includes:

[0200] Determine that the first label of the electronic device is not empty. The first label of the electronic device is used to represent whether the electronic device supports the preview image display method provided in the embodiments of this application. When the first label is empty, the electronic device does not support the preview image display method provided in the embodiments of this application; when the first label is not empty, the electronic device supports the preview image display method provided in the embodiments of this application.

[0201] Determine that the first application is in the application whitelist. The application whitelist includes applications that support the display method of the preview image provided by the embodiments of the present application. When the first application is in the application whitelist, the first application supports the display method of the preview image provided by the embodiments of the present application; when the first application is not in the application whitelist, the first application does not support the display method of the preview image provided by the embodiments of the present application.

[0202] Determine that the stream name of the layer corresponding to the first application is the preview stream or the callback stream, and determine that the width of the target size is less than the maximum height corresponding to the electronic device; the display method of the preview image provided by the present application is applicable to the image display of the preview stream and the callback stream, and moreover, only when the maximum height of the preview image that the electronic device allows to obtain is greater than the width of the target size, can the original preview image in the shape of a square with the width of the target size be obtained. Among them, the maximum height corresponding to the electronic device is generally stored in the second tag of the electronic device. Among them, the width of the target size and the stream name are the stream allocation data carried by the layer corresponding to the first application.

[0203] Further, when it is determined that the electronic device, the first application, and the layer corresponding to the first application meet the preset conditions, create a first cache queue and a second cache queue; the first cache queue includes a first cache, the width of the first cache is the width of the target size, and the height of the first cache is the width of the target size; the second cache queue includes a second cache, the width of the second cache is the width of the target size, and the height of the second cache is the height of the target size. Among them, the first cache in the first cache queue is used to store the original preview image in the shape of a square, and the second cache in the second cache queue is used to store the first preview image of the target size.

[0204] S1303. Process the original preview image to obtain the first preview image of the target size.

[0205] In a possible implementation manner, when the electronic device is in the landscape display state, crop the width of the original preview image to the height of the target size to obtain the cropped preview image; rotate the cropped preview image to obtain the first preview image of the target size.

[0206] Specifically, starting from the center of the original preview image, crop the width of the original preview image to the height of the target size to obtain the cropped preview image; and taking the center of the cropped preview image as the rotation center, rotate the cropped preview image to obtain the first preview image of the target size.

[0207] In a possible implementation manner, when the electronic device is in the portrait display state, crop the height of the original preview image to the height of the target size to obtain the first preview image of the target size.

[0208] In a possible implementation, a first cache storing the original preview image is retrieved from the first cache queue, and the original preview image stored in the first cache is copied to the third cache; the original preview image stored in the third cache is processed to obtain a first preview image of the target size. The third cache is the idle cache of the electronic device itself. Further, the first preview image of the target size is stored in the second cache.

[0209] S1304. Rotate the first preview image to obtain a second preview image.

[0210] Wherein, the width of the second preview image is the height of the target size, and the height of the second preview image is the width of the target size.

[0211] S1305. Vertically display the preview window of the first application, and the second preview image is displayed in the preview window.

[0212] The embodiment of the present application provides a method for displaying a preview image, which is applied to an electronic device. The electronic device includes a camera and a first application, and includes: receiving an operation of the user to call the camera through the first application; in response to the operation of the first application to call the camera, obtaining a square original preview image; wherein, the side length of the square is the width of the target size, and the width of the target size is greater than the height of the target size; processing the original preview image to obtain a first preview image of the target size, rotating the first preview image to obtain a second preview image; vertically displaying the preview window of the first application, and the second preview image is displayed in the preview window. In the present application, the original preview image is a square with the width (the larger side) of the target size as the side length. By processing the original preview size, a preview image that meets the target size can be obtained, so as to realize the normal display of the preview image. During the entire display process of the preview image, no blackening process or overlapping process is required, so there is no black block influence on the preview image displayed by the electronic device.

[0213] In some embodiments, the electronic device may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a notebook computer, an Ultra-mobile Personal Computer (UMPC), a handheld computer, a netbook, a Personal Digital Assistant (PDA), a wearable electronic device, etc. The present application does not impose special restrictions on the specific form of the above-mentioned electronic devices. In this embodiment, the structure of the electronic device may be as Figure 14 shown Figure 14 which is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0214] Such as Figure 14As shown, the electronic device may include a camera 110, a processor 120, an internal memory 130, a display screen 140, etc.

[0215] The processor 120 may include one or more processing units. For example, the processor 120 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0216] A memory may also be set in the processor 120 for storing instructions and data. In some embodiments, the memory in the processor 120 is a cache memory. This memory can save the instructions or data that the processor 120 has just used or recycled. If the processor 120 needs to use this instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 120, and thus improves the efficiency of the system.

[0217] The MIPI interface can be used to connect the processor 120 to peripheral devices such as the display screen 140 and the camera 110. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. For example, the processor 120 and the display screen 140 communicate through the DSI interface to implement the display function of the electronic device.

[0218] The electronic device implements the display function through the GPU, the display screen 140, and the application processor, etc. The GPU is a microprocessor for image processing, which connects the display screen 140 and the application processor.

[0219] The display screen 140 is used to display images, videos, etc. The display screen 140 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oled, a quantum dot light-emitting diode (QLED), etc.

[0220] The camera 110 is used to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. In some embodiments, the electronic device may include one or more cameras 110.

[0221] The internal memory 130 can be used to store computer-executable program codes. The processor 120 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 130. In addition, the internal memory 130 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 120 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 130, and / or the instructions stored in the memory provided in the processor.

[0222] An embodiment of this application also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run, the electronic device implements the above-mentioned various functions or steps.

[0223] An embodiment of this application also provides a computer program product, including a computer program or instruction. When the computer program or instruction is executed by a processor, the electronic device implements the above-mentioned various functions or steps.

[0224] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claimed rights.

Claims

1. A method for displaying a preview image, characterized in that, Applied to an electronic device, the electronic device includes a camera and a first application, and the method includes: Receiving an operation by which a user invokes the camera through the first application; In response to the operation of invoking the camera by the first application, obtaining an original preview image in a square shape; the side length of the square is the width of a target size; the width of the target size is greater than the height of the target size; Processing the original preview image to obtain a first preview image of the target size; Rotating the first preview image to obtain a second preview image; the width of the second preview image is the height of the target size, and the height of the second preview image is the width of the target size; Vertically displaying a preview window of the first application, where the second preview image is displayed in the preview window; When the electronic device is in a landscape screen state, the processing the original preview image to obtain the first preview image of the target size includes: Cropping the width of the original preview image to the height of the target size to obtain a cropped preview image; Rotating the cropped preview image to obtain the first preview image of the target size; The responding to the operation of invoking the camera by the first application and obtaining an original preview image in a square shape includes: Responding to the operation of invoking the camera by the first application, obtaining a layer corresponding to the first application, and the size of the layer corresponding to the first application is the target size; When it is determined that the electronic device, the first application, and the layer corresponding to the first application meet a preset condition, modifying the height of the layer corresponding to the first application to the width of the target size to obtain the original preview image in a square shape.

2. The method according to claim 1, characterized in that, When the electronic device is in a portrait screen display state, the processing the original preview image to obtain the first preview image of the target size includes: Cropping the height of the original preview image to the height of the target size to obtain the first preview image of the target size.

3. The method according to claim 1, characterized in that, The manner of determining that the electronic device, the first application, and the layer corresponding to the first application meet the preset condition includes: Determining that a first label of the electronic device is not empty; Determining that the first application is in an application whitelist; Determining that the stream name of the layer corresponding to the first application is a preview stream or a callback stream, and determining that the width of the target size is less than the maximum height corresponding to the electronic device; the maximum height corresponding to the electronic device is used to indicate the maximum height of the preview image obtained by the electronic device.

4. The method according to claim 3, characterized in that, The width of the target size and the stream name are allocation data carried by the layer corresponding to the first application.

5. The method according to claim 1, characterized in that, When it is determined that the electronic device, the first application, and the layer corresponding to the first application meet the preset condition, the method further includes: Create a first cache queue and a second cache queue; the first cache queue includes a first cache, the width of the first cache is the width of the target size, and the height of the first cache is the width of the target size; the second cache queue includes a second cache, the width of the second cache is the width of the target size, and the height of the second cache is the height of the target size; After obtaining the original preview image of the square, it further includes: storing the original preview image of the square in the first cache; After processing the original preview image to obtain the first preview image of the target size, it further includes: storing the first preview image of the target size in the second cache.

6. The method according to claim 5, characterized in that, Processing the original preview image to obtain the first preview image of the target size includes: Obtain the first cache storing the original preview image from the first cache queue, and copy the original preview image stored in the first cache to a third cache; Process the original preview image stored in the third cache to obtain the first preview image of the target size.

7. An electronic device, characterized in that, It includes a camera, a display screen, a processor, and a memory; the camera, the display screen, and the memory are coupled to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, A computer program or instruction is stored in the computer-readable storage medium, and when the computer program or instruction is run, the method according to any one of claims 1-6 is implemented.

Images