Screen display method and apparatus

Abstract

Embodiments of the present application provide a screen display method and device, which are applied to the technical field of terminals. The method is applied to a foldable terminal device, and the method comprises: the terminal device is in a folded state, an outer display screen displays a first image, and an inner display screen does not display, the first image being an image collected by an outer-screen camera; a screen switching event is detected, the screen switching event being triggered in the case that the state of the terminal device changes; in response to the screen switching event, the inner display screen displays a second image, the second image not being an image collected by the outer-screen camera, and the second image not being an image collected by an inner-screen camera; the terminal is in an unfolded state, the inner display screen displays a third image, and the third image being an image collected by the inner-screen camera. The terminal device can display the image collected by the inner-screen camera on the inner screen when switching to the unfolded state according to the change of the folded state, reduce the use of the image collected by the outer-screen camera in the unfolded state, reduce the interruption of the preview interface and the like, and increase the user experience.

Description

[0001] This application is a divisional application. The original application has the application number 202210023838.7 and the original application date is January 10, 2022. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to screen display methods and apparatus. Background Technology

[0003] With the development of flexible screen technology, flexible foldable screens (also known as foldable screens) have been applied to mobile phones and other terminal devices, allowing users to fold or unfold their phones to meet their needs for different screen sizes.

[0004] In a possible implementation, a foldable phone could include an inner screen, an outer screen, and a back panel, with the inner screen being foldable. Cameras could be located on the inner screen, outer screen, and back panel; the cameras on the inner and outer screens could be called front-facing cameras, and the camera on the back panel could be called a rear-facing camera. Users could use either the front or rear camera for previewing, video calls, or live streaming.

[0005] However, when users use the front-facing camera for previewing, video calls, or live streaming, if the phone switches from unfolded to folded or vice versa, the preview, video call, or live streaming interface on the inner or outer screen may be interrupted, affecting the user experience. Summary of the Invention

[0006] This application provides a screen display method and apparatus, applied to a terminal device. The terminal device can switch between the inner screen front-facing camera and the outer screen front-facing camera according to changes in the foldable screen's state, so that the content displayed on the foldable screen changes accordingly with the foldable screen's state, improving the smoothness of video services and enhancing the user experience.

[0007] In a first aspect, this application proposes a screen display method applied to a terminal device. The terminal device includes an outer display screen and an inner display screen. The outer display screen is equipped with an outer screen camera, and the inner display screen is equipped with an inner screen camera. The terminal device is foldable, and the state of the terminal device includes a folded state and an unfolded state. The state of the terminal device is used to identify the folded state of the terminal device.

[0008] The method includes: the terminal device is in a folded state, the outer display screen displays a first image, and the inner display screen does not display anything, the first image being an image captured by the outer screen camera; a screen switching event is detected, the screen switching event being triggered when the state of the terminal device changes; in response to the screen switching event, the inner display screen displays a second image, the second image being neither an image captured by the outer screen camera nor an image captured by the inner screen camera; the terminal device is in an unfolded state, the inner display screen displays a third image, the third image being an image captured by the inner screen camera.

[0009] As is understandable, screen switching events are used to indicate a change in the foldable screen state from one state to another, such as from folded to unfolded, or from unfolded to folded. Screen switching events can be triggered by determining a change in the foldable screen state, receiving a message indicating a change in the foldable screen state, or receiving a signal indicating a change in the foldable screen state.

[0010] The folded state can be understood as the folded screen state mentioned below.

[0011] In this way, when the terminal device switches from a folded state to an unfolded state, the external screen camera can switch to the internal screen camera, displaying the image captured by the internal screen camera on the internal screen. This reduces the need for the terminal device to use images captured by the external screen camera in the unfolded state, minimizing interruptions to the corresponding interface during camera usage scenarios and improving user experience. The second image can be displayed during the gap between the internal screen lighting up and the image being displayed from the internal screen camera. This allows for immediate display after the internal screen lights up, reducing instances where the camera is not displaying an image, further minimizing interface interruptions during camera use and improving user experience.

[0012] Optionally, the second image is a blurred image.

[0013] Before displaying the image captured by the internal screen camera on the internal screen, a preset blurred image can be displayed. This allows the image to be displayed immediately after the internal screen is turned on, reducing the number of times the internal screen camera fails to send an image, minimizing interruptions to the preview and other interfaces, and improving the user experience.

[0014] Optionally, the second image is a preset image on the terminal device, or the second image is obtained by processing an image captured by an external screen camera or an internal screen camera, or the second image is an image from a photo album application, or the second image is obtained by processing an image from a photo album application.

[0015] In this way, the second image can be changed, and there are multiple ways to set the second image, which enhances the user experience.

[0016] Optionally, the method further includes: displaying a user interface on an internal or external display screen, the user interface including one or more images; detecting a first operation on a fourth image among the one or more images, and setting the fourth image as a second image.

[0017] This allows users to freely choose and set a second image, enhancing the user experience.

[0018] Optionally, the user interface also includes a confirmation control and a control for adding a blur effect. Detecting a first operation on a fourth image among one or more images and setting the fourth image as a second image includes: setting the fourth image as a second image through a second operation on the fourth image among multiple images and a third operation on the confirmation control; or, setting the blurred image of the fourth image as the second image through a second operation on the fourth image among multiple images, a fourth operation on the control for adding a blur effect, and a third operation on the confirmation control.

[0019] This allows you to add a blur effect to any image, creating a blurred image that can be used as a transition, thus enhancing the user experience.

[0020] Optionally, after detecting a screen switching event, the device may also: stop the external screen camera from outputting images and stop displaying images already output by the external screen camera; configure the internal screen camera to output images; and control the internal display screen to light up.

[0021] In this way, when the terminal device switches from a folded state to an unfolded state, the external screen camera can switch to the internal screen camera, reducing the number of times the terminal device uses the external screen camera to output images in the unfolded state, reducing the interruption of the corresponding interface in the camera usage scenario, and improving the user experience.

[0022] Optionally, when the terminal device is in the unfolded state, the inner display screen displays a third image, including: when the terminal device is in the unfolded state and the image output by the inner screen camera is sent for display, the terminal device displays the image sent by the inner screen camera on the lit inner display screen.

[0023] In this way, the terminal device can display the image captured by the internal screen camera after the internal screen is lit, reducing the interruption of the corresponding interface in the scene where the camera is used and improving the user experience.

[0024] Optionally, after the terminal device is in the unfolded state and the inner display screen displays the third image, the method further includes: detecting a screen switching event; responding to the screen switching event, displaying a second image on the outer display screen; and when the terminal device is in the folded state, displaying a fifth image on the outer display screen.

[0025] In this way, when the terminal device switches from the unfolded state to the folded state, the inner screen camera can switch to the outer screen camera, and the image captured by the outer screen camera can be displayed on the outer screen. This reduces the number of times the terminal device uses the image captured by the inner screen camera in the folded state, reduces the interruption of the corresponding interface in the camera usage scenario, and improves the user experience.

[0026] Optionally, when the terminal device is in a folded state, the external display screen displays the first image, including: when the terminal device is in a folded state, the terminal device receives an instruction from the target application to use the camera, and the external display screen displays the first image.

[0027] Optionally, the terminal device receiving an instruction from the target application to use the camera includes: the terminal device receiving an instruction from the target application to launch the preview function.

[0028] The method described in this application embodiment can be applied to a preview scenario.

[0029] Secondly, this application proposes a screen display method applied to a terminal device. The terminal device includes an outer display screen and an inner display screen. The outer display screen is equipped with an outer screen camera, and the inner display screen is equipped with an inner screen camera. The terminal device is foldable, and the state of the terminal device includes a folded state and an unfolded state. The state of the terminal device is used to identify the folded state of the terminal device.

[0030] The method includes: the terminal device is in a folded state, the outer display screen displays a first image, and the inner display screen does not display anything, the first image being an image captured by the outer screen camera; a screen switching event is detected, the screen switching event being triggered when the state of the terminal device changes; in response to the screen switching event, the inner display screen displays the image captured by the outer screen camera for a period of time; the terminal device is in an unfolded state, the inner display screen displays a third image, the third image being an image captured by the inner screen camera.

[0031] As is understandable, screen switching events are used to indicate a change in the foldable screen state from one state to another, such as from folded to unfolded, or from unfolded to folded. Screen switching events can be triggered by determining a change in the foldable screen state, receiving a message indicating a change in the foldable screen state, or receiving a signal indicating a change in the foldable screen state.

[0032] The folded state can be understood as the folded screen state mentioned below.

[0033] In this way, when the terminal device switches from a folded state to an unfolded state, the external screen camera can switch to the internal screen camera, and the image captured by the internal screen camera can be displayed on the internal screen. This reduces the number of times the terminal device uses the image captured by the external screen camera in the unfolded state, reduces the interruption of the corresponding interface in the camera usage scenario, and improves the user experience.

[0034] Optionally, after detecting a screen switching event, the process further includes: configuring the terminal device to output an image from the internal screen camera; maintaining the external screen camera outputting an image until a preset condition is met, at which point the terminal device stops outputting an image from the external screen camera; the terminal device controlling the internal display screen to light up; and the terminal device being in an unfolded state, with the internal display screen displaying a third image, including: when the terminal device is in an unfolded state and the image output from the internal screen camera is being displayed, the terminal device displays the image displayed by the internal screen camera on the lit internal display screen.

[0035] In this way, when the preset conditions are not met, the terminal device uses the external screen camera to output images, and when the preset conditions are met, the terminal device uses the internal screen camera to output images. The terminal device continuously displays the captured images, which prevents the external screen camera from taking too long to start up and causing a black screen when switching between certain situations (such as poor network, different mobile phone models, or continuous switching of folding screen states). This ensures that the camera usage scenarios such as recording, live streaming, and video are uninterrupted, improving the user experience.

[0036] Optionally, the terminal device continues to output images from the external screen camera until a preset condition is met, at which point the terminal device stops outputting images from the external screen camera. This includes: the terminal device continues to output and display images from the external screen camera until the image output from the internal screen camera is displayed, and stops outputting and displaying images already output from the external screen camera; wherein, during the interval between the internal screen being lit and the image output from the internal screen camera being displayed, the internal screen displays the image displayed by the external screen camera.

[0037] In this way, the image captured by the external screen camera is displayed during the interval between the internal screen lighting up and the image output from the internal screen camera. This prevents the external screen camera from taking too long to start up and causing a black screen when switching between different situations (such as poor network, different phone models, or continuous switching of foldable screen states). This ensures that the camera usage scenarios such as recording, live streaming, and video are uninterrupted, improving the user experience.

[0038] Optionally, the terminal device keeps the external screen camera displaying images until a preset condition is met, including: the terminal device keeps the external screen camera displaying images until a preset time period is reached.

[0039] In this way, the external screen camera turns off after the internal screen camera is turned on, and the terminal device continues to send and display images and record, so that the camera-using scenarios such as recording, live streaming, and video streaming are constantly flowing.

[0040] Optionally, the terminal device continues to output images from the external screen camera until a preset time period is reached, including: the terminal device continues to output and display images from the external screen camera until the image output from the internal screen camera is displayed, and the terminal device stops displaying the image output from the external screen camera, and stops outputting images from the external screen camera after the preset time period is reached; wherein, during the interval between the internal display screen lighting up and the image output from the internal screen camera being displayed, the internal screen display screen displays the image displayed by the external screen camera.

[0041] Optionally, the terminal device stops outputting images from the external screen camera when a preset time period is reached, including: the terminal device stops outputting images from the external screen camera when a counter reaches a threshold; wherein the counter is started when the internal screen camera begins outputting images.

[0042] In this way, using a counter to confirm the preset duration is a simple method.

[0043] Optionally, the method further includes: when the terminal device is holding the external screen camera outputting an image, if the terminal device detects a screen switching event, the terminal device controls the image outputting the external screen camera to be displayed, and controls the external display screen to be lit up, and displays the image outputting the external screen camera when the external display screen is lit up.

[0044] In this way, the terminal can also display the images captured by the external camera on the external display screen, making it convenient for the user being photographed to confirm the shooting status, so that the user being photographed can adjust the position, angle, etc., thereby improving the user experience.

[0045] Optionally, if the terminal device detects a screen switching event while the external screen camera is displaying an image, the method further includes: the terminal device controlling the external screen camera to resume displaying an image for a preset duration.

[0046] In this way, the terminal device can extend the time for both the inner and outer screen cameras to output images. When the folding screen state of the terminal device changes again within a preset time, both the inner and outer screen cameras will capture images, enabling rapid switching between cameras and improving the user experience.

[0047] Optionally, the terminal device controls the external screen camera to resume displaying images for a preset duration, including: the terminal device controls the counter to start counting again.

[0048] By using a counter to re-count, the time for both the inner and outer screen cameras to capture images is extended. During continuous changes in the foldable screen state (e.g., the terminal device switches from a folded state to an unfolded state and then back to a folded state), the terminal device can switch to the outer screen camera for image output. The outer screen camera does not need to be turned off and restarted, shortening the camera's response time and improving the user experience.

[0049] Optionally, when the terminal device is in a folded state, the external display screen displays the first image, including: when the terminal device is in a folded state, the terminal device receives an instruction from the target application to use the camera, and the external display screen displays the first image.

[0050] Optionally, the terminal device receiving an instruction from the target application to use the camera includes: the terminal device receiving an instruction from the target application to start a video call function, a recording function, or a live streaming function.

[0051] This approach is suitable for scenarios such as video calls, recording, or live streaming, reducing interruptions to the corresponding interface and enhancing the user experience.

[0052] Optionally, after the terminal device is in the unfolded state and the inner display screen displays the third image, the method further includes: detecting a screen switching event; in response to the screen switching event, displaying the image captured by the inner screen camera on the outer display screen for a period of time; and when the terminal device is in the folded state, displaying the fourth image on the outer display screen.

[0053] In this way, when the terminal device switches from the unfolded state to the folded state, the inner screen camera can be switched to the outer screen camera, reducing the number of times the terminal device uses the inner screen camera to output images in the folded state, reducing the interruption of the corresponding interface in scenarios such as previewing and video calls, and improving the user experience.

[0054] Thirdly, embodiments of this application provide a terminal device, which can also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0055] The terminal device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the terminal device to perform the method of the first aspect or the method of the second aspect.

[0056] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method of the first aspect or performs the method of the second aspect.

[0057] Fifthly, embodiments of this application provide a computer program product, which includes a computer program that, when run, causes a computer to perform the method as described in the first aspect or the method as described in the second aspect.

[0058] It should be understood that the third to fifth aspects of this application correspond to the technical solutions of the first or second aspects of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0060] Figure 2 This is a schematic diagram of one possible use case for a front-facing camera.

[0061] Figure 3 This application provides a schematic diagram of the software architecture of a terminal device.

[0062] Figure 4 A schematic diagram of the module interaction process involved in a screen display method provided in an embodiment of this application;

[0063] Figure 5A This is a schematic diagram of the camera switching timing of a terminal device in one implementation method;

[0064] Figure 5B This application provides a schematic diagram of camera switching timing for a terminal device.

[0065] Figure 5C This is a schematic diagram of camera switching timing for another terminal device provided in an embodiment of this application;

[0066] Figure 6A This application provides a schematic diagram of camera switching timing for a terminal device.

[0067] Figure 6B This application provides a schematic diagram of the interface of a terminal device according to an embodiment of the present application.

[0068] Figure 7A A schematic diagram of the module interaction process involved in a screen display method provided in an embodiment of this application;

[0069] Figure 7B A schematic diagram of the module interaction process involved in a screen display method provided in an embodiment of this application;

[0070] Figure 8 This application provides a schematic diagram of camera switching timing for a terminal device.

[0071] Figure 9 This application provides a schematic diagram of camera switching timing for a terminal device.

[0072] Figure 10 A camera switching timing diagram for a foldable screen switching from a folded state to an unfolded state is provided in an embodiment of this application;

[0073] Figure 11 A timing diagram for camera switching during continuous folding is provided in an embodiment of this application;

[0074] Figure 12 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0075] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "exemplary" or "for example" are used in the embodiments of this application to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0076] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0077] The embodiments of this application can be applied to terminal devices with foldable screens. Terminal devices can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0078] For example, Figure 1 These are front and rear view diagrams of a terminal device with a foldable screen provided in an embodiment of this application.

[0079] like Figure 1 As shown, the terminal device can include a folded state and an unfolded state. The folded state can be understood as the angle between the two sides of the folded screen being less than or equal to a first value, and the unfolded state can be understood as the angle between the two sides of the folded screen being greater than or equal to a second value.

[0080] In one possible implementation, the first value is equal to the second value. This implementation can be understood as the terminal device being either in a folded state or an unfolded state.

[0081] In another possible implementation, the second value is greater than the first value. In this implementation, the terminal device may also include a transition state, which can be understood as a state in which the angle between the two sides of the foldable screen is greater than the first value and less than the second value.

[0082] like Figure 1 As shown, when the terminal device is in a folded state, the display screen facing the user in the front view can be called the outer screen, and the display screen facing the user in the rear view can be called the back panel. When the terminal device is in an unfolded state, the display screen facing the user in the front view can be called the inner screen (or foldable screen), and the outer screen and back panel can be unfolded onto a single plane in the rear view.

[0083] In other words, when the terminal device is in a folded state, the inner screen can be folded and hidden, while the outer screen can be used to display the interface. When the terminal device is in an unfolded state, the inner screen can be unfolded and used to display the interface.

[0084] Furthermore, such as Figure 1 As shown, cameras can be installed in the inner screen, outer screen, and back panel of the terminal device.

[0085] In a possible implementation, the camera of the terminal device can be classified into a front-facing camera and a rear-facing camera. The front-facing camera may include an external screen camera set on the outer screen and an internal screen camera set on the inner screen.

[0086] It is understood that the terminal device may include one or more external screen cameras and / or one or more internal screen cameras. The external screen cameras can be positioned anywhere on the external screen; the internal screen cameras can be positioned anywhere on the internal screen, for example, on the displays on either side of the internal screen when it is unfolded, or on any one side of the internal screen when it is unfolded. This application embodiment does not limit the number of internal screen cameras, their positions, the number of external screen cameras, or their positions.

[0087] For example, such as Figure 1 In the terminal device shown, when the terminal device is in a folded state, the external screen 101 and the external screen camera 102 on the external screen can be seen in the front view; the rear camera 103 can be seen in the corresponding rear view.

[0088] When the terminal device is in the unfolded state, the inner screen is open, and the front view shows the inner screen 104 and the inner screen camera 105 on the inner screen; the corresponding rear view shows the rear camera 107 and the outer screen camera 106 on the outer screen. The inner screen 104 can be divided into left and right parts.

[0089] In one implementation, when the terminal device uses the front-facing camera for previewing, video calls, or live streaming, the camera is activated when the terminal device is in a folded state, and the external screen camera is turned on and displayed on the external screen; when the terminal device is in an unfolded state, the camera is activated, and the internal screen camera is turned on and displayed on the internal screen.

[0090] If the terminal device switches from unfolded to folded, the inner screen camera will not automatically switch to the outer screen camera, and if the terminal device switches from folded to unfolded, the outer screen camera will not automatically switch to the inner screen camera, causing interruptions to the terminal device's preview interface, video call interface, or live broadcast interface.

[0091] For example, Figure 2 This is a schematic diagram illustrating one possible use case for a front-facing camera. For example... Figure 2 As shown,

[0092] When a video call is initiated by launching the application in the unfolded state of the terminal device, the terminal device displays the real-time captured image from the internal screen camera on the internal screen (e.g., ...). Figure 2 (The interface shown in a) When the terminal device receives a folding operation, the terminal device... Figure 2 Enter the interface shown as 'a' in the diagram. Figure 2 As shown in b, the terminal device displays the real-time footage captured by the internal camera on the internal screen.

[0093] When the terminal device receives a folding operation and continues to fold the screen, the terminal device enters... Figure 2 The interface shown in 'c' is as follows. At this time, the outer screen of the phone will block the inner screen camera. The outer screen will display the live shot of the inner screen camera, while the outer screen will be black, and the video call will be interrupted.

[0094] Similarly, if the device switches from a folded state to an unfolded state while using the front-facing camera for previewing, video calls, live streaming, or recording, the external screen camera continues to be used, causing the device's preview interface, video call interface, live streaming interface, or recording interface to display the image transmitted by the external screen camera, and these interfaces will also be interrupted.

[0095] For example, when a terminal device switches from a folded to an unfolded state, the external screen camera moves from the front to the back of the device. This change in the external screen camera's position means the image it captures also changes position, making it impossible to take photos from the front of the device. This results in a poor user experience.

[0096] Based on this, embodiments of this application provide a screen display method and related apparatus that switch between the inner screen camera and the outer screen camera according to the folding screen state, so that the front camera can change with the folding screen state, reducing problems such as black screen caused by the outer screen blocking the inner screen camera and image changes caused by the change of the position of the outer screen camera, thereby increasing the user experience.

[0097] For ease of understanding, the software system of the terminal device will be described below. The software system of the terminal device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc., which will not be elaborated here.

[0098] For example, Figure 3This is a schematic diagram of the software architecture of a terminal device provided in an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the kernel layer, and the hardware layer.

[0099] The application layer can include a series of application packages. For example... Figure 3 As shown, the application package may include applications such as the terminal camera app, third-party camera apps, and calendar apps. The third-party camera apps may include: Applications such as...

[0100] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer can include some predefined functions. For example... Figure 3 As shown, the application framework layer may include one or more of the following: application management module, display management module, foldable screen management module, sensor service, camera framework and camera service, etc.

[0101] The application management module is used to receive the display content from the application in order to prepare the display content for the inner screen and outer screen of the foldable screen.

[0102] The foldable screen management module can manage the foldable screen's state. For example, it can calculate the foldable screen state and notify other modules (such as the terminal camera application, camera frame, and display management module) of the foldable screen state, so that other modules can process the screen accordingly (e.g., camera switching and display screen switching). It is also understood that the foldable screen management module can respond to commands sent by other modules to obtain the foldable screen state and provide feedback on the foldable screen state.

[0103] The camera framework enables applications to manage and access camera devices, such as managing camera image capture. The camera framework can include a camera state management module.

[0104] The camera state management module can establish a connection with the foldable screen management module, receive changes in the foldable screen state, establish a connection with the camera hardware abstraction layer, and notify the camera abstraction layer of changes in the foldable screen state.

[0105] It should be noted that the display management module, sensor service, and camera service can also be set independently of the application framework layer, between the application framework layer and the HAL. This application does not impose specific limitations on these embodiments.

[0106] The purpose of the HAL layer is to abstract hardware, providing a unified interface for upper-layer applications to query hardware devices, or to provide data storage services for upper-layer applications. For example... Figure 3 As shown, the HAL layer may include one or more of the following: display hardware abstraction, sensor hardware abstraction, camera hardware abstraction, etc.

[0107] The display hardware abstraction can control both the outer and inner screens.

[0108] The camera hardware abstraction includes a front-facing camera module. This module determines which camera (output camera) needs to be activated. The camera hardware abstraction stores the folding screen state; for example, when the folding screen is folded, the front-facing camera module determines to activate the outer screen camera; when the folding screen is unfolded, the front-facing camera module determines to activate the inner screen camera.

[0109] Multiple modules in the HAL layer can conform to the Hardware Abstraction Layer Interface Description Language (HIDL).

[0110] The kernel layer is the layer between hardware and software. For example... Figure 3 As shown, the kernel layer may include one or more of the following: display driver, sensor driver, camera driver, etc.

[0111] like Figure 3 As shown, this hardware layer may include one or more of the following: an outer screen, an inner screen, sensors, an inner screen camera, an outer screen camera, etc. The sensors may include one or more of the following: accelerometers, Hall effect sensors, gyroscopes, or angle sensors, etc. Accelerometers are used to monitor the acceleration of the folding action. Hall effect sensors are used to detect the status of components within the foldable screen.

[0112] The following is combined with Figure 3 This document explains the display process of the terminal device and the process of calling the front-facing camera during the folding process of the foldable screen.

[0113] When the terminal device is powered on, its sensors detect the folding screen status, etc. The sensors output corresponding signals, which are then reported to the folding screen management module via sensor drivers and sensor hardware abstraction. The folding screen management module calculates and confirms the folding screen status information based on the signals reported by the sensors (such as the acceleration of the folding action). The folding screen status information can include either a folded state or an unfolded state.

[0114] Taking the foldable screen status information as folded as an example, the foldable screen management module sends the foldable screen status information to the display management module. The display management module then sends the foldable screen status information to the display hardware abstraction. Upon receiving the foldable screen status information, the display hardware abstraction confirms that the foldable screen has switched from the inner screen to the outer screen. The display hardware abstraction sends instructions to the display driver to close the inner screen and open the outer screen. The display driver then drives the hardware layer to close the inner screen and open the outer screen. The terminal device begins displaying images on the outer screen.

[0115] It is understood that the display hardware abstraction can simultaneously issue instructions to turn off the inner screen and instructions to turn on the outer screen; alternatively, it can issue the instruction to turn off the inner screen first, followed by the instruction to turn on the outer screen. This application embodiment does not limit the execution order of the two instructions.

[0116] The foldable screen management module can also send foldable screen status information to the camera status management module of the camera frame. Upon receiving the foldable screen status information, the camera status management module sends it to the camera hardware abstraction. The camera hardware abstraction, upon receiving the foldable screen status information, saves it. Optionally, the foldable screen management module sends the foldable screen status information when the terminal device detects a screen switching event. The screen switching event is triggered when the terminal device's state changes, or it indicates a change in the foldable screen state from one state to another, such as from a folded state to an unfolded state, or vice versa. The screen switching event can be a determination of a foldable screen state change, receipt of a foldable screen state change message, or receipt of a signal indicating a foldable screen state change.

[0117] The front-facing camera module, based on the folded state, calls the external screen camera via the camera driver. The terminal device then takes pictures through the external screen camera.

[0118] In this embodiment, the camera state management module can be a separate module or reused on other modules of the camera frame.

[0119] In one possible implementation, the foldable screen management module can also send the foldable screen status information to the terminal camera application, so that the terminal camera application can determine whether to add effects such as blur animation based on the changes in the foldable screen status.

[0120] For example, when the terminal camera application receives the foldable screen status information sent by the foldable screen status management module, it can retrieve the image information saved in the camera frame and add a blur effect to create a blurred animation. The terminal camera application then sends the blurred animation to the application management module. Upon receiving the blurred animation, the application management module, through the display management module, display hardware abstraction, and display driver, sends it to the foldable screen display.

[0121] It is understood that the terminal camera application may not implement the process of adding blur animation, and the embodiments of this application do not impose specific limitations.

[0122] In another possible implementation, the foldable screen management module may not send the foldable screen status information to the third-party camera application. In this way, the third-party camera application can be unaware of the switching between the inner screen camera and the outer screen camera on the terminal device.

[0123] For example, the process of calling the front-facing camera in a terminal device is as follows: The application at the application layer of the terminal device receives a trigger operation to open the front-facing camera. A third-party camera application (e.g., WeChat, live streaming application, etc.) or the terminal camera application calls the front-facing camera module in the camera hardware abstraction through the camera framework and camera service. The front-facing camera module determines which camera needs to be opened based on the folded state stored in the camera hardware abstraction.

[0124] Taking a folded terminal device as an example, the front-facing camera module confirms the opening of the external screen camera. The front-facing camera module sends a command to the camera driver to instruct the external screen camera to open. Upon receiving this command, the camera driver opens the external screen camera. After opening, the external screen camera begins displaying images. The camera hardware abstraction receives the displayed images from the external screen camera via the camera driver. The camera hardware abstraction then transmits preview images, videos, etc., to the corresponding applications via camera services and camera frames. Upon receiving the preview images, videos, etc., the applications call the corresponding modules to perform operations such as display and encoding.

[0125] The method provided in this application embodiment will be described below with reference to specific application scenarios. There are two types of application scenarios: one is a preview scenario, and the other is a video call scenario, a live broadcast scenario, or a recording scenario.

[0126] The following section explains the method flow in the preview scene. For example, Figure 4 This is a schematic diagram illustrating the module interaction process of a screen display method provided in an embodiment of this application. For example... Figure 4 As shown,

[0127] S401. When the terminal device receives an operation indicating power-on while in folded state, the terminal device calls the camera hardware abstraction and foldable screen management module to power it on.

[0128] In this embodiment of the application, powering on refers to starting the power on from the power-off state.

[0129] S402. After the foldable screen management module is turned on, it calculates the foldable screen status.

[0130] It is understandable that the foldable screen management module calculates the foldable screen state based on information transmitted from one or more sensors. These sensors include, but are not limited to, accelerometers, Hall effect sensors, gyroscopes, or angle sensors.

[0131] In a possible implementation, the foldable screen management module can determine the foldable screen state based on the folding angle. For example, when the folding angle is between 0 and 75 degrees, the foldable screen is in a folded state; when the folding angle is between 75 and 180 degrees, the foldable screen is in an unfolded state. It is understood that the above-mentioned folding angle and the method of determining the foldable screen state based on the folding angle are only examples. The foldable screen state can also be determined based on other information (e.g., acceleration). This application embodiment does not limit the method of determining the foldable screen state.

[0132] S403, the foldable screen management module sends the foldable screen status information to the camera hardware abstraction. Adaptively, the camera hardware abstraction updates the foldable screen status after receiving the foldable screen status information.

[0133] In one possible implementation, the foldable screen management module can send foldable screen status information to the camera hardware abstraction based on pre-set events (e.g., power-on events, camera frame request events, etc.).

[0134] The foldable screen management module can also send foldable screen status information to the camera hardware abstraction based on changes in the foldable screen's state. For example, when the foldable screen state changes, the foldable screen status information is sent to the camera hardware abstraction.

[0135] In some embodiments, the foldable screen management module also sends foldable screen status information to the terminal camera application.

[0136] S404. When the terminal camera application receives a command to instruct the camera to be turned on, the terminal camera application sends the command to the camera hardware abstraction through the camera frame.

[0137] It is understandable that when the terminal device receives an operation on the main interface instructing it to open the terminal camera application, the terminal camera application receives an instruction instructing it to open the camera. Alternatively, when the terminal device receives an operation on the terminal camera application interface instructing it to open the preview function, the terminal camera application receives an instruction instructing it to open the camera.

[0138] S405, when the camera hardware abstraction receives a command to turn on the camera, it powers on both the inner and outer screen cameras via the camera driver.

[0139] S406, When the camera hardware abstraction receives a command to open the camera, it determines the output camera based on the folding screen state.

[0140] It is understandable that when the foldable screen is in a folded state, the camera hardware abstraction determines that the camera outputting the image is the outer screen camera; when the foldable screen is in an unfolded state, the camera hardware abstraction determines that the camera outputting the image is the inner screen camera.

[0141] In S403 above, the foldable screen is in a folded state, and the output camera confirmed in S406 is the external screen camera. The terminal device executes S407 and S408.

[0142] S407, Camera Hardware Abstraction drives the external screen camera to output images via the display driver.

[0143] S408: The external camera sends image information to the terminal camera application via camera driver, camera hardware abstraction, and camera frame.

[0144] Adaptively, the terminal camera application receives image information. After receiving the image information, the terminal camera application performs drawing and rendering operations to display it on the foldable screen.

[0145] S409. When the terminal device receives an operation to instruct the unfolding of the foldable screen, the foldable screen management module receives information to instruct the unfolding of the foldable screen.

[0146] S410: After receiving information indicating that the foldable screen should be unfolded, the foldable screen management module calculates the foldable screen status.

[0147] It is understandable that the foldable screen in the S410 is in the unfolded state.

[0148] S411, the foldable screen management module sends the foldable screen status information calculated by S410 to the camera hardware abstraction and the terminal camera application.

[0149] In one possible implementation, when the foldable screen state changes, the foldable screen management module sends the foldable screen state information calculated by S410. When the foldable screen state does not change, the foldable screen management module does not send the foldable screen state information calculated by S410.

[0150] S412 After receiving the foldable screen status information, the terminal camera application sends a command to the camera hardware abstraction via the camera frame to instruct the reconfiguration of the front camera.

[0151] S413, After receiving an instruction to reconfigure the front camera, the camera hardware abstraction identifies the output camera based on the folding screen state.

[0152] It should be noted that step S412 is optional. The terminal device may or may not execute step S412. The camera hardware abstraction can determine the output camera based on the folding screen state after receiving the folding screen state information.

[0153] For example, taking the above S411 where the folding screen is in the unfolded state and the output camera confirmed in S413 is the inner screen camera, the terminal device executes S414-S416.

[0154] S414, Camera hardware abstraction stops the external screen camera from displaying images via the camera driver.

[0155] In this embodiment of the application, "outputting an image" can be understood as the inner screen camera or the outer screen camera outputting the captured image, or in other words, the camera's sensor capturing an image and outputting the captured image.

[0156] The idea that the camera hardware abstraction stops the external screen camera via the camera driver can be understood as: the camera hardware abstraction stops the external screen camera from acquiring images and stops transmitting image information to the camera hardware abstraction layer.

[0157] S415, Camera hardware abstraction configures the internal screen camera to output images via camera driver configuration.

[0158] S416: The internal screen camera sends image information to the terminal camera application via camera driver, camera hardware abstraction, and camera frame.

[0159] S417. When the terminal camera application receives a command to indicate that the camera should be turned off, the terminal camera application sends the command to the camera hardware abstraction through the camera frame.

[0160] S418, when the camera hardware abstraction receives a command to turn on the camera, it stops the output camera from outputting images through the camera driver and powers down the inner screen camera and the outer screen camera.

[0161] It is understandable that the above explanation uses the example of the image output camera confirmed in S413 being the inner screen camera, and the camera driver stopping the inner screen camera's image output in S418 as an example. In the implementation process of the terminal device switching from an unfolded state to a folded state, it can be... Figure 4 The inner screen camera and the outer screen camera are interchangeable, and the specific implementation process is similar, so it will not be described in detail here.

[0162] In the process shown in this application embodiment, when the terminal device switches from the folded state to the unfolded state, the external screen camera of the terminal device can automatically switch to the internal screen camera, reducing the situation where the terminal device uses the external screen camera to output images after switching from the folded state to the unfolded state, thereby reducing the inability of the terminal device to preview the scene directly in front of it.

[0163] Figure 5A This is a schematic diagram illustrating a camera switching timing method for a terminal device. The following is combined with... Figures 5B-5C The camera switching timing and display interface of the terminal device are described to illustrate the method of the embodiments of this application.

[0164] Figure 5AIn the text, lines 1-5 are used to represent the external screen display status, the internal screen display status, the working status of the external screen camera, the working status of the internal screen camera, and the display interface of the terminal device, respectively.

[0165] Take, for example, a terminal device starts its camera to preview at time A, and then switches the folded screen from folded to unfolded at time B.

[0166] At time A, the terminal device is in a folded state; the outer screen lights up to display the image, while the inner screen is off. When the terminal device activates its camera for preview at time A, the outer screen camera starts displaying the image, and the inner screen camera turns off. At time A, the terminal device displays on the outer screen... Figure 5A The interface shown in 'a' includes a view of the user on the steps.

[0167] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images, while the inner screen camera remains off. The image captured by the outer screen camera changes.

[0168] Understandably, the inner screen requires a certain amount of time to light up. At time C, the inner screen lights up and begins displaying the image. Meanwhile, the external camera on the terminal device continues to output images, which are then displayed on the external screen. Figure 5A The interface shown in b is a screen displaying a television viewing experience.

[0169] At time D, the terminal device is in the unfolded state, and the terminal device displays on the inner screen. Figure 5A The interface shown in 'c' includes a picture of watching television.

[0170] from Figure 5A As can be seen, the terminal device uses the external screen camera to output images after the folded screen state changes. Compared to time A, at time C, the folded screen state changes, the image captured by the terminal device's external screen camera changes, and the image displayed on the terminal device changes. Users cannot preview the image of themselves on the steps before the folded screen state change, resulting in a poor user experience.

[0171] For example, Figure 5B This is a schematic diagram of camera switching timing for a terminal device provided in an embodiment of this application. Figure 5B In the text, lines 1-5 are used to represent the external screen display status, the internal screen display status, the working status of the external screen camera, the working status of the internal screen camera, and the display interface of the terminal device, respectively.

[0172] Take, for example, a terminal device starts its camera to preview at time A, and then switches the folded screen from folded to unfolded at time B.

[0173] At time A, the terminal device is in a folded state; the outer screen lights up to display the image, while the inner screen is off. When the terminal device activates its camera for preview at time A, the outer screen camera starts displaying the image, and the inner screen camera turns off. At time A, the terminal device displays on the outer screen... Figure 5B The interface shown in 'a' includes a view of the user on the steps.

[0174] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images, while the inner screen camera remains off.

[0175] At time C, the terminal device responds to the change in the foldable screen's state and begins to activate the inner screen camera. It can be understood that activation can be considered an action performed at a single point in time; from activation to the inner screen camera starting to function, a startup time is typically required. For example, at time D, the inner screen camera has finished starting and begins to output images.

[0176] Understandably, the internal screen requires a certain amount of time to light up. At time E, the internal screen lights up and begins displaying the image. At this point, the internal screen camera outputs an image, which is then displayed on the terminal device as shown below. Figure 5B The interface shown in b includes a view of the user on the steps.

[0177] At time F, the terminal device is in the unfolded state, and the terminal device displays on the inner screen. Figure 5B The interface shown in b is a view of the user on the steps.

[0178] In other words, after the camera is switched in this embodiment, the terminal device uses the inner screen camera to output the image after the folded screen state changes. The images captured at time C and after time C are consistent with the images before the folded screen state changes, which meets user expectations and enhances the user experience.

[0179] It is understandable that if the internal screen camera takes a long time to start up after being turned on, there may be situations where the internal screen is lit up but the internal screen camera fails to display an image.

[0180] For example, Figure 5C This is a schematic diagram illustrating the camera switching timing of another terminal device provided in an embodiment of this application. For example... Figure 5C As shown, rows 1-5 are used to represent the external screen display status, the internal screen display status, the working status of the external screen camera, the working status of the internal screen camera, and the display interface of the terminal device, respectively.

[0181] Take, for example, a terminal device starts its camera to preview at time A, and then switches the folded screen from folded to unfolded at time B.

[0182] At time A, the terminal device is in a folded state, the outer screen lights up to display the image, and the inner screen is off. When the terminal device starts the camera for preview at time A, the outer screen camera begins to output the image, and the inner screen camera turns off.

[0183] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images, while the inner screen camera remains off.

[0184] At time C, the terminal device responds to the change in the foldable screen's state and begins to activate the inner screen camera. Understandably, the inner screen camera requires a startup time to activate, and the inner screen requires a lighting time to illuminate.

[0185] The internal screen camera may finish starting up after the internal screen lights up. For example, at time D, the internal screen lights up, the internal screen camera is starting up but not displaying an image, and the external screen camera is off and not displaying any images. At time E, the internal screen camera finishes starting up, begins displaying images, and the internal screen displays the image.

[0186] from Figure 5C As can be seen, during the interval between the inner screen lighting up and the image transmission from the inner screen camera, i.e., at time DE, the terminal device may not transmit any image. Therefore, in this embodiment, preset content can be transmitted to the inner screen for display using the terminal camera application during the interval between the inner screen lighting up and the image transmission from the inner screen camera, thus enabling display as soon as the inner screen lights up.

[0187] In this embodiment, the preset content can be any image or preset interface. The preset interface can be an interface including transition animations or an interface with transition images. The transition animation can be a blurred animation or a sharp animation; the transition image can be a blurred image or a sharp image.

[0188] Blurry animation can be understood as a transitional animation in a blurred state, and blurry image can be understood as a transitional image in a blurred state. A blurred state can be understood as a non-sharp state.

[0189] Understandably, transition animations and images can be pre-saved on the device or user-defined and saved on the device. Users can select from multiple options through the relevant user interface, or customize selections from the photo album, or capture animations or images using the camera.

[0190] Alternatively, it can be understood that the preset interface can be an interface that includes images preset on the terminal device, an interface that includes images obtained by processing images captured by the external or internal camera, or an interface that includes images from a photo album application, or an interface that includes images obtained by processing images from a photo album application.

[0191] Images pre-installed on a terminal device can be understood as images pre-loaded into the terminal device before it leaves the factory; images obtained by processing images captured by the external or internal screen camera can be understood as images taken by the user; images in a photo album application can be understood as images downloaded by the user from the internet, screenshots, or images saved in the album after being taken; images obtained by processing images from a photo album application can be understood as images obtained by processing images from the album, including blurring, enhancement, and cropping. Images include, but are not limited to, transition animations and transition images.

[0192] For example, based on the above S401-S418, the terminal device may also execute S419.

[0193] S419. After receiving the foldable screen status information, the terminal camera application retrieves the image information stored in the camera frame from the camera frame and adds a blur effect to obtain a blurred animation.

[0194] For example, the image information stored in the camera frame can be the image information most recently transmitted to the camera frame by the camera hardware abstraction.

[0195] It should be noted that after receiving the folding screen status information in S411, the terminal camera application can immediately execute S419; or, the terminal camera application can execute S419 after a period of time after receiving the folding screen status information in S411; or, the terminal camera application can execute S419 after executing S412.

[0196] It is understood that S419 is an optional step; the terminal camera application can choose to execute S419 or not. When the terminal camera application does not execute S419, the camera switching sequence and display interface can be referenced as described above. Figure 5B Related instructions or Figure 5C The relevant instructions are as follows. When the terminal camera application executes S419, the camera switching sequence and display interface can be referred to below. Figure 6A Related explanations.

[0197] The following is combined with Figure 6A and Figure 6B The method of adding blur animation in S419 above will be explained.

[0198] For example, Figure 6A This is a schematic diagram illustrating the camera switching timing of another terminal device provided in an embodiment of this application. For example... Figure 6A As shown, rows 1-5 are used to represent the external screen display status, the internal screen display status, the working status of the external screen camera, the working status of the internal screen camera, and the display interface of the terminal device, respectively.

[0199] Take, for example, a terminal device starts its camera to preview at time A, and then switches the folded screen from folded to unfolded at time B.

[0200] At time A, the terminal device is in a folded state; the outer screen lights up to display the image, while the inner screen is off. When the terminal device activates its camera for preview at time A, the outer screen camera starts displaying the image, and the inner screen camera turns off. The terminal device displays on the outer screen... Figure 6A The interface shown in 'a' includes a view of the user on the steps.

[0201] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images, while the inner screen camera remains off.

[0202] At time C, the terminal device responds to the change in the foldable screen's state and begins to activate the inner screen camera. Understandably, the inner screen camera requires a startup time to activate, and the inner screen requires a lighting time to illuminate.

[0203] The internal screen camera may complete startup after the internal screen lights up. At time D, the internal screen lights up, the internal screen camera is starting up but not displaying an image, the external screen camera is off, and the terminal device displays on the internal screen. Figure 6A The interface shown in b in the image includes a blur animation.

[0204] At time E, the internal screen camera completes startup and begins image output, which is displayed on the internal screen of the terminal device. Figure 6A The interface shown in 'c' includes a view of the user on the steps.

[0205] from Figure 6A As can be seen, during the interval between the inner screen lighting up and the image being displayed from the inner screen camera, i.e., at time DE, the terminal device displays a blurry animation. This reduces the likelihood of the inner screen not displaying an image when it lights up because the inner screen camera hasn't yet produced an image, thus improving the user experience.

[0206] It is understandable that the above Figure 6A The blur animation in the interface shown in b can be replaced with a blur image, or with other transition animations, or with other transition images.

[0207] Users can select transition images from preset content through the terminal device's interface. Transition images include transition animations or transition pictures. For example, Figure 6B This is a schematic diagram of the interface of a terminal device provided in an embodiment of this application. Figure 6B As shown, the interface includes: animation 601, image 602, more options 603, blur effect control 604, and confirmation control 605.

[0208] Animation 601 includes one or more animations, and image 602 includes one or more images. More options 603 include options to select from the album and options to take a photo.

[0209] When the terminal device is Figure 6B When the terminal device receives an operation to trigger one of the animations in animation 601 in the interface, and then receives an operation to trigger the confirmation control 605, the terminal device sets the animation as a transition animation.

[0210] When the terminal device is Figure 6B When the terminal device receives an operation to trigger one of the transition images in image 602 in the interface, and then receives an operation to trigger the confirmation control 605, the terminal device sets that image as the transition image. Figure 6B When the terminal device receives an operation to trigger the "Select from Album" option in the "More Options" 603 interface, it retrieves images, videos, etc., stored in the album for the user to select any image or animation stored in the album. After the terminal device triggers the "Select from Album" operation in the "More Options" 603, upon receiving an operation to trigger the confirmation control 605, the terminal device sets the image as a transition image or the animation as a transition animation.

[0211] When the terminal device is Figure 6B When the terminal device receives an operation to trigger the "shoot" option in the "more options" 603 interface, it calls the camera to capture an image or an animation. When the terminal device subsequently receives an operation to trigger the confirmation control 605, it sets the captured image as a transition image or the captured animation as a transition animation.

[0212] When the terminal device is Figure 6B When the interface receives an operation to trigger the blur effect control 604, a blur effect is added to the transition animation or transition image, making the transition animation or transition image blurry, thus obtaining a blurred animation or blurred image.

[0213] It is understood that the above embodiments are illustrated using a preview scenario of a terminal camera application as an example. In specific implementations, the terminal camera can be not only a terminal camera application but also a third-party camera application, and the method of this application embodiment can also be applied to scenarios such as video calls, live streaming, or recording. For example, Figures 7A-10 The corresponding implementation can be applied to scenarios such as video preview, video call, live streaming, or recording.

[0214] For example, Figure 7A This is a schematic diagram illustrating the module interaction process of a screen display method provided in an embodiment of this application. Figure 7AThe camera application in the app can include third-party camera applications or the app's own camera application; both types of applications can be used interchangeably. Figure 7A To illustrate the interaction flow, the following example uses a third-party camera application. Figure 7A Corresponding implementation examples. For example... Figure 7A As shown, the method includes:

[0215] S701. When the terminal device receives an operation indicating power-on while in folded state, the terminal device calls the camera hardware abstraction and foldable screen management module to power it on.

[0216] S702, after the foldable screen management module is turned on, calculates the foldable screen status.

[0217] It is understandable that the foldable screen management module calculates the foldable screen state based on information transmitted from one or more sensors. These sensors include, but are not limited to, accelerometers, Hall effect sensors, gyroscopes, or angle sensors.

[0218] In a possible implementation, the foldable screen management module can determine the foldable screen state based on the folding angle. For example, when the folding angle is between 0 and 75 degrees, the foldable screen is in a folded state; when the folding angle is between 75 and 180 degrees, the foldable screen is in an unfolded state. It is understood that the above-mentioned folding angle and the method of determining the foldable screen state based on the folding angle are only examples. The foldable screen state can also be determined based on other information (e.g., acceleration). This application embodiment does not limit the method of determining the foldable screen state.

[0219] S703, the foldable screen management module sends the foldable screen status information to the camera hardware abstraction. Adaptively, the camera hardware abstraction updates the foldable screen status after receiving the foldable screen status information.

[0220] In one possible implementation, the foldable screen management module can send foldable screen status information to the camera hardware abstraction based on pre-set events (e.g., power-on events, camera frame request events, etc.).

[0221] The foldable screen management module can also send foldable screen status information to the camera hardware abstraction based on changes in the foldable screen's state. For example, when the foldable screen state changes, the foldable screen status information is sent to the camera hardware abstraction.

[0222] S704. When a third-party camera application receives a command to instruct the camera to be turned on, the third-party camera application sends the command to the camera hardware abstraction via the camera frame.

[0223] It is understandable that when the terminal device receives an operation in the third-party camera application interface that instructs the user to start the video call function, recording function or live streaming function, the third-party camera application receives an instruction to open the camera.

[0224] When the S705 camera hardware abstraction receives a command to turn on the camera, it powers on both the inner and outer screen cameras via the camera driver.

[0225] S706, when receiving a command to open the camera, the camera hardware abstraction determines the output camera based on the folding screen state.

[0226] It is understandable that when the foldable screen is in a folded state, the camera hardware abstraction determines that the camera outputting the image is the outer screen camera; when the foldable screen is in an unfolded state, the camera hardware abstraction determines that the camera outputting the image is the inner screen camera.

[0227] In S703 above, the folding screen is in a folded state, and the output camera confirmed in S706 is the external screen camera. The terminal device executes S707 and S708.

[0228] S707, Camera Hardware Abstraction drives the external screen camera to output images via the display driver.

[0229] The S708 external camera sends image information to third-party camera applications via camera driver, camera hardware abstraction, and camera frame.

[0230] Adaptively, third-party camera apps receive image information. After receiving the image information, the third-party camera apps perform drawing and rendering operations to display it on the foldable screen.

[0231] S709. When the terminal device receives an operation to instruct the unfolding of the foldable screen, the foldable screen management module receives information to instruct the unfolding of the foldable screen.

[0232] S710: After receiving information indicating that the foldable screen should be unfolded, the foldable screen management module calculates the foldable screen status.

[0233] It is understandable that the foldable screen in the S710 is in the unfolded state.

[0234] The S711 foldable screen management module sends the foldable screen status information calculated by the S710 to the camera hardware abstraction.

[0235] S713, the camera hardware abstraction, after receiving the foldable screen status information, determines the output camera based on the foldable screen status.

[0236] In S711 above, the folding screen is in the unfolded state, and the output camera confirmed in S713 is the inner screen camera. After confirming the output camera, the terminal device executes S714-S716, S721, and S722.

[0237] S714, After confirming the output camera, the camera hardware abstraction stops the external screen camera from outputting images via the camera driver.

[0238] In this embodiment of the application, "outputting the image" can be understood as outputting the acquired image.

[0239] The idea that the camera hardware abstraction stops the external screen camera via the camera driver can be understood as: the camera hardware abstraction stops the external screen camera from acquiring images and stops transmitting image information to the camera hardware abstraction layer.

[0240] S715, Camera Hardware Abstraction configures the internal screen camera to output images via camera driver configuration.

[0241] The S716 internal camera sends image information to third-party camera applications via camera driver, camera hardware abstraction, and camera frame.

[0242] S717. When a third-party camera application receives a command to turn off the camera, the third-party camera application sends the command to the camera hardware abstraction via the camera frame.

[0243] When the S718 camera hardware abstraction receives a command to turn on the camera, it stops the output camera from outputting images via the camera driver and powers down the inner screen camera and the outer screen camera.

[0244] The image output camera confirmed in S713 above is the internal screen camera. In S718, the camera hardware abstraction camera driver stops the internal screen camera from outputting images.

[0245] For example, the camera switching timing and display interface of the terminal device in this application embodiment are the same as those described above. Figure 4 The camera switching sequence of the terminal device in the process is similar, and you can refer to the above for details. Figure 5B and Figure 5C The explanation will not be repeated here.

[0246] Therefore, in the process shown in the embodiments of this application, when the terminal device switches from the folded state to the unfolded state, the external screen camera of the terminal device can automatically switch to the internal screen camera, reducing the situation where the external screen camera is used to output images after the terminal device switches from the folded state to the unfolded state, thereby reducing the scenarios where the terminal device cannot record, live stream, or video directly in front of it.

[0247] It is understood that in the embodiments of this application, the process of switching the terminal device from the unfolded state to the folded state during the use of the camera is similar to the process of switching the terminal device from the folded state to the unfolded state during the use of the camera described above, and will not be repeated here.

[0248] In summary, configuring the front-facing camera for image output based on the foldable screen state allows the front-facing camera to change with the foldable screen state, thereby reducing misuse of the front-facing camera and improving the user experience.

[0249] It should be noted that in the process of switching the terminal device from an unfolded state to a folded state, the following can be done: Figure 7A The inner screen camera and the outer screen camera are interchangeable, and the specific implementation process is similar, so it will not be described in detail here.

[0250] In the above embodiments, the terminal device first turns off the external screen camera and then turns on the internal screen camera. This may result in image interruptions while the internal screen camera is not fully operational. Therefore, the terminal device can also reduce image interruptions when the internal screen camera is not operational by turning on the internal screen camera and then turning off the external screen camera. This way, before the internal screen lights up and the internal screen camera is fully operational, the internal screen can display the image captured by the external screen camera, reducing instances where no image is displayed and further enhancing the user experience. Specific implementation details can be found below. Figures 7B-10 Related explanations.

[0251] For example, Figure 7B This is a schematic diagram illustrating the module interaction process of another screen display control method provided in this application embodiment. Taking a third-party camera application as an example, such as... Figure 7B As shown, the method includes:

[0252] S801. When the terminal device receives an operation indicating power-on while in folded state, the terminal device calls the camera hardware abstraction and foldable screen management module to power it on.

[0253] S802, after the foldable screen management module is turned on, calculates the foldable screen status.

[0254] It is understandable that the foldable screen management module calculates the foldable screen state based on information transmitted from one or more sensors. These sensors include, but are not limited to, accelerometers, Hall effect sensors, gyroscopes, or angle sensors.

[0255] In a possible implementation, the foldable screen management module can determine the foldable screen state based on the folding angle. For example, when the folding angle is between 0 and 75 degrees, the foldable screen is in a folded state; when the folding angle is between 75 and 180 degrees, the foldable screen is in an unfolded state. It is understood that the above-mentioned folding angle and the method of determining the foldable screen state based on the folding angle are only examples. The foldable screen state can also be determined based on other information (e.g., acceleration). This application embodiment does not limit the method of determining the foldable screen state.

[0256] S803, the foldable screen management module sends the foldable screen status information to the camera hardware abstraction. Adaptively, the camera hardware abstraction updates the foldable screen status after receiving the foldable screen status information.

[0257] In one possible implementation, the foldable screen management module can send foldable screen status information to the camera hardware abstraction based on pre-set events (e.g., power-on events, camera frame request events, etc.).

[0258] The foldable screen management module can also send foldable screen status information to the camera hardware abstraction based on changes in the foldable screen's state. For example, when the foldable screen state changes, the foldable screen status information is sent to the camera hardware abstraction.

[0259] S804. When a third-party camera application receives a command to instruct the camera to be turned on, the third-party camera application sends the command to the camera hardware abstraction through the camera frame.

[0260] It is understandable that when the terminal device receives an operation in the third-party camera application interface that instructs the user to start the video call function, recording function or live streaming function, the third-party camera application receives an instruction to open the camera.

[0261] When the S805 camera hardware abstraction receives a command to turn on the camera, it powers on both the inner and outer screen cameras via the camera driver.

[0262] When the S806 camera hardware abstraction receives a command to open the camera, it determines the output camera based on the folding screen state.

[0263] It is understandable that when the foldable screen is in a folded state, the camera hardware abstraction determines that the camera outputting the image is the outer screen camera; when the foldable screen is in an unfolded state, the camera hardware abstraction determines that the camera outputting the image is the inner screen camera.

[0264] In S803 above, the foldable screen is in a folded state, and the output camera confirmed in S806 is the external screen camera. The terminal device executes S807 and S808.

[0265] S807, the camera hardware abstraction drives the external screen camera to output images via the display driver.

[0266] The S808 external camera sends image information to third-party camera applications via camera driver, camera hardware abstraction, and camera frame.

[0267] Adaptively, third-party camera apps receive image information. After receiving the image information, the third-party camera apps perform drawing and rendering operations to display it on the foldable screen.

[0268] S809. When the terminal device receives an operation to instruct the unfolding of the foldable screen, the foldable screen management module receives information to instruct the unfolding of the foldable screen.

[0269] S810: After receiving information indicating that the foldable screen should be unfolded, the foldable screen management module calculates the foldable screen status.

[0270] It is understandable that the foldable screen in the S810 is in the unfolded state.

[0271] The S811 foldable screen management module sends the foldable screen status information calculated by the S810 to the camera hardware abstraction.

[0272] S813, the camera hardware abstraction, after receiving the foldable screen status information, determines the output camera based on the foldable screen status.

[0273] In the above S811, the folding screen is in the unfolded state, and the output camera confirmed in S813 is the inner screen camera.

[0274] S814, After confirming the output camera, the camera hardware abstraction configures the internal screen camera to output the image via the camera driver.

[0275] In this embodiment of the application, "outputting the image" can be understood as outputting the acquired image.

[0276] The idea that the camera hardware abstraction stops the external screen camera via the camera driver can be understood as: the camera hardware abstraction stops the external screen camera from acquiring images and stops transmitting image information to the camera hardware abstraction layer.

[0277] S815, when the camera hardware abstraction receives image information from the external camera transmitted via the camera driver, transmits the image information from the external camera to the application via the camera frame.

[0278] After the S816 internal screen camera starts up, it transmits image information to the camera hardware abstraction via the camera driver.

[0279] S817. After the camera hardware abstraction receives the image information transmitted by the inner screen camera, the camera driver stops the outer screen camera from outputting the image.

[0280] It is understandable that after the camera hardware abstraction receives the image information transmitted by the inner screen camera, it immediately stops the outer screen camera from outputting the image via the camera driver; the camera hardware abstraction can also stop the outer screen camera from outputting the image via the camera driver after a preset time period after the camera hardware abstraction receives the image information transmitted by the inner screen camera.

[0281] S818. When a third-party camera application receives a command to turn off the camera, the third-party camera application sends the command to the camera hardware abstraction via the camera frame.

[0282] When the S819 camera hardware abstraction receives a command to turn on the camera, it stops the output camera from outputting images through the camera driver and powers down the inner screen camera and the outer screen camera.

[0283] The image output camera confirmed in S813 above is the internal screen camera. In S819, the camera hardware abstraction camera driver stops the internal screen camera from outputting images.

[0284] In this way, when the foldable screen is in the unfolded state, the outer screen camera stops shooting images after the inner screen camera outputs an image, which allows the terminal device's camera to continuously capture and transmit images, and thus the terminal device to continuously display images, reducing the occurrence of data interruptions.

[0285] It should be noted that the above Figure 7B The third-party camera application shown in the process can also be replaced by the terminal camera application; the specific implementation process is similar and will not be repeated here. Furthermore, the above... Figure 7B The process shown can also be applied to preview scenarios. The specific implementation process is similar and will not be repeated here.

[0286] It should be noted that in the process of switching the terminal device from an unfolded state to a folded state, the following can be done: Figure 7B The inner screen camera and the outer screen camera are interchangeable, and the specific implementation process is similar, so it will not be described in detail here.

[0287] Therefore, in the process shown in this application embodiment, when the terminal device switches from a folded state to an unfolded state, the external screen camera of the terminal device can automatically switch to the internal screen camera. This reduces the instances where the external screen camera is used to display images after the terminal device switches from a folded state to an unfolded state, thereby reducing scenarios where the terminal device cannot record, broadcast live, or broadcast video directly in front of the user. Furthermore, the external screen camera is turned off after the internal screen camera is turned on, ensuring continuous image display and recording by the terminal device, thus enabling uninterrupted recording, live broadcasting, and video streaming.

[0288] It is understandable that in the above S817, after the camera hardware abstraction receives the image information transmitted by the inner screen camera, it immediately stops the outer screen camera from outputting the image via the camera driver.

[0289] For example, Figure 8 An embodiment provided in this application Figure 7B The process shown corresponds to a sequence diagram. Figure 8 In the text, lines 1-5 are used to represent the terminal device's display interface, external screen display status, internal screen display status, external screen camera's working status, and internal screen camera's working status, respectively.

[0290] Take, for example, a terminal device starts its camera to make a video call at time A, and then switches the folded screen from folded to unfolded at time B.

[0291] At time A, the terminal device is in a folded state; the outer screen lights up to display an image, while the inner screen is off. When the terminal device activates its camera for a video call at time A, the outer screen camera starts displaying an image, and the inner screen camera turns off. The terminal device displays on the outer screen... Figure 8 The interface shown in 'a' includes a view of the user on the steps.

[0292] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to display images. Responding to the change in the folded screen state, the terminal device exits low power consumption (LPM) mode and begins activating the inner screen camera. LPM mode can be understood as activating a single camera.

[0293] It's understandable that "activation" can be viewed as an action performed at a specific point in time, and a startup time is required from activation to the internal screen camera starting to work. At time C, the internal screen camera completes startup and begins displaying images. At time C, the terminal device shuts down the external screen camera and enters LPM mode to reduce power consumption.

[0294] Understandably, the inner screen needs a certain amount of time to light up. At time D, the inner screen lights up, and the terminal device displays on the inner screen. Figure 8 The interface shown in b is a view of the user on the steps.

[0295] At time E, the internal screen camera outputs an image, which is then displayed on the internal screen by the terminal device. Figure 8 The interface shown in 'c' includes a view of the user on the steps.

[0296] Figure 8 In the timing diagram shown, the terminal device switches from a folded state to an unfolded state. The external screen camera turns off after the internal screen camera turns on. The terminal device continuously outputs images from the camera, and the video call scene continues to flow without interruption.

[0297] It's understandable that when a terminal device activates either the external or internal screen camera, it can be considered to be in low-power mode. Compared to when both the external and internal screen cameras are activated, or when both are activated, low-power mode consumes less power.

[0298] It is understandable that in the above S817, the camera hardware abstraction can also stop the external screen camera from outputting images after a preset time has elapsed since the camera hardware abstraction received the image information transmitted by the internal screen camera.

[0299] The following is combined with Figure 9 and Figure 10The following sections explain the different scenarios in the screen display method: when the folded screen state switches from folded to unfolded, and when the folded screen state switches from unfolded to folded.

[0300] For example, Figure 9 An embodiment provided in this application Figure 7B The process shown corresponds to a sequence diagram. Figure 9 In the text, lines 1-5 are used to represent the terminal device's display interface, external screen display status, internal screen display status, external screen camera's working status, and internal screen camera's working status, respectively.

[0301] Take, for example, a terminal device starts its camera to make a video call at time A, and then switches the folded screen from folded to unfolded at time B.

[0302] At time A, the terminal device is in a folded state; the outer screen lights up to display an image, while the inner screen is off. When the terminal device activates its camera for a video call at time A, the outer screen camera starts displaying an image, and the inner screen camera turns off. The terminal device displays on the outer screen... Figure 8 The interface shown in 'a' includes a view of the user on the steps.

[0303] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images. Responding to the change in the folded screen state, the terminal device exits low-power LPM mode and begins activating the inner screen camera.

[0304] It's understandable that "starting" can be viewed as an action performed at a specific point in time. From start-up to the internal camera starting to work, there's usually a startup time. At time C, the internal camera finishes startup and begins displaying images. The external camera is then turned off by the terminal device after a preset duration at time C.

[0305] Understandably, the inner screen needs a certain amount of time to light up. At time D, the inner screen lights up, and the terminal device displays on the inner screen. Figure 9 The interface shown in b above includes a view of the user on the steps. Furthermore, since the preset time has not yet elapsed, both the inner and outer screen cameras are displaying images.

[0306] At time E, the preset duration ends, the external screen camera turns off, and the terminal device enters LPM mode to reduce power consumption. The terminal device then displays on the internal screen. Figure 9 The interface shown in 'c' includes a view of the user on the steps.

[0307] Figure 9In the timing diagram shown, the terminal device switches from the folded state to the unfolded state. The outer screen camera turns off after the inner screen camera turns on. The terminal device continuously outputs images from the camera to prevent the inner screen camera from taking too long to start up and causing a black screen phenomenon in certain situations (such as poor network, different mobile phone models, or continuous switching of folded screen states). This ensures that the video call scene continues uninterrupted and the video call is not interrupted, thus improving the user experience.

[0308] The above Figure 9 This section explains the process of switching the folded screen state from folded to unfolded in the screen display method. The following section will explain the process in conjunction with... Figure 10 This section explains the process of switching the folded screen state from unfolded to folded in the screen display method.

[0309] Figure 10 In the text, lines 1-5 are used to represent the terminal device's display interface, external screen display status, internal screen display status, external screen camera's working status, and internal screen camera's working status, respectively.

[0310] Take, for example, a terminal device starts its camera to make a video call at time A, and then switches the folded screen from unfolded to folded at time B.

[0311] At time A, the terminal device is in its unfolded state, with the inner screen lit up to display an image and the outer screen off. When the terminal device activates its camera for a video call at time A, the inner screen camera starts displaying an image, and the outer screen camera turns off. The terminal device displays on the inner screen... Figure 10 The interface shown in 'a' includes a view of the user on the steps.

[0312] At time B, the terminal device switches from the unfolded state to the folded state. The folded screen state changes, the inner screen turns off, and the outer screen lights up. The inner screen camera continues to output images. In response to the change in the folded screen state, the terminal device exits low-power LPM mode and begins to activate the outer screen camera.

[0313] It's understandable that "starting" can be interpreted as an action performed at a specific point in time. From the moment it's started until the external camera begins working, there's usually a startup time. At time C, the external camera finishes startup and begins displaying images. The terminal device shuts down the internal camera after a preset duration at time C.

[0314] Understandably, the external screen needs a certain amount of time to light up. At time D, the external screen lights up, and the terminal device displays on the external screen. Figure 10 The interface shown in b above includes a view of the user on the steps. Furthermore, since the preset time has not yet elapsed, both the inner and outer screen cameras are displaying images.

[0315] At time E, the preset duration ends, the internal screen camera turns off, and the terminal device enters LPM mode to reduce power consumption. The terminal device then displays on the external screen. Figure 10The interface shown in 'c' includes a view of the user on the steps.

[0316] Figure 10 As shown in the timing diagram, when the terminal device switches from the unfolded state to the folded state, the external screen camera turns off after it is turned on. The terminal device continuously outputs images from the camera to prevent the external screen camera from taking too long to start up and causing a black screen phenomenon in certain situations (such as poor network, different mobile phone models, or continuous switching of folded screen states). This ensures that the video call scene continues uninterrupted and the video call is not interrupted, thus improving the user experience.

[0317] It is understandable that the format of both the external and internal screen cameras outputting images can be interpreted as a dual-stream mode. For example, such as... Figure 9 CE time or Figure 10 During the CE (Content Output) time, the terminal device is in dual-stream mode. The time it takes for both the external and internal screen cameras to output images can be understood as the dual-stream time. When the terminal device activates either the external or internal screen camera, it can be understood as low-power mode. Compared to dual-stream mode, low-power mode consumes less power.

[0318] In possible implementations, the terminal device can determine the preset duration using a counter, timer, or other similar means.

[0319] Understandably, when the camera hardware abstraction receives image information transmitted from the inner screen camera, it starts a counter. Once the counter reaches a threshold, the camera driver stops the outer screen camera from displaying the image. Alternatively, when the camera hardware abstraction receives image information transmitted from the inner screen camera, it starts a timer. Once the timer finishes, the camera driver stops the outer screen camera from displaying the image.

[0320] It is understood that the threshold value in this application embodiment can be any value, and this application embodiment does not limit the value of the threshold. It is understood that the counter can count image information transmitted by the inner screen camera or image information transmitted by the outer screen camera. This application embodiment does not limit the technical method of the counter.

[0321] Furthermore, in the above embodiments Figure 9 and Figure 10 Based on this combination, during the transition from folded to unfolded state and back to folded state, when the folded screen state changes, the terminal device can maintain the image output for a preset duration after the folded screen state changes. In this way, when the folded screen state changes again within the preset duration, both the inner and outer screen cameras capture images, enabling rapid switching between cameras for image output.

[0322] In one possible implementation, the terminal device could also reset the counter or timer after the folding screen state changes, in order to extend the time for both the outer and inner screen cameras to output images.

[0323] Understandably, when the camera hardware abstraction determines that the folded screen state has changed again before the counter reaches the threshold, it resets the counter to extend the time for both the outer and inner screen cameras to capture images. Alternatively, when the camera hardware abstraction determines that the folded screen state has changed again before the timer expires, it resets the timer to extend the time for both the outer and inner screen cameras to capture images. In this way, when the folded screen state changes again after the counter starts, both the inner and outer screen cameras capture images, enabling rapid switching and image capture between the cameras.

[0324] The following is combined with Figure 11 The application of the counter is explained.

[0325] like Figure 11 As shown, rows 1-5 are used to represent the terminal device's display interface, external screen display status, internal screen display status, external screen camera's working status, and internal screen camera's working status, respectively.

[0326] Taking a terminal device that starts a video call at time A, switches the folded screen from folded to unfolded at time B, and switches the folded screen from unfolded to folded at time E as an example.

[0327] At time A, the terminal device is in a folded state; the outer screen lights up to display an image, while the inner screen is off. When the terminal device activates its camera for a video call at time A, the outer screen camera starts displaying an image, and the inner screen camera turns off. The terminal device displays on the outer screen... Figure 11 The interface shown in 'a' includes a view of the user on the steps.

[0328] At time B, the terminal device switches from a folded state to an unfolded state. The folded screen state changes, the outer screen turns off, and the inner screen lights up. The outer screen camera continues to output images. Responding to the change in the folded screen state, the terminal device activates the inner screen camera.

[0329] It's understandable that "starting" can be viewed as an action performed at a specific point in time. From start-up to the inner screen camera starting to operate, a startup time is typically required. At time C, the inner screen camera completes startup and begins displaying images. The terminal device starts a counter, and once the counter reaches a threshold, the outer screen camera is shut down.

[0330] Understandably, the inner screen needs a certain amount of time to light up. At time D, the inner screen lights up, and the terminal device displays on the inner screen. Figure 11 The interface shown in b above includes a view of the user on the steps. Furthermore, because the counter count has not reached the threshold, both the inner and outer screen cameras output images.

[0331] At time E, the terminal device switches from the unfolded state to the folded state. The folded screen state changes, the inner screen turns off, and the outer screen lights up. The terminal device resets the counter, and the counter starts counting again. After the counter reaches the threshold, the inner screen camera is turned off.

[0332] The external screen needs a certain amount of time to light up. At time G, the external screen lights up, and the terminal device displays on the external screen. Figure 11 The interface shown in 'c' includes a view of the user on the steps.

[0333] At time H, after the counter reaches the threshold, the inner screen camera turns off. The terminal device displays on the outer screen. Figure 11 The interface shown as d in the figure includes a view of the user on the steps.

[0334] Figure 11 If the counter cannot be reset, the terminal device may turn off the external screen camera at point E and then restart it. The external screen camera may not be able to display an image when the external screen lights up at time G, resulting in a long response time. Resetting the counter extends the dual-stream time. When the terminal device's folded screen state changes again within a short period, the external screen camera does not need to be restarted, thus shortening its response time.

[0335] In this way, during the continuous changes in the foldable screen state (for example, the terminal device switches from folded to unfolded state and then back to folded state), the terminal device can switch to the external screen camera to output images. The external screen camera does not need to be turned off and restarted, shortening the camera's response time and improving the user experience.

[0336] It should be noted that the above embodiments are all illustrated using the example of a terminal device with a foldable screen that folds in two directions (left and right) and the terminal device having two states: folded and unfolded. In possible implementations, the foldable screen could also be a foldable screen capable of folding in three or four directions (not shown in the figure). The terminal device's state could also include three types: folded, transitional, and unfolded.

[0337] For example, the foldable screen can be an N-foldable screen, where N is an integer greater than or equal to 2. The number of inner screen cameras corresponding to the foldable screen can also be multiple. When the foldable screen state changes, the terminal device can also switch from the outer screen camera to any inner screen camera, or switch from any inner screen camera to the outer screen camera, or switch between any inner screen cameras, based on a method similar to that in the above embodiments. Specific details are not elaborated here.

[0338] For example, when there are two internal screen cameras, the terminal device can switch between the external screen camera and the left internal screen camera, the external screen camera and the right internal screen camera, or the left internal screen camera and the right internal screen camera when the folding screen state changes, etc.

[0339] It can be understood that the switching between the above-mentioned front cameras of the inner screen and the outer screen may be implemented in such a way that the terminal device can also implement the linkage between the front camera and the rear camera. For example, when the front camera and the rear camera are shooting simultaneously, when the state of the folding screen of the terminal device changes, the terminal device can also implement the switching between the front cameras, the switching between the rear cameras, or the switching between the front camera and the rear camera.

[0340] Exemplarily, taking the folding screen of the terminal device with three folds as an example, a back panel without a screen can be arranged behind the middle screen of the folding screen, and the back panel can include multiple rear cameras. When the folding screen is folded, the back panel without a screen can be hidden, and one of the screens of the folding screen can be used as the back panel. The camera in the screen used as the back panel can also be understood as a rear camera; when unfolded, the camera in the screen used as the back panel can be converted into a front camera, and the camera of the back panel of the middle screen is a rear camera. In this implementation, when the state of the folding screen of the terminal device changes, the terminal device can also implement the switching between the front cameras, the switching between the rear cameras, or the switching between the front camera and the rear camera.

[0341] In a possible implementation, when the state of the terminal device includes three states: the folded state, the transition state, and the unfolded state, in the transition state, the terminal device can control both the inner screen and the outer screen to be lit, the inner screen displays the image captured by the inner screen camera, and the outer screen displays the image captured by the outer screen camera. Exemplarily, it can be defined that when the included angle between two adjacent screens of the folding screen is 0 - 30 degrees, it is the folded state, when it is 30 - 45 degrees, it is the transition state, and when it is 45 - 180 degrees, it is the unfolded state.

[0342] Of course, the camera switching control in the embodiments of the present application can also be applied to shooting modes such as dual - camera video recording. In this shooting mode, the outer screen displays the image captured by the rear camera, and the inner screen displays the images of the front camera and the rear camera in a picture - in - picture form. When the folding state changes, the terminal device can also implement the switching between the front cameras, the switching between the rear cameras, or the switching between the front camera and the rear camera.

[0343] The method for displaying the device screen in the embodiments of the present application has been described above. Next, the terminal device for executing the above - mentioned screen display method provided in the embodiments of the present application will be described. Those skilled in the art can understand that the method and the device can be combined and cited with each other. The terminal device provided in the embodiments of the present application can execute the steps in the above - mentioned screen display method.

[0344] The screen display method provided in this application can be applied to electronic devices with foldable screens. Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technology or device form used in the terminal devices.

[0345] For example, Figure 12 This is a schematic diagram of a terminal device provided in an embodiment of this application. The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 180, and a display screen 194, etc. The sensor module 180 may include at least: an angle sensor 180A, a gyroscope sensor 180B, a touch sensor 180C, and an accelerometer sensor 180D, etc.

[0346] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device. In other embodiments of this application, the terminal device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0347] The processor 110 may include one or more processing units. These processing units may be independent devices or integrated within one or more processors. The processor 110 may also include memory for storing instructions and data.

[0348] USB port 130 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge terminal devices, and can also be used for data transfer between terminal devices and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0349] The charging management module 140 receives charging input from the charger. The charger can be a wireless charger or a wired charger. The power management module 141 connects the charging management module 140 to the processor 110.

[0350] The wireless communication function of the terminal device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0351] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Antennas in terminal equipment can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0352] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on terminal devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0353] The wireless communication module 160 can provide solutions for wireless communication applications on terminal devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), etc.

[0354] The terminal device implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering.

[0355] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device may include one or N display screens 194, where N is a positive integer greater than 1. In this embodiment, the display screen 194 may be a foldable screen or a large screen, etc.

[0356] In some embodiments, the foldable screen can be a foldable screen capable of 2 folds, 3 folds, or 4 folds, etc., or a foldable screen capable of 3 folds or 4 folds, etc. (not shown in the figure). For example, the foldable screen is a foldable screen capable of folding left and right.

[0357] The foldable screen state can include two states: folded state and unfolded state; the foldable screen state can also include three states: folded state, transition state, and unfolded state.

[0358] In some embodiments, the display screen 194 may also be the outer screen of the terminal device. The understanding of the outer screen can refer to the description of the outer screen in the above embodiments, and will not be repeated here. In some embodiments, the display screen 194 may also be the inner screen of the terminal device. The understanding of the inner screen can refer to the description of the inner screen in the above embodiments, and will not be repeated here.

[0359] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0360] Internal memory 121 can be used to store executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area.

[0361] Angle sensor 180A is used to measure angles. In this embodiment, when the terminal device is a foldable screen phone, the angle sensor 180A is used to determine the folding angle of the foldable screen phone.

[0362] The gyroscope sensor 180B can be used to determine the motion posture of a terminal device. In this embodiment, the gyroscope sensor 180B can be used to determine the landscape or portrait state of the terminal device. When the terminal device is a foldable phone, the gyroscope sensor 180B can also be used to determine the upper or lower half of the foldable phone screen.

[0363] Touch sensor 180C can be disposed on display screen 194, and touch sensor 180C and display screen 194 constitute a touch screen, or "touchscreen". In this embodiment, touch sensor 180C is used to receive touch operations from the user on the touch screen, such as receiving user trigger operations for opening a video application, opening a short video application, or triggering functional controls in the application.

[0364] Accelerometer 180D is used to monitor the acceleration of the folding action. In this embodiment, when the terminal device is a foldable screen phone, the accelerometer 180D is used to determine the acceleration when the foldable screen is folded, or the acceleration when the foldable screen is unfolded.

[0365] This application provides a terminal device, which can also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc.

[0366] The terminal device includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the terminal device to perform the above-described method.

[0367] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0368] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0369] This application provides a computer program product, which includes a computer program that, when run, causes the computer to perform the above-described method.

[0370] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0371] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A screen display method characterized by, The application is applied to a terminal device, the terminal device comprises an outer display screen, an inner display screen and a rear camera, the outer display screen is provided with an outer screen camera, the inner display screen is provided with an inner screen camera, the terminal device is foldable, and the method comprises the following steps: When the terminal device is folded, the outer display screen displays a first image, the inner display screen does not display, the first image is an image collected by the outer screen camera, and the outer display screen displays a sixth image, the sixth image is an image collected by the rear camera; The terminal device is unfolded; During the process that the terminal device is unfolded, the inner display screen displays a second image; When the terminal device is unfolded, the inner display screen displays a third image, the third image is an image collected by the inner screen camera, and the inner display screen displays the sixth image.

2. The method of claim 1, wherein, The second image is not the first image.

3. The method according to claim 1 or 2, characterized in that, The second image is not the third image.

4. The method of claim 2, wherein, The second image is a blurred image.

5. The method of claim 4, wherein, The second image is obtained by processing an image collected by the outer screen camera.

6. The method according to any one of claims 1-2, 4-5, characterized in that, The method further comprises the following steps: The inner display screen or the outer display screen displays a user interface, the user interface comprises one or more images; A first operation on a fourth image in the one or more images is detected, and the fourth image is set as the second image.

7. The method of claim 6, wherein, The user interface further comprises a confirmation control and a control for adding a blur effect, The detection of the first operation on the fourth image in the one or more images and the setting of the fourth image as the second image comprise the following steps: Through a second operation on the fourth image in the one or more images and a third operation on the confirmation control, the fourth image is set as the second image; or through the second operation on the fourth image in the one or more images, a fourth operation on the control for adding a blur effect and the third operation on the confirmation control, an image obtained by blurring the fourth image is set as the second image.

8. The method of any one of claims 1-2, 4-5, wherein, During the process that the terminal device is unfolded, the method further comprises the following steps: The terminal device stops the outer screen camera from outputting an image, and stops the outer screen camera from displaying the image that has been outputted; The terminal device configures the inner screen camera to output an image; The terminal device controls the inner display screen to be lighted.

9. The method of claim 8, wherein, The inner display screen displays the third image when the terminal device is unfolded comprises the following steps: When the terminal device is in the unfolded state and the inner screen camera outputs an image, the terminal device displays the image outputted by the inner screen camera on the lighted inner display screen.

10. The method of any one of claims 1-2, 4-5, wherein, After the inner display screen displays the third image when the terminal device is unfolded, the method further comprises the following steps: The terminal device is folded; During the process that the terminal device is folded, the outer display screen displays the second image; When the inner display screen displays the second image, the inner display screen displays the sixth image.

11. The method of claim 1, wherein, The outer display screen displays the first image when the terminal device is folded comprises the following steps:

12. The method of claim 1, wherein, ​ In a case where the terminal device is folded, the terminal device receives an indication of a target application using a camera, and the outer display screen displays the first image.

13. The method of claim 12, wherein, The terminal device receiving the indication of the target application using the camera includes the terminal device receiving an indication that the target application starts a preview function.

14. The method according to claim 12 or 13, characterized in that, The terminal device receiving the indication of the target application using the camera and the outer display screen displaying the first image includes: The terminal device receiving the indication of the target application using the camera controls the outer screen camera and the inner screen camera to be powered on. After the outer screen camera is powered on, the outer display screen displays the first image.

15. The method of any of claims 1-2, 4-5, 11-13, wherein: In a case where the terminal device is folded, the terminal device receives an indication of a target application using a camera, and the outer display screen displays the first image. In a case where the terminal device is folded, the terminal device receives an indication of a target application using a camera, and the outer display screen displays the first image.

16. The method of any of claims 1-2, 4-5, 11-13, wherein: In a case where the terminal device is folded, the terminal device receives an indication of a target application using a camera, and the outer display screen displays the first image. In a case where the terminal device is folded, the terminal device receives an indication of a target application using a camera, and the outer display screen displays the first image.

17. A screen display method characterized by comprising: The terminal device includes an outer display screen and an inner display screen, the outer display screen is provided with an outer screen camera, the inner display screen is provided with an inner screen camera, and the terminal device is foldable. The method includes: In a case where the terminal device is folded, the outer display screen displays a first image, and the inner display screen does not display. The terminal device is unfolded. During the process of unfolding the terminal device, the inner display screen displays an image captured by the outer screen camera for a period of time. In a case where the terminal device is unfolded, the inner display screen displays a third image, and the third image is an image captured by the inner screen camera.

18. The method of claim 17, wherein, During the process of unfolding the terminal device, the method further includes: The terminal device configures the inner screen camera to output an image. The terminal device keeps the outer screen camera outputting an image until the terminal device stops the outer screen camera outputting an image when a preset condition is met. The terminal device controls the inner display screen to be lit. In a case where the terminal device is unfolded, the inner display screen displays a third image includes: In a case where the terminal device is unfolded and the inner screen camera outputs an image, the terminal device displays the image output by the inner screen camera on the lit inner display screen.

19. The method of claim 18, wherein, The terminal device keeps the outer screen camera outputting an image until the terminal device stops the outer screen camera outputting an image when a preset condition is met includes: The terminal device keeps the outer screen camera outputting an image and outputs the image output by the outer screen camera until the terminal device stops the outer screen camera outputting an image and does not display the image output by the outer screen camera after the image output by the inner screen camera is output. The inner display screen displays the image sent by the outer screen camera in the interval between the inner display screen lighting and the image sent by the inner screen camera.

20. The method of claim 18, wherein, The terminal device stops the outer screen camera from sending the image when a preset condition is met. The terminal device stops the outer screen camera from sending the image when a preset time length is reached.

21. The method of claim 20, wherein, The terminal device stops the outer screen camera from sending the image when a preset time length is reached, including: The terminal device keeps the outer screen camera from sending the image and sends the image sent by the outer screen camera, and stops the outer screen camera from sending the image after the image sent by the inner screen camera is sent, and stops the outer screen camera from sending the image when the preset time length is reached. The inner display screen displays the image sent by the outer screen camera in the interval between the inner display screen lighting and the image sent by the inner screen camera.

22. The method of claim 20 or 21, wherein, The terminal device stops the outer screen camera from sending the image when a preset time length is reached, including: The terminal device stops the outer screen camera from sending the image when a counter reaches a threshold value.

23. The method of claim 20 or 21, wherein, The method further includes: When the terminal device keeps the outer screen camera from sending the image, if the terminal device detects that the terminal device is folded, the terminal device controls the image sent by the outer screen camera to be sent and controls the outer display screen to be lighted, and displays the image sent by the outer screen camera when the outer display screen is lighted.

24. The method of claim 23, wherein, When the terminal device keeps the outer screen camera from sending the image, if the terminal device detects that the terminal device is folded, the method further includes: The terminal device controls the outer screen camera to keep sending the image for the preset time length again.

25. The method of claim 24, wherein, The terminal device controls the outer screen camera to keep sending the image for the preset time length again, including: The terminal device controls a counter to count again.

26. The method of any one of claims 17-21, 24-25, wherein, The terminal device further includes a rear camera. When the outer display screen displays the first image, the outer display screen further displays a sixth image, and the sixth image is an image collected by the rear camera. When the inner display screen displays the third image, the inner display screen further displays the sixth image.

27. The method of claim 26, wherein, During unfolding of the terminal device, the inner display screen displays the sixth image.

28. The method of any one of claims 17-21, 24-25, wherein, The outer display screen displays the first image when the terminal device is folded, including: When the terminal device is folded, the terminal device receives an indication of using a camera of a target application, and the outer display screen displays the first image.

29. The method of claim 28, wherein, The terminal device receives an indication of using a camera of a target application, including that the terminal device receives an indication that the target application starts a video call function, a video recording function or a live broadcast function.

30. The method of claim 29, wherein, The terminal device receives an indication of using a camera of a target application, and the outer display screen displays the first image, including: The terminal device receives an indication of using a camera of a target application, and controls the outer screen camera and the inner screen camera to be powered on; After the outer screen camera is powered on, the outer display screen displays the first image.

31. The method of any one of claims 17-21, 24-25, wherein, After the inner display screen displays the third image in the unfolded state, the method further comprises: The terminal device is folded; During folding of the terminal device, the outer display screen displays the image captured by the inner screen camera for a period of time; In the folded state of the terminal device, the outer display screen displays a fourth image.

32. The method of any of claims 17-21, 24-25, wherein: When the angle between the two sides of the inner display screen is less than or equal to a first value, the terminal device is in a folded state; When the angle between the two sides of the inner display screen is greater than or equal to a second value, the terminal device is in an unfolded state, the second value being greater than the first value.

33. The method of any of claims 17-21, 24-25, wherein: When the angle between the two sides of the inner display screen is less than a third value, the terminal device is in a folded state; When the angle between the two sides of the inner display screen is greater than or equal to a third value, the terminal device is in an unfolded state.

34. An image processing method, characterized by, The method is applied to a terminal device, the terminal device comprising an outer display screen and an inner display screen, the outer display screen being provided with an outer screen camera, the inner display screen being provided with an inner screen camera, the terminal device being foldable, the method comprising: In a time period from a first time to a second time, the terminal device is in a folded state, the outer display screen is on, the outer screen camera captures an image and outputs the image, the outer display screen displays the image output by the outer screen camera, and the inner display screen is off; In a time period from the second time to a third time, the terminal device is unfolded from the folded state, the outer display screen is switched from on to off, and the inner display screen is turned on; In a time period from the third time to a fourth time, the outer screen camera is turned off, the inner screen camera is started, and the inner display screen is turned on; After the fourth time, the inner screen camera outputs an image, and the inner display screen displays the image output by the inner screen camera; In a time period from when the inner display screen is turned on to when the inner screen camera outputs the image, the terminal device processes at least one image captured by the outer screen camera, so that the processed image has a blur effect when displayed on the inner display screen.

35. The method of claim 34, wherein, The method further comprises, after the inner screen camera is started, the terminal device configuring an instruction for the inner screen camera to output an image.

36. The method of claim 34 or 35, wherein, The method further comprises, after the inner display screen is turned on and before the inner display screen displays the image output by the inner screen camera, the inner display screen displaying a blurred image, the blurred image being an image obtained by processing at least one image captured by the outer screen camera.

37. The method of claim 34 or 35, wherein, The terminal device processes at least one image captured by the outer screen camera, comprising: The camera application of the terminal device acquires image information of at least one frame of image captured by the outer screen camera within a time period from the first time point to the second time point from the camera framework, and adds a blur effect.

38. The method of claim 34 or 35, wherein, The method further comprises: after the outer screen camera is closed and before the inner display screen displays the image output by the inner screen camera, the inner display screen displays a blur animation.

39. The method of claim 34 or 35, wherein, The method further comprises: After the terminal device is unfolded from the folded state, the terminal device is in an unfolded state.

40. The method of claim 34, wherein, when the angle between the two sides of the inner display screen is less than or equal to a first value, the terminal device is in the folded state; when the angle between the two sides of the inner display screen is greater than or equal to a second value, the terminal device is in an unfolded state, and the second value is greater than the first value.

41. The method of claim 40, wherein, The first value is 30 degrees, and the second value is 45 degrees.

42. The method of any one of claims 34-35, 40-41, wherein, when the angle between the two sides of the inner display screen is less than a third value, the terminal device is in the folded state; when the angle between the two sides of the inner display screen is greater than or equal to the third value, the terminal device is in the unfolded state.

43. The method of claim 42, wherein, The third value is 75 degrees.

44. The method of any one of claims 34-35, 40-41, wherein, The terminal device further comprises a rear camera; within the time period from the first time point to the second time point, the outer display screen also displays an image captured by the rear camera; after the fourth time point, the inner display screen also displays the image captured by the rear camera.

45. A terminal device, comprising: comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of any one of claims 1-44. 46.A computer readable storage medium, storing a computer program, characterized in that, When the computer program is run by the terminal device, the terminal device executes the method of any one of claims 1-44.

Images