Display method and related device

Abstract

The invention provides a display method and a related device, and the method comprises the steps: after a Sink end receives a picture projected by a Source end, the Sink end can display an originally displayed picture and a screen projection picture in a split-screen display manner by default, determines the transverse length of a screen projection window according to the length-width ratio of a display screen of the Source end, and displays the screen projection picture in a split-screen display manner. Therefore, the length-width ratio of the projection screen window is matched with the length-width ratio of the Source end display screen, the area of filling areas on the left side and the right side of the projection screen window during split-screen display is reduced, the utilization rate of the Sink end to the area of the display screen is increased, and the browsing experience of a user can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a display method and related devices. Background Art

[0002] With the development of computer technology, different devices can be projected to each other, so that screen sharing can be achieved. Taking the projection scenario including mobile phones and folding screen devices as an example, when the mobile phone is projected to the folding screen device, a projection window can be presented on the display screen of the folding screen device, and the screen content obtained from the mobile phone can be displayed in the projection window.

[0003] In some cases, users will adjust the projection window and the original screen displayed on the sink end to split the screen to the left and right or top and bottom. However, in the split-screen display scenario, the sink end has a low utilization rate of the display area, which damages the user's browsing experience.

[0004] Therefore, for screen projection scenarios, it is necessary to study display methods that can better utilize the display area. Summary of the invention

[0005] The purpose of the present application is to provide a display method and related devices. When the method is implemented, after the Sink end receives the screen projected from the Source end, the Sink end can display the originally displayed screen and the projected screen in a split-screen display by default, and determine the horizontal length of the projection window according to the aspect ratio of the Source end display screen, so that the aspect ratio of the projection window is adapted to the aspect ratio of the Source end display screen, reducing the area of ​​the filling area on the left and right sides of the projection window during split-screen display, improving the utilization rate of the display screen area on the Sink end, and effectively improving the user's browsing experience.

[0006] The above-mentioned and other objects are achieved by the features of the independent claims. Further implementations are reflected in the dependent claims, the description and the drawings.

[0007] In a first aspect, the present application provides a display method, which is applied to a first device, the first device establishes a communication connection with a second device, the first device displays a first screen, and the second device displays a second screen, the method comprising: receiving a screen projection operation from a user; in response to the screen projection operation, displaying the first screen and the second screen as split screens on a display screen of the first device, wherein the sum of the length of the first screen and the length of the second screen is equal to the length of the display screen of the first device, or the sum of the width of the first screen and the width of the second screen is equal to the width of the display screen of the first device.

[0008] In this method, the first device is the Sink end in the screen projection process, and the second device is the Source end in the screen projection process. After the second screen displayed on the Source end is projected to the first device, the second screen will not be displayed on the first device in the form of a floating window, but will be displayed on the first device in a split-screen display together with the first screen originally displayed on the first device, and when the first screen and the second screen are displayed in a left-right split-screen manner, the horizontal length of the first screen and the second screen is equal to the horizontal length of the first device display screen; when the first screen and the second screen are displayed in a top-bottom split-screen manner, the vertical width of the first screen and the second screen is equal to the vertical width of the first device display screen. In other words, when the first screen and the second screen are displayed on the first device in a split-screen manner, the total area of ​​the two screens is equal to the area of ​​the first device display screen, and there will be no filling area in the first device for filling the screen (or the area of ​​the life filling area has been adjusted to be small enough by the first device, so it can be ignored). In this way, improving the utilization rate of the display screen area by the Sink end can effectively improve the user's browsing experience.

[0009] Optionally, the first device may be a folding screen device, and the second device may be a mobile phone.

[0010] Optionally, the first screen may be the home screen interface (i.e., desktop) of the first device, or an application interface provided by an application installed on the first device. Similarly, the second screen may be the home screen interface (i.e., desktop) of the second device, or an application interface provided by an application installed on the second device.

[0011] In combination with the first aspect, in a possible implementation, the application to which the first screen belongs is a focus application of the first device, and the focus application is the application on the first device that has most recently received a user operation.

[0012] In this embodiment, before screen projection, if the first device only displays one application interface (i.e., the first screen), the application to which the first screen belongs is the focus application of the first device, and the first device can display the received second screen in split screen with the first screen after establishing a screen projection connection with the second device. If the first device has simultaneously displayed two application interfaces (including the first screen) in a split screen or floating window manner, and the application to which the first screen belongs is the application that has recently received user operations on the first device, the electronic device can use the application to which the first screen belongs as the focus application of the first device, and the first device can display the received second screen in split screen with the first screen after establishing a screen projection connection with the second device. Since the focus application is more likely to be an application that users pay more attention to, after receiving the screen projected from the Source end, the electronic device can only retain the interface corresponding to the focus application in the above two application interfaces, and display it in split screen with the projection window, so that the content displayed on the Sink end is more in line with user expectations.

[0013] In combination with the first aspect, in a possible implementation, before displaying the first screen and the second screen in split screen on the display screen of the first device, the method also includes: determining whether, when the first screen and the second screen are displayed in split screen on the first device, the length or width of the first screen and the length or width of the second screen are both greater than a first threshold; displaying the first screen and the second screen in split screen on the display screen of the first device includes: displaying the first screen and the second screen in split screen on the display screen of the first device when the length or width of the first screen and the length or width of the second screen are both greater than or equal to the first threshold.

[0014] It should be understood that in the case of split-screen display, if the horizontal or vertical length of the split-screen window is too small, it will affect the user's browsing experience. Therefore, in this embodiment, the first device can determine the horizontal length (when the left and right split-screen display) or vertical width (when the top and bottom split-screen display) of the first screen and the second screen during split-screen display, and determine whether the horizontal length (when the left and right split-screen display) or vertical width (when the top and bottom split-screen display) of the two windows after the split-screen display is greater than the above-mentioned first threshold.

[0015] Taking the left and right split screen display as an example, if the horizontal lengths of the first screen and the second screen when they are displayed in left and right split screens are greater than or equal to the above-mentioned first threshold, it means that while ensuring that the first screen and the second screen fill the display screen of the first device horizontally, the horizontal width of the two windows is sufficient to ensure the user's browsing experience. The first device can display the first screen and the second screen in left and right split screens on the first device according to the calculated window ratio (that is, the ratio of the horizontal lengths of the first screen and the second screen calculated while ensuring that there is no padding area in the interface).

[0016] Optionally, when the first screen and the second screen are displayed in a left-right split screen, the first threshold can be 1 / 3 of the horizontal length of the first device display screen for example; optionally, when the first screen and the second screen are displayed in a top-down split screen, the first threshold can be 1 / 3 of the vertical width of the first device display screen.

[0017] In combination with the first aspect, in a possible implementation, the method also includes: when the length or width of the first screen or the length or width of the second screen is less than the first threshold, the first screen is displayed in full screen on the display screen of the first device, and the second screen is displayed as a floating window on the display screen of the first device.

[0018] Similarly, in this embodiment, if the display screen of the first device is filled horizontally when the first screen and the second screen are displayed in left and right split screens, and the calculated horizontal length of either the first screen or the second screen is less than the first threshold, in order to ensure the user's browsing experience, the first device may no longer display the first screen and the second screen in split screens on the first device, but display the second screen in the form of a floating window on the first screen, and the first screen is displayed in full screen on the first device.

[0019] In combination with the first aspect, in a possible implementation, before displaying the first screen and the second screen in a split-screen manner on a display screen of the first device, the method also includes: determining whether an application to which the first screen belongs supports split-screen display; displaying the first screen and the second screen in a split-screen manner on the display screen of the first device includes: displaying the first screen and the second screen in a split-screen manner on the display screen of the first device according to the first information, when the application to which the first screen belongs supports split-screen display.

[0020] In this embodiment, the application to which the first screen belongs is the current focus application of the first device. It is understandable that the application interface provided by some applications on the first device may only support full-screen display, and does not support split-screen display on the first device together with interfaces provided by other applications (including the second screen projected from the second device to the first device). Therefore, in an embodiment of the present application, the first device can first determine whether the currently launched focus application supports split-screen. If the focus application supports split-screen, the first device will subsequently display the first screen and the second screen on the first device in split-screen display.

[0021] In combination with the first aspect, in a possible implementation, the method also includes: when it is determined that the application to which the first screen belongs does not support split-screen display, displaying the first screen in full screen on the display screen of the first device, and displaying the second screen as a floating window on the display screen of the first device.

[0022] Similarly, when the focus application (i.e., the application to which the first screen belongs) does not support split screen, the first device will not subsequently display the first screen and the second screen in split screen on the first device, but will display the second screen as a floating window on the first screen, and the first screen will be displayed in full screen on the first device.

[0023] In combination with the first aspect, in a possible implementation, when the first screen and the second screen are displayed in a left-right split-screen manner on the display screen of the first device, the first screen includes a window control bar and a three-button navigation bar, and before the first screen and the second screen are displayed in split-screen on the display screen of the first device, the method also includes: determining first information, the first information being the difference between the width of the display screen of the first device and the width of the window control bar and the width of the three-button navigation bar; determining second information based on the aspect ratio of the display screen of the second device and the first information, the second information representing the length of the first screen when the second screen is displayed in split-screen on the first device, and the ratio of the length corresponding to the second information to the width corresponding to the first information is equal to the aspect ratio of the display screen of the second device.

[0024] It can be understood that in this embodiment, when the first image and the second image are displayed on the display screen of the first device in a left-right split screen mode, the vertical width of the first image and the second image is equal to the width of the display screen of the first device. Therefore, before completing the screen projection with the second device, the first device as the sink end can obtain information including the aspect ratio of the display screen of the second device (L source / W source ), the horizontal length L of the display screen of the first device sink and the vertical width W sink In the case where the projection window includes a window control bar and a three-button navigation bar, the first device also stores a default vertical width W1 of the window control bar and a default vertical width W2 of the three-button navigation bar. From a simple mathematical graphic relationship, it can be seen that the horizontal lengths of the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c are all L x , and the aspect ratio of the projection screen (i.e., the area of ​​the first screen except the width of the window control bar and the width of the three-button navigation bar) is the aspect ratio of the Source display. Combining the above relationship with simple algebraic conversion, we can get the following relationship: L x / (W sink -W1-W2)=L source / W source ; L x That is the length of the first picture, that is, the first information.

[0025] Correspondingly, the length of the second screen is the length of the first device display screen minus the length of the first screen. The first device can display the first screen and the second screen in the display screen of the first device in a left-right split-screen manner according to the calculated length, and the first screen and the second screen can fill the display screen of the first device.

[0026] In combination with the first aspect, in a possible implementation, when the first picture and the second picture are displayed on a display screen of the first device in a left-right split-screen manner, the length of the first picture is different from the length of the second picture.

[0027] In this embodiment, when the first screen and the second screen are displayed on the first device in a left-right split screen manner, the first device does not set the horizontal width of the first screen and the second screen in an evenly divided manner, but sets the length of the display area where the second screen is located on the basis of ensuring that no padding area is added on the left and right sides of the second screen, so that the second screen can fill the display area horizontally. Therefore, when the first screen and the second screen are displayed on the first device in a left-right split screen manner, the length of the first screen is different from the length of the second screen.

[0028] In a second aspect, the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect.

[0029] In a third aspect, the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions so that the electronic device executes a method as in the first aspect or any possible implementation of the first aspect.

[0030] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method in the first aspect or any possible implementation of the first aspect.

[0031] In a fifth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a method in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of a process of projecting a mobile phone screen onto a folding screen device provided in an embodiment of the present application;

[0033] Figure 2 A user interface diagram of a projection screen and an application screen displayed in a left-right split screen manner provided in an embodiment of the present application;

[0034] Figure 3 An interactive flow chart of a display method provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a scenario in which a screen corresponding to a focus application is determined to be a projection screen provided in an embodiment of the present application;

[0036] Figure 5 A schematic diagram of a scenario for determining a split-screen window ratio provided in an embodiment of the present application;

[0037] Figure 6 A software structure diagram of an electronic device provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0040] To facilitate understanding, the relevant terms involved in the embodiments of the present application are first introduced below.

[0041] (1) Folding screen devices

[0042] A foldable screen device is a smart device that uses a flexible display screen. Its feature is that the screen can change size by folding inward or outward like a book. This design allows the phone to switch between smaller and larger sizes to adapt to different usage scenarios. The most common foldable screen device is the foldable screen mobile phone, which usually uses an OLED panel. The OLED panel can emit light by itself, and after being bent at a certain angle, combined with the folding mechanism of the fuselage, the folding effect is achieved.

[0043] The folded state and the unfolded state are the two most basic forms of foldable screen devices. When in the unfolded state, the foldable screen device will have a larger screen size and can display the application content more fully. Correspondingly, when in the folded state, the screen size of the foldable screen device becomes smaller. In the present application, when the electronic device is a foldable screen device, and the foldable screen device receives the projection window from other devices as a sink, the electronic device can determine the display mode of the projection window according to the specific form. For example, when the electronic device is in the folded state, the electronic device displays the projection window in the form of a floating window; when the electronic device is in the unfolded state, the electronic device can display the projection window in a left-right split-screen manner (that is, the projection window is displayed on one side of the screen, and the interface originally displayed by the foldable screen device is displayed on the other side of the screen).

[0044] (2) Wireless screen projection

[0045] Wireless screen projection is a wireless transmission technology that allows users to transfer content (such as video, audio, images, etc.) on mobile devices such as mobile phones, tablets, and laptops to other external display devices (such as TVs, tablets, etc.) via wireless networks for display. Wireless screen projection technology is usually used for screen mirroring functions, allowing users to easily share content on their devices and expand the display screen.

[0046] In the screen projection process, two roles are usually involved, one is the sender (also called the Source) and the other is the receiver (also called the Sink). The Source usually refers to the device that sends content to the display device. The Sink usually refers to the device that receives and displays content from the Source. For example, when a user projects the screen on a mobile phone to a tablet, the mobile phone is the Source and the tablet is the Sink. During the screen projection process, the Source needs to complete operations such as screen content capture, encoding, and network transmission to obtain the projection data from the Source and send the projection data to the Sink. After receiving the projection data sent by the Source, the Sink usually decodes, renders, and displays it.

[0047] For example, after the folding screen device and the mobile phone are connected and multi-screen collaboration is turned on, the picture displayed on the mobile phone screen can be projected to the folding screen device and displayed in the form of a small window (hereinafter referred to as the "projection window") on the display screen of the folding screen device.

[0048] (3) Floating window

[0049] "Floating window" is a system tool of smart devices, which can float the window above the interface or on the desktop. As a reduced application interface, the floating window can be displayed on top of other application interfaces (such as full-screen application interfaces), and the floating window can still display the inherent interface style and function list of its corresponding application. Users can directly operate the controls in the floating window interface to use the corresponding functions without closing or exiting the application interface of other applications. For example, when playing games or watching videos in full screen, you suddenly receive messages from WeChat, QQ, etc., and the corresponding chat interface can be directly above the game interface in the form of a floating window. Users can view and reply to messages in the chat interface displayed in the small window without switching or exiting the currently open application.

[0050] In an embodiment of the present application, the projection window of other devices projected on the electronic device can be displayed on the screen of the electronic device in a floating window interface by default; the user can display the projection window and the interface originally displayed on the electronic device on the screen in a split-screen manner or a split-screen manner by dragging the projection window to the side of the screen. Alternatively, in some embodiments, the projection window of other devices projected on the electronic device can be displayed on the screen in a split-screen manner or a split-screen manner by default with the interface originally displayed on the electronic device, and the electronic device can adaptively adjust the aspect ratio of the projection window so that the projection window and the interface originally displayed on the electronic device cover the entire screen together, minimize the solid color filling area in the screen, and improve the utilization rate of the folding screen screen area to enhance the user's browsing experience.

[0051] (4) Split-screen display

[0052] The device's display screen is divided into two left and right display interfaces or two upper and lower display interfaces, and the two display interfaces do not overlap. The areas of the two interfaces can be the same or different, and the user can drag the dividing line between the two interfaces to change the areas of the two interfaces synchronously.

[0053] (5) Three-button navigation bar and window control bar

[0054] The three-button navigation bar is one of the core interface elements for users to interact with smart devices. It usually includes three core buttons, namely the return button, the home screen button, and the history task button. Among them, the home screen button can respond to the user's click operation to switch the electronic device from any user interface to the home screen interface. The return button can respond to the user's click operation to switch the electronic device from the currently displayed user interface to the previous user interface. The history task button can respond to the user's click operation to enable the electronic device to display the user interface corresponding to one or more tasks that the user has recently opened. Of course, the names of the various navigation keys in the three-button navigation bar can also be other, for example, the return button can be called Back Button, the home screen button can be called Home button, and the history task button can be called MenuButton, which is not limited in this application. In the embodiment of the present application, unless otherwise specified, each navigation key in the three-button navigation bar is a virtual button.

[0055] In addition, some electronic devices also support users to implement the functions corresponding to each navigation button in the three-button navigation bar through specific gesture operations. For example, the user can switch the electronic device from the currently displayed user interface to the previous user interface by sliding inward on the left and right sides of the electronic device screen (this operation is equivalent to the user clicking the return button); the user can switch the electronic device from the currently displayed user interface to the home screen interface by sliding up on the bottom of the electronic device screen (this operation is equivalent to the user clicking the home screen button); the user can also slide up and pause on the bottom of the electronic device screen (this operation is equivalent to the user clicking the historical task button) to make the electronic device display the user interface corresponding to one or more tasks that the user has recently opened. Therefore, in some embodiments, the three-button navigation bar may not be displayed on the display screen of the electronic device. In the screen projection scenario, regardless of whether the three-button navigation bar is displayed on the display screen of the Source end, when the screen projection window is displayed on the Sink end, the Sink end can display the three-button navigation bar at the bottom of the screen projection window, and the user can switch the screen projection window and the interface displayed on the Source end through the three-button navigation bar.

[0056] The window control bar is located at the top of the window and is an area used to display information such as the current application name and file name. For windows displayed in the Windows system, the window control bar usually contains multiple elements such as program icons, file names, and window controls; in an embodiment of the present application, the window control bar of the projection window displayed on the Sink end may only contain multiple window controls, which may specifically include a minimize control for minimizing the projection window, a maximize control for displaying the projection window in full screen on the Sink end, and a close control for closing the projection window and disconnecting the connection between the Sink end and the Source end. In addition, in some embodiments, the title bar of the projection window may also include a move control for moving the projection window on the screen.

[0057] With the development of computer technology, different devices can be projected to each other, so that screen sharing can be achieved. Taking the projection scenario including mobile phones and tablets as an example, when the mobile phone is projected to a folding screen device, a projection window can be presented on the display screen of the folding screen device, and the screen content obtained from the mobile phone can be displayed in the projection window.

[0058] Next, combine Figure 1 and Figure 2 This chapter describes the specific process of completing screen projection on the Sink and Source ends and adjusting the image displayed on the Sink end to a left-right split-screen display.

[0059] First of all, it should be noted that in the embodiments of the present application and subsequent embodiments, the electronic device serving as the Sink end may be a mobile phone (including a folding screen device), a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, etc., and the present application does not impose any restrictions on the specific type of the electronic device. In addition, other devices serving as the Source end may be mobile phones, tablet computers, wearable devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, etc., and the present application does not impose any restrictions on the specific types of the other devices. Specifically, the other devices serving as the Source end may project the application interface displayed by them on the display screen of the electronic device serving as the Sink end via WLAN or Bluetooth. For the sake of convenience, the following description will be given by taking the Sink end as a folding screen device and the Source end as a mobile phone as an example.

[0060] Figure 1 The user interface displayed on the folding screen device is shown during the process of projecting the screen of a mobile phone to the folding screen device.

[0061] like Figure 1As shown in (A) in FIG. 1 , the user interface 10 is the main interface of the application "Notes" displayed on the folding screen device after the user starts the application "Notes" on the folding screen device. At this time, no other device has projected the screen to the folding screen device.

[0062] At a certain moment, after the folding screen device as the Sink receives the screen projection from the mobile phone as the Source, the Sink can display the following Figure 1 The user interface 11 shown in (B) in FIG.

[0063] like Figure 1 As shown in (B), the user interface 11 may include a main interface 111 and a projection window 112. The main interface 111 may be the main interface of the application "Notes" installed on the Sink. The projection window 112 displays the screen projected from the Source to the Sink. It is assumed here that the main screen interface of the Source is currently displayed on the Source, so the main screen interface of the Source is also displayed in the projection window 112 on the Sink.

[0064] The projection window 112 may include a window control bar 112a and a three-button navigation bar 112b. Figure 1 As can be seen from (B), in the embodiment of the present application, after the screen projection connection is established between the Sink end and the Source end, the screen projection window 112 can be displayed in the user interface 11 as a floating window by default.

[0065] The window control bar 112a may include an expansion control 2a1, a minimize control 2a2, a close control 2a3, and a move control 2a4. Among them, the expansion control 2a1 can respond to the user's click operation to enable the Sink end to display the projection window 112 in full screen; the minimize control 2a2 can respond to the user's click operation to enable the Sink end to collapse the projection window 112 and display it on the side of the screen in the form of a capsule; the close control 2a3 can respond to the user's click operation to enable the Sink end to close the projection window 112 and disconnect the projection connection between the Sink end and the Source end; the move control 2a4 can respond to the user's long press and drag operation to enable the Sink end to change the position of the projection window 112 in the user interface 11 as the user drags.

[0066] The three-button navigation bar 112b may include a return button 2b1, a home screen button 2b2, and a history task button 2b3. Any of the buttons can respond to user operations, switch the user interface displayed on the Source end, and enable the Sink end to synchronously update the interface content displayed in the projection window 112. For details, please refer to the above description, which will not be repeated here and in subsequent embodiments.

[0067] It is understandable that the projection window displayed in the form of a floating window on the Sink end will inevitably block the interface originally displayed on the Sink end, causing inconvenience to the user. Therefore, in some embodiments, when the display area of ​​the Sink end is large enough (for example, the folding screen device in the unfolded state has a large enough display area), after the Source end successfully projects the screen to the Sink end, the user can operate on the Sink end to display the interface originally displayed on the Sink end and the projection window obtained by projecting the Source end in a left-right split screen or a top-bottom split screen on the display screen of the Sink end.

[0068] like Figure 1 As shown in (B) in the figure, by long pressing and dragging the mobile control 2a4 on the projection window 112, the user can change the position of the projection window 112 in the user interface 11. When the user drags the projection window 112 to the edge of the display screen of the sink end, the sink end can respond to the operation and display the main interface 111 and the projection window 112 in the display screen in a left-right split screen display, and display the following Figure 1 The user interface 13 shown in (C) in FIG.

[0069] like Figure 1 As shown in (C), the user interface 13 may include a sub-interface 131, a sub-interface 132, a dividing line 133, an area 134, and an area 135. Among them:

[0070] The screen displayed in the sub-interface 131 corresponds to the aforementioned Figure 1 The main interface 111 of the "Notes" application.

[0071] The screen displayed in the sub-interface 132 corresponds to the aforementioned Figure 1 The picture in the projection screen window 112.

[0072] It is understandable that due to the change in the display area, the size and shape of each element in the sub-interface 131 and the main interface 111 may be different. Similarly, the size and shape of each element in the sub-interface 132 and the projection window 112 may also be different.

[0073] The dividing line 133 is a dividing line between the sub-interface 131 and the sub-interface 132. In some embodiments, if the folding screen device is a two-folding screen device (ie, there is only one foldable portion of the device screen), the dividing line 133 may coincide with the folding portion by default.

[0074] Area 134 and area 135 are filled areas on the left and right sides of sub-interface 132, respectively. It is understandable that area 134 and area 135 can be solid color (e.g., pure black) areas. In some embodiments, the sink end can first obtain the background image corresponding to area 134 and area 135 on the desktop of the sink end as the bottom layer, and then add a solid color layer with a certain transparency and blur effect on the layer, so as to obtain the image for display in area 134 and area 135.

[0075] It is understandable that in the case where the folding screen device is a two-folding screen device, the fold line of the folding screen is generally the center line of the display screen. Combined with the above description, it can be seen that in the case of left and right split screen display, the two display areas divided by the dividing line 133 are both half of the display area of ​​the folding screen device. For the display area on the left side of the dividing line 133, the Sink end can adjust the width of the main interface of the application note so that the main interface of the application "Note" can horizontally cover 1 / 2 of the Sink end display screen. However, for the display area on the right side of the dividing line 133, since the aspect ratio of this area is likely not compatible with the aspect ratio of the Source end display screen, in order to keep the layout of each element in the projection screen on the Sink end (that is, the entire screen in the sub-interface 131 except the top window control bar and the bottom three-button navigation bar) as consistent as possible with the Source end display screen, the Sink end needs to keep the aspect ratio of the projection screen the same as the aspect ratio of the Source end display screen, so the Sink end cannot arbitrarily stretch the projection screen window horizontally to make the projection screen cover the other half of the display area. Therefore, on the premise of ensuring that the aspect ratio of the projected screen is the same as the aspect ratio of the Source display screen, the Sink end can center the sub-interface 132 in the other half of the display area of ​​the Sink end, and set a filling area on the left and right of the sub-interface 132, namely area 134 and area 135, so that these two areas and the sub-interface 132 can together cover the right half of the display area of ​​the Sink end.

[0076] Combining the above Figure 1 and Figure 2It can be seen from the description that after the screen projection process is completed, the screen projection window will be displayed on the Sink end in the form of a floating window, which will block the interface originally displayed on the Sink end; even after the user adjusts the display mode of the Sink end to split-screen display, when the aspect ratio of the split-screen window does not match the aspect ratio of the Source end device, the utilization rate of the display area of ​​the Sink end is still low, and the content that the user really needs to browse will not be displayed in area 134 and area 135 (these two areas are used as filling areas and are neither used to display the interface content of the application "Notes" nor to display the screen content projected from the Source end to the Sink end). Therefore, the existence of area 134 and area 135 reduces the utilization rate of the display area of ​​the Sink end to a certain extent, and also damages the user's browsing experience. Especially when the Sink end is a folding screen device, the user uses the folding screen device to obtain a better browsing experience through a larger display screen. The existence of area 134 and area 135 actually violates the original intention of the user to use the folding screen device.

[0077] In response to the defects of the above-mentioned method, the present application provides a display method. After implementing the method, after the Sink end receives the screen projected from the Source end, the Sink end can display the originally displayed screen and the projected screen in split-screen mode by default, and determine the horizontal length of the projection window according to the aspect ratio of the Source end display screen, so that the aspect ratio of the projection window is adapted to the aspect ratio of the Source end display screen, reducing the area of ​​the filling area on the left and right sides of the projection window during split-screen display, improving the utilization rate of the display screen area on the Sink end, and effectively improving the user's browsing experience.

[0078] Figure 2 The user interface displayed on the Sink end after executing the above display method is exemplified.

[0079] like Figure 2As shown, the user interface 21 may include a sub-interface 211, a sub-interface 212, and a dividing line 213. Among them, the sub-interface 211, the sub-interface 212 and the dividing line 213 are the sub-interface 211, the sub-interface 212 and the dividing line 213 in the user interface 21, respectively. The difference from the user interface 13 is that after the Sink end determines the horizontal length of the projection window according to the aspect ratio of the display screen of the Source end, due to the reduction in the area of ​​the display area on the right side of the dividing line 213, when the horizontal length of the display area on the right side is basically the same as the horizontal length of the sub-interface 212, the sub-interface 212 can basically fill the display area alone, and at this time, the Sink end can no longer display the filling area on the left and right sides of the sub-interface 212 (it can be considered that the horizontal length of the filling area on both sides of the sub-interface 212 has been reduced to a very small value, so these two filling areas can be ignored, and the sum of the horizontal lengths of the sub-interface 211 and the sub-interface 212 is equal to the length of the display screen of the Sink end). At this time, the display area on the left side of the dividing line 213 can be completely used to display the main interface of the application "Notes", and the display area on the right side of the dividing line 213 can be completely used to display the projection window.

[0080] In the embodiment of the present application, the Sink end can be called the "first device", the Source end can be called the "second device", the sub-interface 211 can be called the "first screen", and the sub-interface 212 can be called the "second screen".

[0081] Next, the interactive process of the display method provided by the present application is introduced. In the display method provided by the present application, the Sink end can be a folding screen device, a tablet computer, a laptop computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, etc., and the present application does not impose any restrictions on the specific type of the electronic device. In addition, other devices that act as the Source end to project the screen on the above-mentioned Sink end can be mobile phones, tablet computers, wearable devices, laptop computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPC), netbooks, etc., and the present application does not impose any restrictions on the specific type of the Source end. For the sake of ease of explanation, the following explanation will be given by taking the above-mentioned Sink end as a folding screen device and the Source end as a mobile phone as an example. Please refer to the details. Figure 3 .

[0082] like Figure 3As shown, the Sink end provided by the present application may include a first screen projection service module (PCAssistant1), a screen projection business module (AssociateAssistant), an activity management module (AMS / ATMS) and a window management module, and the Sink end may include a second screen projection service module (PCAssistant2). The display method provided by the present application may include but is not limited to the following steps:

[0083] S101. The Source end receives the user's screen projection operation.

[0084] When Wi-Fi is turned on on both the Source and Sink, users can perform corresponding screen projection operations on the Source (for example, turn on the screen projection function in the "Settings" application) to enable the Source to periodically send out request signals. When the Sink successfully detects the request signal sent by the Source, it means that the Sink has discovered the Source, and a communication connection can be established between the Sink and the Source.

[0085] In some embodiments, the user can also perform screen projection operations on the Sink end so that the Sink end periodically sends a request signal to the outside. When the Source end successfully detects the request signal sent by the Sink end, a communication connection can also be established between the Sink end and the Source end.

[0086] S102. The Source end sends a screen projection start instruction to the screen projection service module of the Sink end.

[0087] After the connection between Source and Sink is established, Source can send the screen projection start command to the screen projection service module of Sink. It should be understood that the screen projection start command is a pre-start command. Before the Sink officially displays the screen projection window, the command can be used to control the Sink to complete the preparatory work for displaying the screen projection window, including determining the display mode of the screen projection window on the Sink (for example, displaying it as a floating window or in split-screen mode), the shape and size of the screen projection window, determining the specific style of the screen projection window (including whether to add a window control bar, a three-button navigation bar, and the specific styles of the window control bar and the three-button navigation bar), and a series of preparatory work.

[0088] S103. The second screen projection service module at the Source end sends the aspect ratio of the display screen at the Source end to the first screen projection service module at the Sink end.

[0089] S104. The first screen projection service module at the Sink end sends the aspect ratio of the display screen at the Source end to the screen projection service module at the Sink end.

[0090] After the Source sends the screen projection start instruction to the screen projection service module on the Sink side, the screen projection service module can respond to the instruction and request the second screen projection service module on the Source side to obtain the aspect ratio of the Source side display screen. Correspondingly, the second screen projection service module on the Source side will respond to the request and send the aspect ratio of the Source side display screen to the screen projection service module on the Sink side, and the first screen projection service module on the Sink side will further send the aspect ratio of the Source side display screen to the screen projection service module on the Sink side.

[0091] S105. The first screen projection service module at the Sink end sends indication information indicating whether the currently launched focus application supports split screen to the screen projection service module at the screen projection Sink end.

[0092] It is understandable that in this application, the application interface of a certain application is originally displayed on the Sink end, and the application interface can be an application interface provided by a third-party application installed in the Sink end, such as an application The main interface may also be the application interface provided by the local application on the Sink side, such as the main interface of the application "Notes" or the main interface provided by the application "Desktop / Settings" (i.e., the desktop of the Sink side). The application interfaces provided by some applications on the Sink side may only support full-screen display, and do not support split-screen display on the Sink side together with interfaces provided by other applications (including the screen projected from the Source side to the Sink side). Therefore, in an embodiment of the present application, the Sink side can first determine whether the currently launched focus application supports split-screen through the first projection service module. If the focus application supports split-screen, the Sink side will subsequently display the screen originally displayed and the screen projected from the Source side in split-screen on the Sink side.

[0093] It should be noted that the above-mentioned focus application is the application that the user last operated on the Sink end. When the Sink end currently displays only one application interface, the above-mentioned focus application is the application to which the application interface belongs; when there are multiple application interfaces currently displayed on the Sink end (these multiple application interfaces are displayed in a split-screen manner, and these multiple interfaces can belong to the same application or different applications), the Sink end can determine the application to which the interface that last received the user's operation among multiple pages belongs as the focus application. This will be explained in more detail later, so I will not go into details here.

[0094] Optionally, in some embodiments, the Sink end may first execute step S105, and after determining that the current focus application on the Sink end supports split-screen display, execute steps S103-S104 to obtain the aspect ratio of the Source end display screen; if it is determined that the current focus application on the Sink end does not support split-screen display, the Sink end may not execute steps S103-S104 but directly execute step S106.

[0095] S106. The screen projection service module at the Sink end calculates the ratio of the split-screen window according to the aspect ratio of the display screen at the Source end.

[0096] Specifically, the first screen projection service module on the Sink end can calculate the ratio of the split-screen window according to the aspect ratio of the display screen on the Source end and the setting style of the projection window on the Sink end. When the picture originally displayed on the Sink end and the projection window are split-screen and displayed according to the calculated ratio, there may be no padding area on both sides of the projection window (or the desired width of the padding area on both sides of the projection window is small enough, so the padding area can be ignored). The setting style of the projection window mainly includes whether the projection window contains a window control bar, whether it contains a three-button navigation bar, and other factors that will affect the aspect ratio of the projection window. It is understandable that the default setting style of the projection window is set when the Sink end leaves the factory, so these parameters have been pre-stored in the Sink end when the Sink end leaves the factory.

[0097] Specifically, when the split-screen window is split into left and right windows, the ratio of the split-screen window is the ratio of the horizontal lengths of the two windows.

[0098] For the specific method of calculating the ratio of the split-screen window on the Sink side according to the aspect ratio of the Source side display screen, please refer to the relevant description of the subsequent embodiments, which will not be repeated here.

[0099] S107. The activity management module on the Sink side sends the window threshold of the split-screen window to the screen projection service module.

[0100] Optionally, the above ratio threshold may be 1 / 3 of the horizontal length of the Sink end display screen, or other values, which is not limited in the present application.

[0101] It should be understood that in the case of split-screen display, if the horizontal or vertical length of the split-screen window is too small, it will affect the user's browsing experience. Therefore, in the embodiment of the present application, the activity management module on the Sink side will send the window threshold of the split-screen window to the projection service module; after the Sink side calculates the ratio of the split-screen window, the projection service module will determine whether the original screen displayed on the Sink side and the projection window can be split-screen displayed according to the calculated ratio and the obtained window threshold based on the calculated ratio.

[0102] Specifically, based on the calculated ratio of the split-screen windows, determine whether the horizontal lengths of the two windows after the split-screen display (assuming that the above-mentioned window threshold is the threshold of the horizontal length, and the calculated ratio of the split-screen windows is also the ratio of the horizontal lengths of the two windows) are both greater than the above-mentioned window threshold. If the horizontal lengths of the two windows after the split-screen display are greater than or equal to the above-mentioned window threshold, it means that after the picture originally displayed on the Sink end and the projection window are split-screened according to the calculated ratio, the horizontal widths of the two windows are sufficient to ensure the user's browsing experience, then the Sink end can split-screen the picture originally displayed on the Sink end and the projection window according to the calculated ratio; if the horizontal length of any one of the two windows after the split-screen display is less than the above-mentioned window threshold, it means that the Sink cannot split-screen the picture originally displayed on the Sink end and the projection window according to the calculated ratio.

[0103] Taking the above-mentioned window threshold as 1 / 3 of the horizontal length of the Sink end display screen as an example, assuming that the ratio of the split-screen windows calculated by the aforementioned projection service module is 1:1, it means that when the picture originally displayed on the Sink end and the projection window are split-screen and displayed according to this ratio, the horizontal lengths of the two windows are both 1 / 2 of the horizontal length of the display screen, and both are greater than 1 / 3. The Sink end can split-screen and display the picture originally displayed on the Sink end and the projection window according to a 1:1 ratio; correspondingly, assuming that the ratio of the split-screen windows calculated by the aforementioned projection service module is 1:3, it means that when the picture originally displayed on the Sink end and the projection window are split-screen and displayed according to this ratio, the horizontal length of one window is 1 / 4 of the horizontal length of the display screen, and the horizontal length of the other window is 3 / 4 of the horizontal length of the display screen. Since 1 / 4 is less than 1 / 3, the Sink end cannot split-screen and display the picture originally displayed on the Sink end and the projection window according to a 1:3 ratio.

[0104] S108, the screen projection service module on the Sink side configures the startup items.

[0105] The Sink end configures the startup items of the projection window based on the two pieces of information: whether the focus application confirmed above supports split-screen and whether the screen originally displayed on the Sink end and the projection window can be split-screen and displayed according to the calculated ratio. The above configuration items may specifically include the display method of the projection window and the size, style and other parameters of the projection window.

[0106] Specifically, if the focus application supports split screen and the Sink end can split the originally displayed screen and the projection window according to the calculated ratio, then the display mode of the projection window in the above configuration item is to split the screen with the screen originally displayed on the Sink end; if the focus application does not support split screen and / or the Sink end cannot split the originally displayed screen and the projection window according to the calculated ratio, then the display mode of the projection window in the above configuration item is to be displayed in the form of a floating window on the screen originally displayed on the Sink end.

[0107] Optionally, in some embodiments, after the screen projection service module on the Sink side calculates the ratio of the split-screen window, the screen projection service module can first send the ratio of the split-screen window to the activity management module, which determines whether the currently started application is compatible with the acquired configuration items and window ratios, and returns the judgment result to the screen projection service module. Afterwards, the screen projection service module can configure the startup items of the screen projection window based on the judgment result and whether the focus application on the Sink side supports split-screen.

[0108] S109. The screen projection business module on the Sink side sends the configuration items and window ratios to the activity management module on the Sink side.

[0109] S110. The screen projection service module on the Sink side sends a window startup instruction to the activity management module on the Sink side.

[0110] S111. The activity management module at the Sink end sends an adaptive window start instruction to the window management module at the Sink end.

[0111] After the screen projection business module on the Sink side completes the configuration startup items, the screen projection business module on the Sink side will send the configuration items and the calculated window ratio to the activity management module on the Sink side, and send the window startup instruction (startActivity) to the activity management module on the Sink side. The activity management module will further process this request and send an adaptive window startup instruction to the window management module on the Sink side. The window management module is mainly responsible for creating and managing the projection window and controlling the display position of the projection window on the Sink side. Specifically, the above-mentioned adaptive window startup instruction may include information such as the aspect ratio of the projection window and the style of the projection window. The window management module can control the layout of the projection window on the Sink side based on this information, and determine the size and position of the window and the hierarchical relationship between the layers.

[0112] S112. The window management module on the Sink side sends callback parameters to the screen projection service module on the Sink side.

[0113] S113. The screen projection service module on the Sink side adjusts the interface window size.

[0114] The window management module on the sink side sends callback parameters to the screen projection service module on the sink side. The screen projection service module on the sink side can draw the display interface based on the callback parameters returned by the window management module and in combination with the hardware on the sink side including the CPU and GPU. Finally, the drawn interface (for details, please refer to Figure 2 The user interface 21) shown is output to the display screen at the Sink end for display.

[0115] In an embodiment of the present application, the Sink end can be referred to as a "first device", the Source end can be referred to as a "second device", and the window threshold can be referred to as a "first threshold".

[0116] Combined with the above description, it can be known that in this application, the Sink end will determine the display mode of the projection window according to whether the focus application supports split-screen display. Therefore, when the Sink end itself has already displayed two application interfaces in split-screen (these two application interfaces can belong to the same application or different applications), since the focus application is more likely to be an application that users pay more attention to, after receiving the screen projection from the Source end, the electronic device can only retain the interface corresponding to the focus application in the above two application interfaces, and display it in split-screen with the projection window, so that the content displayed on the Sink end is more in line with user expectations. Next, combined with Figure 4 Provide explanation.

[0117] like Figure 4 As shown in (A) of FIG. 1 , the folding screen device displays a user interface 41, and the user interface 41 may include a sub-interface 411, a sub-interface 412, and a dividing line 413. Among them, the screen displayed in the sub-interface 411 is an application The screen displayed in the sub-interface 412 is the main interface of the application "Notes", and the dividing line 413 is the dividing line between the sub-interface 411 and the sub-interface 412. At this time, the sub-interface 411 and the sub-interface 412 are in the user interface 41 in a left-right split screen manner.

[0118] Both sub-interface 411 and sub-interface 412 include mobile controls for moving the corresponding sub-interfaces. In addition, in an embodiment of the present application, the mobile controls included on each sub-interface can correspond to two different states, namely a selected state and a non-selected state, which can indicate whether the application corresponding to the interface is the focus application.

[0119] refer to Figure 4 In (A), the mobile control 411a on the sub-interface 411 is in a non-selected state (i.e., the filling color of the mobile control 411a is white), indicating that the user's last operation on the folding screen device was not performed on the sub-interface 411, that is, the application corresponding to the sub-interface 411 is not selected. At this moment, it is not the focus application on the foldable screen device; but the mobile control 412a on the sub-interface 412 is in a non-selected state (i.e., the fill color of the mobile control 412a is black), indicating that the user's last operation on the foldable screen device was not performed on the sub-interface 411, that is, the application "Notes" corresponding to the sub-interface 411 is now the focus application on the foldable screen device. It is understandable that if the user subsequently performs a certain user operation on the sub-interface 411, such as a click operation, the focus application on the foldable screen device can be switched from "Notes" to "Notes". Correspondingly, the mobile control 412a on the sub-interface 412 will switch to a selected state, and the mobile control 412a on the sub-interface 411 will switch to an unselected state.

[0120] When the focus application of the foldable screen device is "Notes", at a certain moment, Figure 4 After the folding screen device in the above example receives the screen projection from the mobile phone as the Source end as the Sink end, the folding screen device can display the following Figure 4 The user interface 42 shown in (B) in FIG.

[0121] like Figure 4 As shown in (B) in FIG. 4 , the user interface 41 may include a sub-interface 421, a sub-interface 412, and a dividing line 423. The sub-interface 421 is a screen projection window, in which the screen displayed is the screen projected from the Source end to the Sink, and the sub-interface 412 is the aforementioned Figure 4 In the sub-interface 412 of (A), the screen displayed is the main interface of the application "Notes", and the dividing line 423 is the dividing line between the sub-interface 411 and the sub-interface 412. At this time, the sub-interface 421 and the sub-interface 412 are in the user interface 42 in a left-right split screen manner.

[0122] Combined with the above description, it can be seen that since the focus application on the Sink end is "Notes" before the screen is projected, after the screen is projected with the Source end, the electronic device can retain the sub-interface 412 in the user interface 41 and the application Move to the background and it will not be displayed on the screen. The application interface.

[0123] Accordingly, in the present application, the folding screen device will adaptively adjust the horizontal length of the projection window (i.e., sub-interface 421) to avoid the appearance of padding areas on both sides thereof and improve the utilization rate of the display area at the sink end. Therefore, compared with the sub-interface 412 in the user interface 41, the horizontal length of the sub-interface 412 in the user interface 42 may also change accordingly.

[0124] Optionally, after the screen projection is completed, the electronic device will still keep the focus on the Sink end as "note", so in the user interface 42, the mobile control 412a on the sub-interface 412 is selected, and the mobile control 412a on the sub-interface 421 is unselected.

[0125] Optionally, in some embodiments, before and after the screen projection is completed, the sink end may keep the sub-interface corresponding to the focus application displayed on the same side of the dividing line. Figure 4 As shown in (A) and (B) in the figure, before the screen projection is completed, the sub-interface 412 is displayed on the right side of the dividing line 413. After the screen projection is completed, the Sink end can still display the sub-interface 412 on the right side of the dividing line 423, and display the projection window (i.e., sub-interface 421) on the left side of the dividing line 423. Optionally, in some embodiments, the Sink end can display the projection window on one side of the dividing line by default. Assuming that the Sink end can display the projection window on the right side of the dividing line by default, even if the sub-interface 412 was originally displayed on the right side of the dividing line 413 before the screen projection is completed, after the screen projection is completed, the Sink end can display the projection window (i.e., sub-interface 421) on the right side of the dividing line 423, and move the sub-interface 412 to the left side of the dividing line 423 for display.

[0126] Optionally, in some embodiments, before the screen projection is completed, one of the two application interfaces displayed on the Sink end can be displayed in full screen on the screen, and the other can be displayed in the form of a floating window on the screen. After the screen projection is completed, the Sink end will still retain the interface of the focus application among the two application interfaces, and display it together with the projection window in a split-screen manner on the display screen of the Sink end.

[0127] In the embodiment of the present application, the Sink end can be referred to as the "first device", the Source end can be referred to as the "second device", the sub-interface 412 can be referred to as the "first screen", and the sub-interface 421 can be referred to as the "second screen". When the Sink end displays the user interface 41, the application corresponding to the sub-interface 411 It can be called a background application, and the application "Notes" corresponding to the sub-interface 412 can be called a "focus application".

[0128] Next, combine Figure 5 The specific method of determining the split screen window ratio by the Sink end according to the aspect ratio of the display screen of the Source end is described. Since the projection window generally includes a window control bar and a three-button navigation bar, the embodiment of the present application takes the scenario where the projection window includes a window control bar and a three-button navigation bar as an example for description.

[0129] like Figure 5As shown in (A), after the folding screen device as the Sink receives the screen projected from the mobile phone as the Source, the folding screen device displays the user interface 51. The user interface 51 may include a sub-interface 511, a sub-interface 512, and a dividing line 513. Among them, the sub-interface 512 is the screen projection window, in which the screen displayed is the screen projected from the Source end to the Sink, the sub-interface 511 is the application interface of the application "Notes", and the dividing line 513 is the dividing line between the sub-interface 511 and the sub-interface 512. At this time, the sub-interface 5111 and the sub-interface 512 are in the user interface 51 in a left-right split screen manner, and the folding screen device has determined the horizontal length of the screen projection window (i.e., the sub-interface 512) according to the aspect ratio of the Source end display screen to avoid the appearance of padding areas on both sides.

[0130] It is understandable that before the screen is projected with the Source, the folding screen device as the Sink can obtain information including the aspect ratio of the Source display (L source / W source ), the horizontal length L of the Sink end display sink and the vertical width W sink When the projection window includes a window control bar and a three-button navigation bar, the Sink end will also store the default vertical width W1 of the window control bar and the default vertical width W2 of the three-button navigation bar. The main purpose of the Sink end is to calculate the horizontal length L of the projection window (i.e., the sub-interface 512) based on the above-mentioned information. x .

[0131] like Figure 5 As shown in (B) in FIG. 5 , the sub-interface 512 can be divided into three parts, namely, the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c. Since the sub-interface 512 can cover the entire folding screen device in the vertical direction, it can be seen from a simple mathematical graphic relationship that the horizontal lengths of the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c are all L x , and the aspect ratio of the projection screen 512b is the aspect ratio of the Source end display screen, that is, (L x / W1)=(L source / W source ), the vertical width of the projection screen 512b can be expressed as W1=(W sink -W1-W2), combined with the above relationship, after simple algebraic conversion, we can get the following relationship:

[0132] L x / (W sink -W1-W2)=L source / W source ;

[0133] Among them, W sink , W1, W2, L source , W source are all determined values, and L x That is, the Sink end needs to determine the horizontal length of the projection window. X The specific value of can be expressed as:

[0134] L x =[(W sink -W1-W2)*L source ] / W source ;

[0135] Among them, "*" represents multiplication operation, and " / " represents division operation.

[0136] Correspondingly, the lateral length of the sub-interface 511 can be expressed as (L source -L x ), then the ratio of the horizontal lengths of the two split-screen windows can be expressed as (L source -L x ): L x , L x =[(W sink -W1-W2)*L source ] / W source Plug it in to calculate the specific ratio.

[0137] In some embodiments, in addition to the screen projection screen projected by the Source end on the Sink end, the screen projection window may also include more or fewer elements due to the personalized design of the Sink end. For example, the screen projection window may not include the window control bar and the three-button navigation bar, or may only include one of the window control bar and the three-button navigation bar, and may also include other content. When the vertical width of these elements and the aspect ratio of the Source end display screen are determined, the Sink end can determine the horizontal length of the screen projection window, i.e., the sub-interface 512, according to similar logic as above, to avoid the appearance of padding areas on both sides of the screen projection window, so as to maximize the use of the display area of ​​the Sink end display screen. In addition, in some embodiments, when the screen projection screen of the Source end is displayed in landscape mode, the Sink end can also display the originally displayed screen and the screen projection window in a split-screen manner. At this time, the electronic device can calculate the vertical width of the screen projection window by the same logic, and display the screen projection window on the Sink end according to the width. Correspondingly, the sum of the vertical widths of the screen originally displayed by the Sink end and the screen projection window is equal to the width of the Sink end display screen, and examples are not given one by one here.

[0138] In the embodiment of the present application, the sink end can be referred to as the "first device", the source end can be referred to as the "second device", the sub-interface 511 can be referred to as the "second screen", and the sub-interface 512 can be referred to as the "second screen". The vertical width W1 of the projection screen 512b is (W sink -W1-W2) can be called "first information", that is, the lateral length L of the sub-interface 512 x =[(W sink -W1-W2)*L source ] / W source Can be called "second information".

[0139] Next, combine Figure 6 The software structure of the electronic device provided by this application is introduced.

[0140] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: application layer (applications), application framework layer (application framework), system library, kernel layer (kernel) and hardware layer.

[0141] The application layer may include a series of application packages. For example, the application layer may include a desktop APP, a notepad APP, and a screen sharing module, and the present application embodiment does not impose any restrictions on this.

[0142] Applications in the application layer can be divided into system applications and non-system applications. System applications can specifically include desktop apps, gallery apps, call apps, system user interfaces (SystemUI), etc. Non-system applications can include notepad apps, map apps, shopping apps, etc.

[0143] In an embodiment of the present application, the application layer also includes an APP for screen projection function, which can respond to user operations and provide a corresponding screen projection startup interface or screen projection receiving interface to facilitate establishing a screen projection connection with other devices.

[0144] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 6 As shown, the application framework layer may include a view system, a window manager, a content provider, an activity manager, a task manager, etc., and the embodiments of the present application do not impose any restrictions on this.

[0145] In the embodiment of the present application, the application framework layer can provide a corresponding API interface for application calls of the application layer program. For example, the screen projection application APP of the application layer can call the API interface of the application framework layer to issue instructions for the user to start the screen projection process and to switch the Sink end view to the split-screen display mode.

[0146] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. The display interface can be composed of one or more views.

[0147] In an embodiment of the present application, the view system can be used to display application icons on the desktop, to display application cards on the desktop according to user operations, and to call out a function menu window according to user operations on application icons or application cards.

[0148] In the embodiment of the present application, the window manager is used to manage the display content of the function menu window of each APP. The function menu window may include common function icons of the APP and corresponding function descriptions.

[0149] The software structure may also include Android Runtime, which includes core libraries and a virtual machine and is responsible for scheduling and management of the Android system.

[0150] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0151] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0152] The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0153] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver and sensor driver.

[0154] For the sake of convenience, the figure also shows a hardware layer that interacts with the above software structure. For example, the hardware layer may include a CPU, a GPU, and a display screen.

[0155] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0156] Figure 7 A schematic structural diagram of an electronic device 100 is shown.

[0157] The electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0158] Specifically, the electronic device 100 may be the foldable device serving as a sink end in the aforementioned description.

[0159] The electronic device 100 may include a processor 110, an internal memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 2, a wireless communication module 160, a button 190, a display screen 194, etc. It is understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0160] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0161] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0162] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0163] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0164] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 120, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0165] The wireless communication function of the electronic device 100 can be implemented through the antenna 2, the wireless communication module 160, the modem processor and the baseband processor.

[0166] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0167] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0168] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 may be a folding screen.

[0169] The internal memory 120 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0170] The electronic device 100 can implement audio functions such as music playing and recording through an audio module, a speaker, a receiver, a microphone, an earphone interface, and an application processor.

[0171] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0172] During the screen projection process, the wireless communication module 160 is used to establish a screen projection channel with other devices, and further call the corresponding interface provided by the processor 110, and send a connection request signal based on the established screen projection channel, so that other devices can receive the connection request signal and find the electronic device 100. After that, both parties can perform data transmission based on the TCP protocol.

[0173] After establishing the screen projection path, the processor 110 can obtain the aspect ratio of the Source-end display screen, and determine the ratio of the split-screen window based on the acquired aspect ratio of the Source-end display screen, so that the aspect ratio of the split-screen window used to display the projected image is adapted to the aspect ratio of the Source-end display screen, and together with the GPU, draw the interface containing the two split-screen windows and output it to the display screen 194, thereby improving the utilization rate of the display screen area on the Sink end while reducing user operations.

Claims

1. A display method, characterized in that: The method is applied to a first device, the first device establishes a communication connection with a second device, the first device displays a first screen, and the second device displays a second screen, and the method includes: Receive the user's screen projection operation; In response to the screen projection operation, the first screen and the second screen are displayed in split screen on the display screen of the first device, and the sum of the length of the first screen and the length of the second screen is equal to the length of the display screen of the first device, or the sum of the width of the first screen and the width of the second screen is equal to the width of the display screen of the first device.

2. The method according to claim 1, characterized in that The application to which the first screen belongs is a focus application of the first device, and the focus application is an application on the first device that has most recently received a user operation.

3. The method according to claim 1 or 2, characterized in that: Before displaying the first picture and the second picture in split screen on a display screen of the first device, the method further includes: determining whether, when the first screen and the second screen are displayed in split screen on the first device, a length or a width of the first screen and a length or a width of the second screen are both greater than a first threshold; The displaying the first picture and the second picture in split screen on the display screen of the first device comprises: When the length or width of the first picture and the length or width of the second picture are both greater than or equal to a first threshold, the first picture and the second picture are displayed in split screens on a display screen of the first device.

4. The method according to any one of claim 3, characterized in that: The method further comprises: When the length or width of the first picture or the length or width of the second picture is less than the first threshold, the first picture is displayed in full screen on the display screen of the first device, and the second picture is displayed in the display screen of the first device as a floating window.

5. The method according to any one of claims 1 to 4, characterized in that: Before displaying the first picture and the second picture in split screen on a display screen of the first device, the method further includes: Determining whether the application to which the first screen belongs supports split-screen display; The displaying the first picture and the second picture in split screen on the display screen of the first device comprises: In a case where the application to which the first screen belongs supports split-screen display, the first screen and the second screen are displayed in split-screen on a display screen of the first device.

6. The method according to claim 5, characterized in that The method further comprises: When it is determined that the application to which the first screen belongs does not support split-screen display, the first screen is displayed in full screen on the display screen of the first device, and the second screen is displayed in the display screen of the first device in the form of a floating window.

7. The method according to any one of claims 1 to 6, characterized in that: When the first screen and the second screen are displayed on the display screen of the first device in a left-right split-screen manner, the first screen includes a window control bar and a three-button navigation bar, and before displaying the first screen and the second screen in a split-screen manner on the display screen of the first device, the method further includes: Determine first information, where the first information is a difference between a width of a display screen of the first device and a width of the window control bar and a width of the three-button navigation bar; The second information is determined based on the aspect ratio of the display screen of the second device and the first information, the second information representing the length of the first screen when the second screen is displayed in split screen on the first device, and the ratio of the length corresponding to the second information to the width corresponding to the first information is equal to the aspect ratio of the display screen of the second device.

8. The method according to any one of claims 1 to 7, characterized in that: When the first picture and the second picture are displayed on the display screen of the first device in a left-right split-screen manner, the length of the first picture is different from the length of the second picture.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors, a memory and a display screen; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 8.

10. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-8.

11. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 8.

Images