Display method and related devices

Abstract

This application provides a display method and related apparatus. By implementing this method, after the sink receives the screen projection from the source, the sink can display the original screen and the projection screen in a split-screen mode by default. The horizontal length of the projection window is determined according to the aspect ratio of the source screen, so that the aspect ratio of the projection window matches that of the source screen. This reduces the area of ​​the filling area on the left and right sides of the projection window when displaying in a split-screen mode, improves the utilization rate of the display screen area by the sink, and effectively enhances the user's browsing experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to display methods and related devices. Background Technology

[0002] With the development of computer technology, screen mirroring between different devices is now possible, enabling screen sharing. Taking a screen mirroring scenario involving mobile phones and foldable screen devices as an example, when a mobile phone screen is mirrored to a foldable screen device, a mirroring window can be displayed on the foldable screen device's screen, showing the content captured from the mobile phone.

[0003] In some situations, users will adjust the screen mirroring window and the original screen displayed on the Sink end to display in a split-screen or top-bottom split-screen configuration. However, in split-screen scenarios, the Sink end makes lower use of the screen area, which detracts from the user's browsing experience.

[0004] Therefore, for screen projection scenarios, it is necessary to study display methods that make better use of the display screen area. Summary of the Invention

[0005] The purpose of this application is to provide a display method and related apparatus. By implementing this method, after the sink receives the screen projection from the source, the sink can display the original screen and the projection screen in a split-screen mode by default. The horizontal length of the projection window is determined according to the aspect ratio of the source screen, so that the aspect ratio of the projection window is adapted to the aspect ratio of the source screen. This reduces the area of ​​the filling area on the left and right sides of the projection window when displaying in a split-screen mode, improves the utilization rate of the display screen area of ​​the sink, and effectively enhances the user's browsing experience.

[0006] The aforementioned and other objectives will be achieved through the features described in the independent claims. Further implementations are illustrated in the dependent claims, the specification, and the drawings.

[0007] In a first aspect, this application provides a display method applied to a first device, which establishes a communication connection with a second device. The first device displays a first screen, and the second device displays a second screen. The method includes: receiving a user's screen projection operation; and responding to the screen projection operation by splitting the first screen and the second screen and displaying them separately on the display screen of the first device, wherein the sum of the length of the first screen and the length of the second screen is equal to the length of the display screen of the first device, or the sum of the width of the first screen and the width of the second screen is equal to the width of the display screen of the first device.

[0008] In this method, the first device is the Sink during the screen mirroring process, and the second device is the Source. After the second screen displayed on the Source is mirrored to the first device, it is not displayed as a floating window. Instead, it is displayed on the first device along with the original first screen in a split-screen manner. When the first and second screens are displayed side-by-side, their horizontal length is equal to the horizontal length of the first device's display screen. When they are displayed vertically, their vertical width is equal to the vertical width of the first device's display screen. In other words, when the first and second screens are displayed in a split-screen manner on the first device, their total area is equal to the area of ​​the first device's display screen. There is no filling area on the first device to fill the screen (or the filling area is already sufficiently small and negligible). This improves the utilization of the display area on the Sink, effectively enhancing the user's browsing experience.

[0009] Optionally, the first device can be a foldable screen device, and the second device can be a mobile phone.

[0010] Optionally, the first screen can be the home screen (i.e., desktop) of the first device, or the application interface provided by an application installed on the first device. Similarly, the second screen can be the home screen (i.e., desktop) of the second device, or the application interface provided by an application installed on the second device.

[0011] In conjunction with the first aspect, in one possible implementation, the application to which the first screen belongs is the focus application of the first device, which is the application on the first device that has recently received a user operation.

[0012] In this embodiment, before screen mirroring, if the first device displays only one application interface (i.e., the first screen), the application to which the first screen belongs is the focus application of the first device. After establishing a screen mirroring connection with the second device, the first device can display the received second screen and the first screen in a split-screen manner. If the first device has already displayed two application interfaces (including the first screen) simultaneously in a split-screen or floating window manner, and the application to which the first screen belongs is the application that was most recently interacted with by the user on the first device, the electronic device can designate the application to which the first screen belongs as the focus application of the first device. After establishing a screen mirroring connection with the second device, the first device can display the received second screen and the first screen in a split-screen manner. Since the focus application is more likely to be the application that the user pays more attention to, after receiving the screen mirrored from the source end, the electronic device can retain only the interface corresponding to the focus application from the two application interfaces and display it in a split-screen manner with the screen mirroring window, so that the content displayed on the sink end better meets the user's expectations.

[0013] In conjunction with the first aspect, in one possible implementation, before displaying the first screen and the second screen in a split-screen manner on the display screen of the first device, the method further includes: determining whether the length or width of the first screen and the length or width of the second screen are both greater than a first threshold when the first screen and the second screen are displayed in a split-screen manner on the first device; displaying the first screen and the second screen in a split-screen manner on the display screen of the first device includes: displaying the first screen and the second screen in a split-screen manner on the display screen of the first device when the length or width of the first screen and the length or width of the second screen are both greater than or equal to the first threshold.

[0014] It should be understood that in split-screen display, if the horizontal or vertical length of the split-screen window is too small, it will affect the user's browsing experience. Therefore, in this embodiment, the first device can determine the horizontal length (when displaying horizontally and vertically) or the vertical width (when displaying vertically and horizontally) of the first screen and the second screen in split-screen display, and determine whether the horizontal length (when displaying horizontally and vertically) or the vertical width (when displaying vertically and horizontally) of the two windows after split-screen display is greater than the aforementioned first threshold.

[0015] Taking a split-screen display as an example, if the horizontal length of both the first and second screens is greater than or equal to the first threshold mentioned above, it means that the horizontal width of the two windows is sufficient to ensure the user's browsing experience while ensuring that the first and second screens fill the display screen of the first device horizontally. In this case, the first device can split the first and second screens horizontally on the first device according to the calculated window ratio (i.e., the ratio of the horizontal length of the first and second screens calculated while ensuring that there is no filling area in the interface).

[0016] Optionally, when the first screen and the second screen are displayed side-by-side, the first threshold can be 1 / 3 of the horizontal length of the first device's display screen; alternatively, when the first screen and the second screen are displayed top-bottom, the first threshold can be 1 / 3 of the vertical width of the first device's display screen.

[0017] In conjunction with the first aspect, in one possible implementation, the method further includes: if the length or width of the first screen or the length or width of the second screen is less than the first threshold, displaying the first screen in full screen on the display of the first device, and displaying the second screen in the form of a floating window on the display of the first device.

[0018] Similarly, in this embodiment, if the screen of the first device is horizontally filled while ensuring that the first and second screens are displayed side-by-side, and the length of either the first or second screen is less than the first threshold, in order to ensure the user's browsing experience, the first device may no longer display the first and second screens side-by-side on the first device, but instead display the second screen as a floating window on the first screen, while the first screen is displayed in full screen on the first device.

[0019] In conjunction with the first aspect, in one possible implementation, before displaying the first screen and the second screen in a split-screen manner on the display screen of the first device, the method further includes: determining whether the application to which the first screen belongs supports split-screen display; displaying the first screen and the second screen in a split-screen manner on the display screen of the first device includes: if the application to which the first screen belongs supports split-screen display, displaying the first screen and the second screen in a split-screen manner on the display screen of the first device according to first information.

[0020] In this embodiment, the application to which the first screen belongs is the currently focused application of the first device. It is understood that some applications on the first device may only support full-screen display and may not support split-screen display on the first device together with interfaces provided by other applications (including the second screen projected from the second device). Therefore, in this embodiment, the first device can first determine whether the currently launched focused application supports split-screen. If the focused application supports split-screen, the first device will then display the first and second screens on the first device in a split-screen manner.

[0021] In conjunction with the first aspect, in one possible implementation, the method further includes: if it is determined that the application to which the first screen belongs does not support split-screen display, displaying the first screen in full-screen mode on the display screen of the first device, and displaying the second screen as a floating window on the display screen of the first device.

[0022] Similarly, if the focus application (i.e. the application to which the first screen belongs) does not support split-screen, the first device will not display the first and second screens in a split-screen manner. Instead, the second screen will be displayed as a floating window on the first screen, while the first screen will be displayed in full screen on the first device.

[0023] In conjunction with the first aspect, in one possible implementation, when the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the first screen includes a window control bar and a three-button navigation bar. Before displaying the first screen and the second screen on the display screen of the first device, the method further includes: determining first information, which is the difference between the width of the first device display screen and the width of the window control bar and the width of the three-button navigation bar; determining second information based on the aspect ratio of the second device display screen and the first information, whereby the second information represents the length of the first screen when the second screen is displayed on the first device in a split-screen manner, and the ratio of the length corresponding to the second information to the width corresponding to the first information is equal to the aspect ratio of the second device display screen.

[0024] Understandably, in this embodiment, when the first and second images are displayed on the display screen of the first device in a split-screen manner, the vertical width of the first and second images is equal to the width of the first device's display screen. Therefore, before screen mirroring with the second device is completed, the first device, as the sink end, can obtain information including the aspect ratio (L / V) of the second device's display screen. source / W source The horizontal length L of the first device's display screen sink and vertical width W sink When the projection window includes a window control bar and a three-button navigation bar, the first device will also store the default vertical width W1 of the window control bar and the default vertical width W2 of the three-button navigation bar. From a simple mathematical relationship, it can be seen that the horizontal length of the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c is all L. x The aspect ratio of the projected screen (i.e., the area in the first screen excluding the width of the window control bar and the three-button navigation bar) is the same as the aspect ratio of the source screen. Combining the aforementioned relationship with simple algebraic conversion, we can obtain the following formula: L x / (W sink -W1-W2)=L source / W source L x This refers to the length of the first frame, which is also the first piece of information.

[0025] Correspondingly, the length of the second screen is the length of the first device's display screen minus the length of the first screen. The first device can then display the first and second screens in a left-right split manner according to the calculated length, and the first and second screens can fill the first device's display screen.

[0026] In conjunction with the first aspect, in one possible implementation, when the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the lengths of the first screen and the second screen are different.

[0027] In this embodiment, when the first screen and the second screen are displayed side-by-side on the first device, the first device does not set the horizontal width of the first screen and the second screen in an even manner. Instead, it sets the length of the display area where the second screen is located while ensuring that no filling area is added to the left and right sides of the second screen, so that the second screen can horizontally fill the display area. Therefore, when the first screen and the second screen are displayed side-by-side on the first device, the length of the first screen is different from the length of the second screen.

[0028] In a second aspect, this application provides an electronic device comprising: one or more processors and a memory; the memory being coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method of the first aspect or any possible implementation thereof.

[0029] Thirdly, this application provides a chip system applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the methods as described in the first aspect or any possible implementation thereof.

[0030] Fourthly, this application provides a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method as described in the first aspect or any possible implementation thereof.

[0031] Fifthly, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in the first aspect or any possible implementation thereof. Attached Figure Description

[0032] Figure 1 This is a schematic diagram illustrating a process of projecting a mobile phone screen onto a foldable screen device, as provided in an embodiment of this application.

[0033] Figure 2 A user interface diagram showing a screen projection screen and an application screen displayed in a split-screen manner, as provided in an embodiment of this application;

[0034] Figure 3 An interactive flowchart of a display method provided in an embodiment of this application;

[0035] Figure 4 This is a schematic diagram illustrating a scenario where the screen corresponding to the focus application is determined to be the projection screen, as provided in an embodiment of this application.

[0036] Figure 5 A schematic diagram illustrating a method for determining the split-screen window ratio provided in an embodiment of this application;

[0037] Figure 6 A software structure diagram of an electronic device provided in an embodiment of this application;

[0038] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0039] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0040] To facilitate understanding, the relevant terms involved in the embodiments of this application will be introduced below.

[0041] (1) Foldable screen devices

[0042] Foldable screen devices are smart devices that use flexible displays, characterized by screens that can change size by folding inwards or outwards, much like a book. This design allows the phone to switch between smaller and larger sizes, adapting to different usage scenarios. The most common foldable screen devices are foldable phones, which typically use OLED panels. OLED panels are self-illuminating, and when bent at a certain angle, combined with the folding mechanism of the body, they achieve the folding effect.

[0043] Folded and unfolded states are the two most basic forms of foldable screen devices. In the unfolded state, the foldable screen device has a larger screen size, allowing for more complete display of application content. Conversely, in the folded state, the screen size is smaller. In this application, when the electronic device is a foldable screen device, and it receives a projection window from another device as a sink, the electronic device can determine the display method of the projection window based on its specific form. For example, when the electronic device is in the folded state, the projection window is displayed as a floating window; when the electronic device is in the unfolded state, it can display the projection window in a split-screen manner (i.e., the projection window is displayed on one side of the screen, and the interface originally displayed by the foldable screen device is displayed on the other side).

[0044] (2) Wireless screen projection

[0045] Wireless screen mirroring is a wireless transmission technology that allows users to transmit content (such as videos, audio, and images) from mobile devices like smartphones, tablets, and laptops to other external display devices (such as TVs and tablets) via a wireless network. Wireless screen mirroring technology is typically used for screen mirroring functions, allowing users to easily share content from their devices and expand their display screen.

[0046] Screen mirroring typically involves two roles: the sender (also known as the source) and the receiver (also known as the sink). The source is the device that sends content to the display device. The sink receives and displays the content from the source. For example, when a user mirrors their phone screen to a tablet, the phone is the source, and the tablet is the sink. During mirroring, the source needs to capture, encode, and transmit the screen content over the network to obtain the mirrored data, which is then sent to the sink. After receiving the mirrored data from the source, the sink typically decodes, renders, and displays it.

[0047] For example, when a foldable screen device and a mobile phone are connected and multi-screen collaboration is enabled, the screen displayed on the mobile phone can be projected onto the foldable screen device and displayed in a small window (hereinafter referred to as the "projection window") on the foldable screen device's display.

[0048] (3) Floating window

[0049] "Floating windows" are system tools on smart devices that allow windows to float above the interface or desktop. As a shrunk application interface, a floating window can be displayed on top of other application interfaces (such as full-screen applications), while still displaying the application's inherent interface style and function list. Users can directly operate the controls in the floating window to use their corresponding functions without closing or exiting other application interfaces. For example, if you receive a message from WeChat or QQ while playing a game or watching a video in full-screen mode, the corresponding chat interface can appear as a floating window directly above the game interface. Users can view and reply to messages in the small chat window without switching or exiting the currently open application.

[0050] In this embodiment, the projection window from another device onto the electronic device can be displayed as a floating window on the screen of the electronic device by default. Users can drag the projection window to the side of the screen to display the projection window and the original interface displayed on the electronic device in a split-screen or horizontal manner. Alternatively, in some embodiments, the projection window from another device onto the electronic device can be displayed in a split-screen or horizontal manner along with the original interface displayed on the electronic device by default. The electronic device can adaptively adjust the aspect ratio of the projection window so that the projection window and the original interface displayed on the electronic device fill the entire screen, minimizing solid color fill areas on the screen, improving the utilization rate of the foldable screen area, and enhancing the user's browsing experience.

[0051] (4) Split-screen display

[0052] The device's display screen is divided into two separate screens, either side-by-side or top-bottom, without overlapping. The two screens can be the same size or different; users can drag the dividing line between them to change their sizes simultaneously.

[0053] (5) Three-button navigation bar and window control bar

[0054] The three-button navigation bar is one of the core interface elements for user interaction with smart devices. It typically includes three core buttons: a back button, a home screen button, and a history button. The home screen button responds to user clicks, allowing the electronic device to switch from any user interface to the home screen. The back button responds to user clicks, allowing the electronic device to switch from the currently displayed user interface to the previous user interface. The history button responds to user clicks, displaying the user interface for one or more recently opened tasks. Of course, the navigation buttons in the three-button navigation bar can be named in other ways; for example, the back button can be called Back Button, the home screen button can be called Home button, and the history button can be called MenuButton. This application does not impose any limitations on this. In the embodiments of this application, unless otherwise specified, all navigation buttons in the three-button navigation bar are virtual buttons.

[0055] Furthermore, some electronic devices also support users to perform the functions corresponding to each navigation button in the three-button navigation bar through specific gesture operations. For example, users can swipe inward from the left and right edges of the electronic device screen (equivalent to clicking the back button) to switch the electronic device from the currently displayed user interface to the previous user interface; users can swipe up from the bottom of the electronic device screen (equivalent to clicking the home button) to switch the electronic device from the currently displayed user interface to the home screen; users can also swipe up from the bottom of the electronic device screen and pause (equivalent to clicking the history button) to display the user interface corresponding to one or more recently opened tasks. Therefore, in some embodiments, the three-button navigation bar may not be displayed on the display screen of the electronic device. In the screen mirroring scenario, regardless of whether the three-button navigation bar is displayed on the source end's display screen, when the screen mirroring window is displayed on the sink end, the sink end can display the three-button navigation bar at the bottom of the screen mirroring window, and users can use the three-button navigation bar to switch between the screen mirroring window and the interface displayed on the source end.

[0056] The window control bar is located at the very top of the window and is used to display information such as the current application name and file name. For windows displayed in a Windows system, the window control bar typically contains multiple elements such as the program icon, file name, and window controls; however, in this embodiment, the window control bar of the projection window displayed on the Sink end may only contain multiple window controls, specifically including a minimize control for minimizing the projection window, a maximize control for maximizing the projection window to full screen on the Sink end, and a close control for closing the projection window and disconnecting the Sink end from the Source end. Furthermore, in some embodiments, the title bar of the projection window may also include a movement control for moving the projection window around the screen.

[0057] With the development of computer technology, screen mirroring between different devices is now possible, enabling screen sharing. Taking screen mirroring scenarios involving mobile phones and tablets as an example, when a mobile phone screen is mirrored to a foldable screen device, a mirroring window can be displayed on the foldable screen device's display, showing the content captured from the mobile phone.

[0058] Next, combine Figure 1 and Figure 2 This document explains the specific process of casting the screen from the Sink end to the Source end and adjusting the display on the Sink end to a split-screen display.

[0059] First, it should be noted that in the embodiments of this application and subsequent embodiments, the electronic device serving as the Sink can be a mobile phone (including foldable screen devices), tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, etc., and this application does not impose any restrictions on the specific type of the electronic device. Furthermore, other devices serving as the Source can be mobile phones, tablet computers, wearable devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, etc., and this application does not impose any restrictions on the specific type of these other devices. Specifically, the aforementioned other devices serving as the Source can project their displayed application interface onto the display screen of the electronic device serving as the Sink via WLAN or Bluetooth. For ease of explanation, the following description will use a foldable screen device as the Sink and a mobile phone as the Source as an example.

[0060] Figure 1 This shows the user interface displayed on the foldable device during the process of projecting the phone's screen onto the foldable device.

[0061] like Figure 1As shown in (A), user interface 10 is the main interface of the "Notes" application displayed on the foldable screen device after the user launches the application. At this time, no other device has yet projected its screen onto the foldable screen device.

[0062] At a certain moment, after the foldable screen device, acting as the sink, receives the screen projection from the phone, acting as the source, the sink can display something like this. Figure 1 User interface 11 is shown in (B) of the diagram.

[0063] like Figure 1 As shown in (B), the user interface 11 may include a main interface 111 and a projection window 112. The main interface 111 can be the main interface of the application "Notes" installed on the Sink. The projection window 112 displays the screen projected from the Source to the Sink. It is assumed here that the Source is currently displaying its main screen interface; therefore, the projection window 112 on the Sink also displays the Source's main screen interface.

[0064] The projection window 112 may include a window control bar 112a and a three-button navigation bar 112b. Figure 1 As can be seen in (B) of this application embodiment, after the screen projection connection is established between the Sink end and the Source end, the screen projection window 112 can be displayed in the user interface 11 by default as a floating window.

[0065] The window control bar 112a may include an expand control 2a1, a minimize control 2a2, a close control 2a3, and a move control 2a4. The expand control 2a1 responds to a user's click, causing the sink to display the projection window 112 in full screen. The minimize control 2a2 responds to a user's click, causing the sink to collapse the projection window 112 and display it as a capsule on the side of the screen. The close control 2a3 responds to a user's click, causing the sink to close the projection window 112 and disconnect the projection connection between the sink and the source. The move control 2a4 responds to a user's long-press and drag operation, allowing the sink to change the position of the projection window 112 within the user interface 11 as the user drags it.

[0066] The three-button navigation bar 112b may include a back button 2b1, a home screen button 2b2, and a history task button 2b3. Any one of these buttons can respond to user operation, switching the user interface displayed on the source end and causing the sink end to synchronously update the interface content displayed in the projection window 112. For details, please refer to the foregoing description, which will not be repeated here or in subsequent embodiments.

[0067] Understandably, the projection window displayed as a floating window on the Sink end will inevitably obscure the original interface displayed on the Sink end, causing inconvenience to the user. Therefore, in some embodiments, when the display area on the Sink end is large enough (for example, a foldable screen device in its unfolded state has a sufficiently large display area), after the Source end successfully projects the screen to the Sink end, the user can operate on the Sink end, displaying the original interface displayed on the Sink end and the projection window obtained from the Source end in a split-screen manner on the Sink end's display screen.

[0068] like Figure 1 As shown in (B), by long-pressing and dragging the motion control 2a4 on the projection window 112, the user can change the position of the projection window 112 in the user interface 11. When the user drags the projection window 112 to the edge of the Sink end display screen, the Sink end can respond to this operation and display the main interface 111 and the projection window 112 in a split-screen manner on the display screen, and display as shown in (B). Figure 1 User interface 13 is shown in (C).

[0069] like Figure 1 As shown in (C), the user interface 13 may include sub-interface 131, sub-interface 132, boundary line 133, region 134, and region 135. Wherein:

[0070] The screen displayed in sub-interface 131 corresponds to the aforementioned... Figure 1 The main interface of the "Notes" application in China 111.

[0071] The screen displayed in sub-interface 132 corresponds to the aforementioned... Figure 1 The image displayed in window 112 of the projection screen.

[0072] Understandably, due to the change in the display area, the size and shape of each element in the sub-interface 131 and the main interface 111 can be different. Similarly, the size and shape of each element in the sub-interface 132 and the projection window 112 can also be different.

[0073] The dividing line 133 is the dividing line between sub-interface 131 and sub-interface 132. In some embodiments, if the foldable screen device is a two-foldable screen device (i.e., the device screen has only one foldable part), the dividing line 133 can coincide with the foldable part by default.

[0074] Regions 134 and 135 are the filled areas on the left and right sides of sub-interface 132, respectively. Understandably, regions 134 and 135 can be solid color areas (e.g., pure black). In some embodiments, the sink end can first obtain the background image on the sink end desktop corresponding to the positions of regions 134 and 135 as the bottom layer, and then add a solid color layer with a certain degree of transparency and blur effect on this layer to obtain the image to be displayed in regions 134 and 135.

[0075] Understandably, in the case of a two-folding folding device, the fold line is generally the center line of the display. Based on the above explanation, in the case of a split-screen display, the two display areas divided by the dividing line 133 are both half the display area of ​​the folding device. For the display area to the left of the dividing line 133, the Sink end can adjust the width of the main interface of the "Notes" application so that the main interface of the "Notes" application can horizontally fill half of the Sink end's display. However, for the display area to the right of the dividing line 133, since the aspect ratio of this area may not match the aspect ratio of the Source end's display, in order to keep the layout of each element in the projection screen on the Sink end (i.e., the entire screen in sub-interface 131 excluding the top window control bar and the bottom three-button navigation bar) as consistent as possible with the Source end's display, the Sink end needs to maintain the same aspect ratio as the Source end's display. Therefore, the Sink end cannot arbitrarily stretch the projection window horizontally to fill the other half of the display area. Therefore, while ensuring that the aspect ratio of the projected screen is the same as that of the Source screen, the Sink can center the sub-interface 132 in the other half of the Sink display area, and set a filling area on the left and right sides of the sub-interface 132, namely area 134 and area 135, so that these two areas and the sub-interface 132 can fill the right half of the Sink display area together.

[0076] In combination with the above Figure 1 and Figure 2As explained, after the screen mirroring process is complete, the mirroring window appears as a floating window on the Sink device, thus obscuring the original interface displayed on the Sink device. Even after the user adjusts the display mode of the Sink device to split-screen, the utilization rate of the Sink device's screen area remains low due to the mismatch between the aspect ratio of the split-screen window and the aspect ratio of the Source device. Areas 134 and 135 do not display the content the user actually needs to browse (these two areas, as filler areas, are neither used to display the interface content of the "Notes" application nor the content mirrored from the Source device to the Sink). Therefore, the existence of areas 134 and 135 reduces the utilization rate of the Sink device's screen area to some extent and also impairs the user's browsing experience. Especially when the Sink device is a foldable screen device, users use foldable screen devices to obtain a better browsing experience through a larger screen; the existence of areas 134 and 135 actually contradicts the user's original intention for using a foldable screen device.

[0077] To address the shortcomings of the aforementioned methods, this application provides a display method. Implementing this method, after receiving the screen projection from the source end, the sink end can, by default, display the original screen and the projection screen in a split-screen mode. The horizontal length of the projection window is determined based on the aspect ratio of the source end's display screen, ensuring that the aspect ratio of the projection window matches that of the source end's display screen. This reduces the area of ​​the filling area on the left and right sides of the projection window during split-screen display, improves the utilization rate of the display screen area on the sink end, and effectively enhances the user's browsing experience.

[0078] Figure 2 An example is shown of the user interface displayed on the Sink end after the above display method has been executed.

[0079] like Figure 2As shown, the user interface 21 may include sub-interfaces 211, 212, and a dividing line 213. Sub-interfaces 211, 212, and 213 are respectively the sub-interfaces 211, 212, and 213 within the user interface 21. Unlike user interface 13, after the Sink end determines the horizontal length of the projection window based on the aspect ratio of the Source end's display, the area of ​​the display area to the right of the dividing line 213 decreases. When the horizontal length of the right-side display area is approximately the same as the horizontal length of the sub-interface 212, the sub-interface 212 can essentially fill the entire display area. At this point, the Sink end no longer needs to display filling areas on the left and right sides of the sub-interface 212 (it can be considered that the horizontal length of the filling areas on both sides of the sub-interface 212 has been reduced to a very small value, therefore these two filling areas can be ignored, and the sum of the horizontal lengths of the sub-interfaces 211 and 212 equals the length of the Sink end's display). At this time, the display area to the left of the dividing line 213 can be used entirely to display the main interface of the application "Notes", and the display area to the right of the dividing line 213 can be used entirely to display the projection window.

[0080] In this embodiment of the application, the Sink end can be referred to as the "first device", the Source end can be referred to as the "second device", the sub-interface 211 can be referred to as the "first screen", and the sub-interface 212 can be referred to as the "second screen".

[0081] The interaction flow of the display method provided in this application will be described below. In the display method provided in this application, the sink can be a foldable screen device, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, etc., and this application does not impose any restrictions on the specific type of the electronic device. Furthermore, other devices that project the screen onto the aforementioned sink as the source can be mobile phones, tablet computers, wearable devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, etc., and this application does not impose any restrictions on the specific type of the source. For ease of explanation, the following description will use a foldable screen device as the sink and a mobile phone as the source. Please refer to [link to relevant documentation] for details. Figure 3 .

[0082] like Figure 3As shown, the Sink terminal provided in this application may include a first screen mirroring service module (PCAssistant1), a screen mirroring business module (AssociateAssistant), an activity management module (AMS / ATMS), and a window management module. The Sink terminal may also include a second screen mirroring service module (PCAssistant2). The display method provided in this application may include, but is not limited to, the following steps:

[0083] S101, The Source end receives the user's screen mirroring operation.

[0084] With Wi-Fi enabled on both the Source and Sink ends, the user can perform screen mirroring operations on the Source end (e.g., enabling the screen mirroring function in the "Settings" app) to cause the Source end to periodically send request signals. Once the Sink end successfully detects the request signal sent by the Source end, it means that the Sink end has discovered the Source end, and a communication connection can be established between the Sink end and the Source end.

[0085] In some embodiments, users can also perform screen mirroring on the Sink end, so that the Sink end can periodically send request signals outward. When the Source end successfully detects the request signal sent by the Sink end, a communication connection can be established between the Sink end and the Source end.

[0086] S102. The Source end sends a screen mirroring start command to the screen mirroring service module on the Sink end.

[0087] After the connection is established between the Source and Sink ends, the Source end can send the screen casting start command to the screen casting service module on the Sink end. It should be understood that this screen casting start command is a pre-start command. Before the Sink end officially displays the screen casting window, this command can be used to control the Sink end to complete a series of preparatory work for displaying the screen casting window, including determining the display method of the screen casting window on the Sink end (e.g., displaying it as a floating window or in a split-screen mode), the shape and size of the screen casting window, and determining the specific style of the screen casting window (including whether to add a window control bar, a three-button navigation bar, and the specific styles of the window control bar and the three-button navigation bar), etc.

[0088] S103, The second projection service module on the Source end sends the aspect ratio of the display screen on the Source end to the first projection service module on the Sink end.

[0089] S104. The first screen mirroring service module on the Sink end sends the aspect ratio of the display screen on the Source end to the screen mirroring service module on the Sink end.

[0090] Once the Source sends the screen casting start command to the Sink-side screen casting service module, the Sink-side screen casting service module can respond to the command and request the aspect ratio of the Source's display screen from the second screen casting service module on the Source side. Accordingly, the second screen casting service module on the Source side will respond to the request and send the aspect ratio of the Source's display screen to the Sink-side screen casting service module, and the first screen casting service module on the Sink side will further send the aspect ratio of the Source's display screen to the Sink-side screen casting service module.

[0091] S105. The first screen mirroring service module on the Sink end sends an indication message to the screen mirroring service module on the Sink end, indicating whether the currently launched focus application supports split-screen.

[0092] Understandably, in this application, the Sink terminal already displays the application interface of a certain application. This application interface can be the application interface provided by a third-party application installed on the Sink terminal, such as an application... The main interface can be the application interface provided by a local application on the Sink side, such as the main interface of the "Notes" application or the main interface provided by the "Desktop / Settings" application (i.e., the Sink's desktop). However, some applications on the Sink side may only support full-screen display and not split-screen display on the Sink side along with interfaces provided by other applications (including the screen projected from the Source side to the Sink side). Therefore, in this embodiment, the Sink side can first determine whether the currently launched focused application supports split-screen through the first projection service module. If the focused application supports split-screen, the Sink side will then display its original screen and the screen projected from the Source side in a split-screen manner on the Sink side.

[0093] It should be noted that the aforementioned focused application is the application that the user last interacted with on the Sink. When the Sink currently displays only one application interface, the focused application is the application to which that interface belongs. When the Sink currently displays multiple application interfaces (these multiple application interfaces are displayed in a split-screen manner, and these multiple interfaces can belong to the same application or different applications), the Sink can determine the application to which the last interface that received user interaction among the multiple pages belongs as the focused application. This will be explained in more detail later, and will not be elaborated on here.

[0094] Optionally, in some embodiments, the Sink end can first execute step S105, and after determining that the current focus application of the Sink end supports split-screen display, execute steps S103-S104 to obtain the aspect ratio of the Source end display screen; if it is determined that the current focus application of the Sink end does not support split-screen display, the Sink end can skip steps S103-S104 and directly execute step S106.

[0095] S106, the screen mirroring module on the Sink end calculates the ratio of the split-screen window based on the aspect ratio of the Source end display screen.

[0096] Specifically, the first screen mirroring service module on the Sink end can calculate the ratio of the split-screen window based on the aspect ratio of the Source end's display and the Sink end's settings for the screen mirroring window. When the original screen displayed on the Sink end and the screen mirroring window are displayed in split-screen mode according to the calculated ratio, there may be no fill area on both sides of the screen mirroring window (or the fill area on both sides of the screen mirroring window is very small enough that it can be ignored). The settings for the screen mirroring window mainly include factors that affect the aspect ratio of the screen mirroring window, such as whether the screen mirroring window includes a window control bar and whether it includes a three-button navigation bar. Understandably, the default settings for the screen mirroring window are set at the factory on the Sink end, so these parameters are pre-stored in the Sink end at the factory.

[0097] Specifically, when the split-screen window is split into two horizontal sections, the ratio of the split-screen window is the ratio of the horizontal lengths of the two windows.

[0098] For details on how the Sink calculates the ratio of the split-screen window based on the aspect ratio of the Source display, please refer to the relevant descriptions in the subsequent embodiments. They will not be repeated here.

[0099] S107. The activity management module on the Sink end sends the window threshold of the split-screen window to the screen casting service module.

[0100] Optionally, the aforementioned ratio threshold can be 1 / 3 of the horizontal length of the Sink-end display screen, or other values, which are not limited in this application.

[0101] It should be understood that in split-screen display, if the horizontal or vertical length of the split-screen window is too small, it will affect the user's browsing experience. Therefore, in this embodiment, the activity management module on the Sink end sends the window threshold of the split-screen window to the screen casting service module; after the Sink end calculates the ratio of the split-screen window, the screen casting service module determines whether the screen originally displayed on the Sink end and the screen casting window can be split and displayed according to the calculated ratio based on the calculated ratio and the obtained window threshold.

[0102] Specifically, the horizontal lengths of the two windows after split-screen display (assuming the above window threshold is the horizontal length threshold, and the calculated split-screen window ratio is also the ratio of the horizontal lengths of the two windows) are both greater than the above window threshold. If the horizontal lengths of the two windows after split-screen display are both greater than or equal to the above window threshold, it means that after splitting the original screen displayed on the Sink end and the projection window according to the calculated ratio, the horizontal width of the two windows is sufficient to ensure the user's browsing experience, and the Sink end can split the original screen displayed on the Sink end and the projection window according to the calculated ratio; if the horizontal length of either window after split-screen display is less than the above window threshold, it means that the Sink cannot split the original screen displayed on the Sink end and the projection window according to the calculated ratio.

[0103] Taking the aforementioned window threshold of 1 / 3 of the horizontal length of the Sink display as an example, assuming the screen mirroring module calculates a split-screen window ratio of 1:1, this means that when the original display of the Sink and the screen mirroring window are split-screened according to this ratio, the horizontal length of both windows is 1 / 2 of the horizontal length of the display, which is greater than 1 / 3. Therefore, the Sink can split-screen the original display of the Sink and the screen mirroring window according to a 1:1 ratio. Conversely, assuming the aforementioned screen mirroring module calculates a split-screen window ratio of 1:3, this means that when the original display of the Sink and the screen mirroring window are split-screened according to this ratio, the horizontal length of one window is 1 / 4 of the horizontal length of the display, and the horizontal length of the other window is 3 / 4 of the horizontal length of the display. Since 1 / 4 is less than 1 / 3, the Sink cannot split-screen the original display of the Sink and the screen mirroring window according to a 1:3 ratio.

[0104] Configure startup items for the screen mirroring service module on the S108 and Sink ends.

[0105] The Sink end configures the startup items of the projection window based on two pieces of information: whether the previously confirmed focus application supports split-screen and whether it can split the screen originally displayed on the Sink end and the projection window according to the calculated ratio. The above configuration items may specifically include the display method of the projection window and parameters such as the size and style of the projection window.

[0106] Specifically, if the featured application supports split-screen and the sink can display the original screen and the projection window in a split-screen manner according to the calculated ratio, then the projection window in the above configuration items will be displayed in a split-screen manner with the original screen displayed on the sink. If the featured application does not support split-screen and / or the sink cannot display the original screen and the projection window in a split-screen manner according to the calculated ratio, then the projection window in the above configuration items will be displayed as a floating window on the original screen displayed on the sink.

[0107] Optionally, in some embodiments, after the screen mirroring module on the Sink side calculates the ratio of the split-screen window, it can first send the ratio to the activity management module. The activity management module then determines whether the currently launched application matches the obtained configuration items and window ratio, and returns the determination result to the screen mirroring module. Afterward, the screen mirroring module can configure the launch items of the screen mirroring window based on the determination result and whether the focused application on the Sink side supports split-screen.

[0108] S109. The screen mirroring service module on the Sink end sends configuration items and window ratio to the activity management module on the Sink end.

[0109] The S110 and Sink end screen mirroring service modules send window start commands to the Sink end activity management module.

[0110] S111: The activity management module on the Sink end sends an adaptive window startup command to the window management module on the Sink end.

[0111] After the screen mirroring service module on the Sink side completes the configuration and startup items, it sends the configuration items and the calculated window ratio to the activity management module on the Sink side, and also sends a window startup command (startActivity) to the activity management module. The activity management module further processes this request, sending an adaptive window startup command to the window management module on the Sink side. The window management module is primarily responsible for creating and managing the screen mirroring window and controlling its display position on the Sink side. Specifically, the adaptive window startup command may include information such as the aspect ratio and style of the screen mirroring window. The window management module can use this information to control the layout of the screen mirroring window on the Sink side, determining the window size, position, and hierarchical relationships between layers.

[0112] S112. The window management module on the Sink end sends callback parameters to the screen mirroring service module on the Sink end.

[0113] Adjust the size of the screen mirroring service module on the S113 and Sink ends.

[0114] The window management module on the sink side sends callback parameters to the screen mirroring service module on the sink side. The screen mirroring service module on the sink side can then use the callback parameters returned by the window management module, along with the hardware on the sink side including the CPU and GPU, to render the display interface. Finally, the rendered interface (see reference for details) is displayed. Figure 2 The user interface 21 shown is output to the display screen at the Sink end.

[0115] In this embodiment of the application, the Sink end can be referred to as the "first device", the Source end can be referred to as the "second device", and the window threshold can be referred to as the "first threshold".

[0116] As explained above, in this application, the Sink determines the display method of the projection window based on whether the focused application supports split-screen display. Therefore, when the Sink itself is already displaying two application interfaces (which may belong to the same application or different applications), since the focused application is more likely to be the application that the user is more interested in, after receiving the projection from the Source, the electronic device can retain only the interface corresponding to the focused application from the two application interfaces and display it in a split-screen format with the projection window. This makes the content displayed on the Sink more in line with the user's expectations. The following will further explain... Figure 4 Please provide an explanation.

[0117] like Figure 4 As shown in (A), the foldable screen device displays a user interface 41, which may include sub-interfaces 411 and 412, and a dividing line 413. The sub-interface 411 displays the application. The main interface is displayed in the sub-interface 412, which is the main interface of the application "Notes". The dividing line 413 is the dividing line between the sub-interface 411 and the sub-interface 412. At this time, the sub-interface 411 and the sub-interface 412 are displayed in the user interface 41 in a split-screen manner.

[0118] Both sub-interfaces 411 and 412 include movement controls for moving the corresponding sub-interfaces. Furthermore, in this embodiment, each movement control on each sub-interface can correspond to two different states: a selected state and an unselected state, which can indicate whether the application corresponding to the interface is the focus application.

[0119] refer to Figure 4 In (A), the movable control 411a on the sub-interface 411 is in an unselected state (i.e., the fill color of the movable control 411a is white), indicating that the user's last operation on the foldable screen device was not performed on the sub-interface 411, that is, the application corresponding to the sub-interface 411... At this moment, the application is not the focus on the foldable screen device; however, the movable control 412a on sub-interface 412 is in an unselected state (i.e., the fill color of movable control 412a is black), indicating that the user's last operation on the foldable screen device was not performed on sub-interface 411. Therefore, the application "Notes" corresponding to sub-interface 411 is currently the focus on the foldable screen device. Understandably, if the user subsequently performs a user operation on sub-interface 411, such as a click, the focus on the foldable screen device will switch from "Notes" to [other application]. Correspondingly, the movement control 412a on sub-interface 412 will switch to the selected state, while the movement control 412a on sub-interface 411 will switch to the unselected state.

[0120] With the focus of foldable screen devices being "notes," at a certain moment, Figure 4 After the foldable screen device, acting as the sink, receives the screen projection from the mobile phone, which acts as the source, the foldable screen device can display, as shown in the image above. Figure 4 User interface 42 is shown in (B) of the diagram.

[0121] like Figure 4 As shown in (B), the user interface 41 may include sub-interface 421, sub-interface 412, and dividing line 423. Sub-interface 421 is the projection window, displaying the image projected from the Source end onto the Sink. Sub-interface 412 is the aforementioned... Figure 4 The sub-interface 412 of (A) displays the main interface of the application "Notes". The dividing line 423 is the dividing line between sub-interface 411 and sub-interface 412. At this time, sub-interface 421 and sub-interface 412 are in the user interface 42 in a split-screen manner.

[0122] As explained above, since the focused application on the Sink end is "Notes" before screen mirroring, the electronic device can retain the sub-interface 412 in the user interface 41 and display the application after screen mirroring with the Source end. When moved to the background, it will not be displayed on the screen. The application interface.

[0123] Accordingly, in this application, the foldable screen device will adaptively adjust the horizontal length of the projection window (i.e., sub-interface 421) to avoid filling areas on both sides and improve the utilization rate of the display area on the sink end. Therefore, compared with the sub-interface 412 in the user interface 41, the horizontal length of the sub-interface 412 in the user interface 42 may also change accordingly.

[0124] Optionally, after screen mirroring is completed, the electronic device will still keep the focus on the Sink end as "Notes". Therefore, in the user interface 42, the motion control 412a on the sub-interface 412 is selected, while the motion control 412a on the sub-interface 421 is unselected.

[0125] Optionally, in some embodiments, before and after screen mirroring is completed, the sink can keep the sub-interface corresponding to the focused application displayed on the same side of the dividing line. For example... Figure 4 As shown in (A) and (B), before screen mirroring is completed, sub-interface 412 is displayed to the right of boundary line 413. After screen mirroring is completed, the sink end can still display sub-interface 412 to the right of boundary line 423, and display the screen mirroring window (i.e., sub-interface 421) to the left of boundary line 423. Optionally, in some embodiments, the sink end can display the screen mirroring window on one side of the boundary line by default. Assuming that the sink end can display the screen mirroring window to the right of the boundary line by default, even if sub-interface 412 was originally displayed to the right of boundary line 413 before screen mirroring is completed, after screen mirroring is completed, the sink end can display the screen mirroring window (i.e., sub-interface 421) to the right of boundary line 423, and move sub-interface 412 to the left of boundary line 423 for display.

[0126] Optionally, in some embodiments, before screen mirroring is completed, one application interface displayed on the Sink end can be displayed in full screen, while the other is displayed as a floating window. After screen mirroring is completed, the Sink end will still retain the interface of the focused application from both application interfaces and display it together with the mirroring window on the Sink end's display screen in a split-screen manner.

[0127] In this embodiment, the Sink end can be referred to as the "first device," the Source end can be referred to as the "second device," the sub-interface 412 can be referred to as the "first screen," and the sub-interface 421 can be referred to as the "second screen." When the user interface 41 is displayed on the Sink end, the application corresponding to the sub-interface 411... It can be called a background application, and the application "Notes" corresponding to sub-interface 412 can be called the "focus application".

[0128] Next, combine Figure 5 The specific method by which the Sink determines the split-screen window ratio based on the aspect ratio of the Source's display screen is explained. Since the projection window generally includes a window control bar and a three-button navigation bar, this embodiment of the application uses a scenario where the projection window includes both a window control bar and a three-button navigation bar as an example for explanation.

[0129] like Figure 5As shown in (A), after the foldable screen device, acting as the sink, receives the screen projection from the mobile phone, acting as the source, the foldable screen device displays the user interface 51. The user interface 51 may include sub-interfaces 511, 512, and a dividing line 513. Sub-interface 512 is the projection window, displaying the screen projected from the source onto the sink. Sub-interface 511 is the application interface of the "Notes" application. The dividing line 513 is the dividing line between sub-interfaces 511 and 512. At this time, sub-interfaces 5111 and 512 are displayed in the user interface 51 in a split-screen manner, and the foldable screen device has determined the horizontal length of the projection window (i.e., sub-interface 512) according to the aspect ratio of the source's display screen to avoid filling areas on both sides.

[0130] Understandably, before completing screen mirroring with the source, the foldable screen device, acting as the sink, can obtain information including the aspect ratio (L / W) of the source display. source / W source The horizontal length L of the Sink-end display screen sink and vertical width W sink When the projection window includes a window control bar and a three-button navigation bar, the sink will also store the default vertical width W1 of the window control bar and the default vertical width W2 of the three-button navigation bar. The main purpose of the sink is to calculate the horizontal length L of the projection window (i.e., sub-interface 512) based on the information obtained above. x .

[0131] like Figure 5 As shown in (B), the sub-interface 512 can be divided into three parts: the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c. Since the sub-interface 512 can fill the entire foldable screen device vertically, it can be seen from simple mathematical relationships that the horizontal length of the window control bar 512a, the projection screen 512b, and the three-button navigation bar 512c is all L. x The aspect ratio of the projected image 512b is the same as the aspect ratio of the source display screen, i.e. (L... x / W1)=(L source / W source The vertical width of the projected screen 512b can be expressed as W1 = (W sink -W1-W2), combined with the aforementioned relationship and through simple algebraic transformation, we can obtain the following relation:

[0132] L x / (W sink -W1-W2)=L source / W source ;

[0133] Among them, W sink W1, W2, L source W source All of these are predetermined values, while L x This means that the horizontal length of the projection window needs to be determined on the Sink side. Therefore, L X The specific value can be expressed as:

[0134] L x =[(W sink -W1-W2)*L source ] / W source ;

[0135] In this context, "*" represents multiplication and " / " represents division.

[0136] Correspondingly, the horizontal length of sub-interface 511 can be expressed as (L source -L x If the horizontal lengths of the two split-screen windows are equal, then the ratio of their horizontal lengths can be expressed as (L...). source -L x ):L x , will L x =[(W sink -W1-W2)*L source ] / W source You can calculate the specific ratio by substituting the values.

[0137] In some embodiments, besides the screen projected from the Source to the Sink, the projection window may contain more or fewer elements depending on the Sink's personalized design. For example, the projection window may not include a window control bar or a three-button navigation bar, or may only include one of the window control bar and the three-button navigation bar, or may include other content. Given that the vertical width of these elements and the aspect ratio of the Source's display screen are determined, the Sink can determine the horizontal length of the projection window (i.e., the sub-interface 512) using similar logic to avoid filling areas on both sides of the projection window, thus maximizing the utilization of the Sink's display area. Furthermore, in some embodiments, when the screen projected from the Source is displayed in landscape mode, the Sink can also display the original screen and the projection window in a split-screen manner. In this case, the electronic device can calculate the vertical width of the projection window using the same logic and display the projection window on the Sink according to that width. Correspondingly, the sum of the vertical widths of the original screen displayed on the Sink and the projection window is equal to the width of the Sink's display screen. Further examples will not be provided here.

[0138] In this embodiment, the Sink end can be referred to as the "first device," the Source end can be referred to as the "second device," the sub-interface 511 can be referred to as the "second screen," and the sub-interface 512 can be referred to as the "second screen." The vertical width W1 of the projection screen 512b is = (W sink -W1-W2) can be referred to as "first information", that is, the horizontal length L of sub-interface 512. x =[(W sink -W1-W2)*L source ] / W source It can be called "second information".

[0139] Next, combine Figure 6 The software structure of the electronic device provided in this application is described.

[0140] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: applications, application framework, system libraries, kernel, and hardware.

[0141] The application layer may include a series of application packages. For example, the application layer may include a desktop app, a notepad app, and a screen sharing module; however, this application embodiment does not impose any limitations on this.

[0142] Applications in the application layer can be divided into system applications and non-system applications. System applications can include desktop apps, gallery apps, call apps, system user interfaces (SystemUI), etc., while non-system applications can include notepad apps, map apps, shopping apps, etc.

[0143] In this embodiment of the application, the application layer also includes a screen mirroring app, which can respond to user operations and provide a corresponding screen mirroring startup interface or screen mirroring receiving interface to facilitate the establishment of screen mirroring connections with other devices.

[0144] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 6 As shown, the application framework layer may include a view system, window manager, content provider, activity manager, task manager, etc., and this application embodiment does not impose any limitations on this.

[0145] In this embodiment, the application framework layer can provide corresponding API interfaces for application layer programs to call. For example, the screen mirroring application (APP) in the application layer can call the API interface of the application framework layer to issue instructions to the user to start the screen mirroring process and to switch the Sink view to split-screen display mode.

[0146] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views.

[0147] In this embodiment of the application, the view system can be used to display application icons on the desktop, and to display application cards on the desktop according to user operations, and to bring up a function menu window according to user operations on application icons or application cards.

[0148] In this embodiment, the window manager is used to manage the content displayed in the function menu windows of each app. The function menu window may include commonly used function icons of the app and their corresponding function descriptions.

[0149] The software architecture may also include the Android Runtime. The Android Runtime consists of core libraries and a virtual machine, and is responsible for the scheduling and management of the Android system.

[0150] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0151] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0152] The system library can include multiple functional modules. Examples include: a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL). The surface manager manages the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0153] The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers and sensor drivers.

[0154] For clarity, the diagram also illustrates the hardware layer that interacts with the aforementioned software architecture. For example, the hardware layer may include a CPU, GPU, and display screen.

[0155] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0156] Figure 7 A schematic diagram of the structure of the electronic device 100 is shown.

[0157] Electronic device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose any special restrictions on the specific type of electronic device.

[0158] Specifically, the electronic device 100 can be a foldable device as the sink end as described above.

[0159] Electronic device 100 may include a processor 110, internal memory 120, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 2, wireless communication module 160, buttons 190, display screen 194, etc. It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on electronic device 100. In other embodiments of this application, electronic device 100 may include more or fewer components than illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0160] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0161] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0162] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0163] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0164] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 120, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0165] The wireless communication function of the electronic device 100 can be realized through the antenna 2, the wireless communication module 160, the modem processor, and the baseband processor.

[0166] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0167] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0168] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1. Display screen 194 may be a foldable screen.

[0169] Internal memory 120 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0170] Electronic device 100 can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors, such as music playback and recording.

[0171] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0172] During the screen mirroring process, the wireless communication module 160 establishes a screen mirroring path with other devices and further calls the corresponding interface provided by the processor 110 to send a connection request signal based on the established screen mirroring path. This allows other devices to receive the connection request signal and discover the electronic device 100. Afterward, both parties can transmit data based on the TCP protocol.

[0173] After establishing the projection path, the processor 110 can obtain the aspect ratio of the Source display screen and determine the ratio of the split-screen window based on the obtained aspect ratio of the Source display screen, so that the aspect ratio of the split-screen window used to display the projection screen is adapted to the aspect ratio of the Source display screen. Together with the GPU, the interface containing the two split-screen windows is drawn and output to the display screen 194, thereby improving the utilization rate of the display screen area on the Sink end while reducing user operations.

Claims

1. A display method, characterized in that, The method is applied to a first device, which establishes a communication connection with a second device. The first device displays a first screen, and the second device displays a second screen. When the first screen and the second screen are displayed on the display screen of the first device in a left-right split-screen manner, the first screen includes a window control bar and a three-button navigation bar. The method includes: Receive screen mirroring requests from users; In response to the screen mirroring operation, first information is determined, which is the difference between the width of the display screen of the first device and the width of the window control bar and the width of the three-button navigation bar; The second information is determined based on the aspect ratio of the display screen of the second device and the first information. The second information represents the length of the first screen when the second screen is displayed in split screen on the first device. The ratio of the length corresponding to the second information to the width corresponding to the first information is equal to the aspect ratio of the display screen of the second device. The first screen and the second screen are displayed in a split-screen format on the display screen of the first device. The sum of the length of the first screen and the length of the second screen is equal to the length of the display screen of the first device, or the sum of the width of the first screen and the width of the second screen is equal to the width of the display screen of the first device.

2. The method according to claim 1, characterized in that, The application to which the first screen belongs is the focus application of the first device, and the focus application is the application on the first device that has recently received a user operation.

3. The method according to claim 1 or 2, characterized in that, Before displaying the first screen and the second screen separately on the display screen of the first device, the method further includes: When the first screen and the second screen are displayed in split screen on the first device, it is determined whether the length or width of the first screen and the length or width of the second screen are both greater than a first threshold. The step of displaying the first screen and the second screen in a split-screen format on the display screen of the first device includes: If the length or width of the first screen and the length or width of the second screen are both greater than or equal to a first threshold, the first screen and the second screen are displayed in a split-screen format on the display screen of the first device.

4. The method according to claim 3, characterized in that, The method further includes: If the length or width of the first screen or the length or width of the second screen is less than the first threshold, the first screen is displayed in full screen on the display screen of the first device, and the second screen is displayed in the form of a floating window on the display screen of the first device.

5. The method according to claim 1 or 2, characterized in that, Before displaying the first screen and the second screen separately on the display screen of the first device, the method further includes: Determine whether the application to which the first screen belongs supports split-screen display; The step of displaying the first screen and the second screen in a split-screen format on the display screen of the first device includes: If the application to which the first screen belongs supports split-screen display, the first screen and the second screen are displayed in a split-screen manner on the display screen of the first device.

6. The method according to claim 3, characterized in that, Before displaying the first screen and the second screen separately on the display screen of the first device, the method further includes: Determine whether the application to which the first screen belongs supports split-screen display; The step of displaying the first screen and the second screen in a split-screen format on the display screen of the first device includes: If the application to which the first screen belongs supports split-screen display, the first screen and the second screen are displayed in a split-screen manner on the display screen of the first device.

7. The method according to claim 4, characterized in that, Before displaying the first screen and the second screen separately on the display screen of the first device, the method further includes: Determine whether the application to which the first screen belongs supports split-screen display; The step of displaying the first screen and the second screen in a split-screen format on the display screen of the first device includes: If the application to which the first screen belongs supports split-screen display, the first screen and the second screen are displayed in a split-screen manner on the display screen of the first device.

8. The method according to claim 5, characterized in that, The method further includes: If it is determined that the application to which the first screen belongs does not support split-screen display, the first screen is displayed in full screen on the display screen of the first device, and the second screen is displayed in the form of a floating window on the display screen of the first device.

9. The method according to claim 1 or 2, characterized in that, When the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the length of the first screen is different from the length of the second screen.

10. The method according to claim 3, characterized in that, When the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the length of the first screen is different from the length of the second screen.

11. The method according to claim 4, characterized in that, When the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the length of the first screen is different from the length of the second screen.

12. The method according to claim 5, characterized in that, When the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the length of the first screen is different from the length of the second screen.

13. The method according to claim 8, characterized in that, When the first screen and the second screen are displayed on the display screen of the first device in a split-screen manner, the length of the first screen is different from the length of the second screen.

14. An electronic device, characterized in that, The electronic device includes: one or more processors, memory, and a display screen; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-13.

15. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-13.

16. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-13.

Images