A method, electronic device and system for tiled display

Abstract

The application provides a method, an electronic device and a system for splicing display, the method comprising: a first electronic device displaying a first interface; a second electronic device sending first indication information to the first electronic device in response to detecting a first input of a user, the first indication information being used to indicate that the first electronic device and the second electronic device perform splicing display; the first electronic device displaying first partial image information and sending second partial image information to the second electronic device in response to receiving the first indication information, the first partial image information and the second partial image information being associated with the first interface according to position information of the first electronic device and the second electronic device; and the second electronic device displaying the second partial image information in response to receiving the second partial image information. The application can splice and display the screens of multiple electronic devices, which helps to improve the experience of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the terminal field, and more particularly, to a spliced display method, an electronic device and a system. BACKGROUND

[0002] The number of electronic devices owned by current users is increasing, and a survey shows that an average user owns 7 electronic devices. When a user uses a single electronic device (for example, a mobile phone) to watch a video or a document, the user is limited by the screen size, resulting in a poor viewing effect. Therefore, how to interconnect multiple electronic devices to form a whole so that the user can watch on a larger screen has become a problem to be solved. SUMMARY

[0003] The present application provides a spliced display method, an electronic device and a system, which can splice and display the screens of multiple electronic devices, and help improve the user experience.

[0004] In a first aspect, a system is provided, which includes a first electronic device and a second electronic device. The first electronic device communicates with the second electronic device through a close-range wireless connection. The first electronic device is configured to display a first interface. The second electronic device is configured to send first indication information to the first electronic device in response to detecting a first input of a user, the first indication information being used to instruct the first electronic device and the second electronic device to perform spliced display. The first electronic device is further configured to display first partial image information and send second partial image information to the second electronic device in response to receiving the first indication information, the first partial image information and the second partial image information being associated with the first interface, according to position information of the first electronic device and the second electronic device. The second electronic device is further configured to display the second partial image information in response to receiving the second partial image information.

[0005] In the embodiments of the present application, the first input of the user is detected on the second electronic device, which triggers the first electronic device and the second electronic device to perform spliced display, so that the screens of multiple electronic devices are spliced, which helps improve the user's viewing experience and thus improves the user's experience.

[0006] In some possible implementations, the first input is an operation of the user swiping from left to right. The second electronic device determines the device on the left side of the second electronic device as the source device in response to detecting the first input of the user.

[0007] In some possible implementations, if the left side of the second electronic device includes multiple devices, the second electronic device can determine the device on the leftmost side of the multiple devices as the source device.

[0008] In some possible implementation manners, a system is provided, which includes a first electronic device and a second electronic device, the first electronic device is in communication with the second electronic device through a short-range wireless connection, the first electronic device is configured to display a first interface; the first electronic device is configured to, in response to detecting an input of a user, display first partial image information and send second partial image information to the second electronic device according to position information of the first electronic device and the second electronic device, the first partial image information and the second partial image information are associated with the first interface; and the second electronic device is further configured to, in response to receiving the second partial image information, display the second partial image information.

[0009] In the embodiments of the present application, the first electronic device and the second electronic device are triggered to perform splicing display through detecting the input of the user on the first electronic device, so that the screens of multiple electronic devices are spliced, which helps to improve the user's visual experience and thus improves the user's experience.

[0010] With reference to the first aspect, in some possible implementation manners of the first aspect, the first electronic device is further configured to receive first information sent by the second electronic device, the first information being used to indicate the size of the display screen of the second electronic device; and the first electronic device is specifically configured to: according to the number of electronic devices to be spliced, the size of the display screen of the first electronic device, and the size of the display screen of the second electronic device, expand and split the image information corresponding to the first interface to obtain the first partial image information and the second partial image information.

[0011] In the embodiments of the present application, the first electronic device can perform splicing display with the second electronic device in a full-screen mode after receiving the first indication information sent by the second electronic device. For the scenario of watching videos or photos, the screen splicing of multiple electronic devices can improve the user's visual experience of watching videos or pictures, and thus improves the user's experience.

[0012] With reference to the first aspect, in some possible implementation manners of the first aspect, the size of the display screen of the first electronic device in a first direction is a first size, the size of the first electronic device in a second direction is a second size, the size of the display screen of the second electronic device in the first direction is a third size, the first direction and the second direction are perpendicular, and the first electronic device is specifically configured to: when the first electronic device and the second electronic device perform splicing display along the first direction, the first electronic device expands the size of the image information corresponding to the first interface in the first direction to a fourth size and keeps the size of the image information corresponding to the first interface in the second direction unchanged, the fourth size being the sum of the first size and the third size.

[0013] With reference to the first aspect, in some implementations of the first aspect, the enlarged image information is further divided into third image information, the third image information being located between the first image information and the second image information, and the third image information being determined by the first electronic device according to a distance between the first electronic device and the second electronic device.

[0014] In the embodiments of the present application, the first electronic device can select a division mode or a mask mode to divide the enlarged image information. In some possible implementations, when the distance between the first electronic device and the second electronic device is less than or equal to a preset distance (for example, 3 cm), the first electronic device can perform splicing display with the second electronic device by the mask mode; when the distance between the first electronic device and the second electronic device is greater than the preset distance, the first electronic device can perform splicing display with the second electronic device by the division mode.

[0015] With reference to the first aspect, in some implementations of the first aspect, the first interface displays first page content of the first file, and the first electronic device is specifically configured to: when it is determined that the first file includes multiple pages of content, display the first page content and send second page content of the first file to the second electronic device; and the second electronic device is specifically configured to: in response to receiving the second page content, display the second page content.

[0016] In the embodiments of the present application, after receiving the first indication information sent by the second electronic device, the first electronic device and the second electronic device can perform splicing display in the page-by-page mode. For the scenario of a user watching a document application, the efficiency of the user reading the document can be improved, and the user's page turning operation can be reduced, thereby helping to improve the user's experience.

[0017] With reference to the first aspect, in some implementations of the first aspect, the first file further includes third page content and fourth page content, and the first electronic device is further configured to: in response to detecting a second input of the user, display the third page content and send the fourth page content to the second electronic device; and the second electronic device is further configured to: in response to receiving the fourth page content, display the fourth page content.

[0018] In the embodiments of the present application, when the first electronic device detects the second input of the user, the first electronic device and the second electronic device can be controlled to turn pages, so that the first electronic device and the second electronic device display the third page content and the fourth page content respectively. The efficiency of the user reading the document can be improved, and the user's page turning operation can be reduced, thereby helping to improve the user's experience.

[0019] With reference to the first aspect, in some implementations of the first aspect, the first file further includes third page content and fourth page content, and the second electronic device is further configured to, in response to detecting the third input of the user, send second indication information to the first electronic device, the second indication information being used to indicate that the second electronic device detects the third input; and the first electronic device is further configured to, in response to receiving the second indication information, display the third page content and send the fourth page content to the second electronic device; and the second electronic device is further configured to, in response to receiving the fourth page content, display the fourth page content.

[0020] In the embodiments of the present application, when the second electronic device detects the input of the user, the second electronic device can send second indication information to the first electronic device, so that the first electronic device determines that the user detects a page turning operation on the second electronic device, and the first electronic device and the second electronic device display the third page content and the fourth page content respectively, which can improve the efficiency of the user reading the document, reduce the page turning operation of the user, and thus help improve the experience of the user.

[0021] With reference to the first aspect, in some implementations of the first aspect, the first interface is a desktop of the first electronic device, and the first electronic device is specifically configured to: in response to receiving the first indication information, display the desktop and send image information corresponding to the desktop to the second electronic device; and the second electronic device is specifically configured to: in response to receiving the image information corresponding to the desktop, display the desktop.

[0022] In the embodiments of the present application, after receiving the first indication information sent by the second electronic device, the first electronic device and the second electronic device can be spliced and displayed in a page mode. For a scenario in which the user needs to start multiple application programs at the same time, for example, the user needs to send a message to a contact while watching a video, or the user receives a message sent by a contact while playing a game, the user does not need to switch the application program, but can use the first electronic device and the second electronic device, which helps improve the experience of the user.

[0023] With reference to the first aspect, in some implementations of the first aspect, the desktop includes an icon of a first application program, and the second electronic device is further configured to, in response to a fourth input of the user, send third indication information to the first electronic device, the third indication information being used to indicate that the second electronic device detects the fourth input, the fourth input being an input to the icon of the first application program; the first electronic device is further configured to, in response to receiving the third indication information, send image information corresponding to a display interface of the first application program to the second electronic device; and the second electronic device is further configured to, in response to receiving the image information corresponding to the display interface of the first application program, display the display interface of the first application program.

[0024] With reference to the first aspect, in some implementations of the first aspect, the desktop further includes an icon of a second application program, and the first electronic device is further configured to, in response to detecting a fifth input of the user on the icon of the second application program, display a display interface of the second application program.

[0025] With reference to the first aspect, in some implementations of the first aspect, the first interface is a display interface of a third application program, and the display interface includes a first interface element associated with a second interface of the third application program, where the first part of the image information is image information corresponding to the first interface, and the second part of the image information is image information corresponding to the first interface; or the first part of the image information is image information corresponding to the first interface, and the second part of the image information is image information corresponding to the second interface.

[0026] In the embodiments of the present application, after receiving the first indication information sent by the second electronic device, the first electronic device can be spliced and displayed with the second electronic device in a paging mode. Through the parallel mode, the first electronic device and the second electronic device can display the upper and lower activity pages of the application program. The application program can adjust to display two activity pages at the same time in this mode, thereby bringing better user experience when multiple devices are spliced.

[0027] In some possible implementations, the first interface element can be any interface element on the first interface that is associated with other display interfaces; or the first interface element can also be an interface element at a preset position.

[0028] With reference to the first aspect, in some implementations of the first aspect, the first interface further includes a second interface element associated with a third interface of the third application program, and the first electronic device is further configured to, in response to detecting a sixth input of the user on the second interface element, display the image information corresponding to the first interface and send the image information corresponding to the third interface to the second electronic device; and the second electronic device is further configured to, in response to receiving the image information corresponding to the third interface, display the third interface.

[0029] In the embodiments of the present application, when the first electronic device detects the operation of the user clicking the second interface element on the first interface, the first electronic device can continue to display the first interface and send the image information corresponding to the third interface to the second electronic device, so that the first electronic device and the second electronic device display two upper and lower activity pages of the first interface and the third interface respectively, without the user switching back and forth between the upper and lower activity pages, which helps to improve the user experience.

[0030] With reference to the first aspect, in some implementations of the first aspect, the third interface includes a third interface element, the third interface element is associated with a fourth interface of the third application, and the second electronic device is further configured to: in response to detecting the seventh input of the user, send fourth indication information to the first electronic device, the fourth indication information being used to indicate that the second electronic device detects the seventh input, the seventh input being an input to the third interface element; and the first electronic device is further configured to: in response to receiving the fourth indication information, display the fourth interface and send image information corresponding to the third interface to the second electronic device; and the second electronic device is further configured to: in response to receiving the image information corresponding to the third interface, display the third interface.

[0031] With reference to the first aspect, in some implementations of the first aspect, the second electronic device is further configured to: display a first control while displaying the second partial image information; and in response to detecting an eighth input of the user to the first control, display a fifth interface and send fifth indication information to the first electronic device, the fifth indication information being used to instruct the first electronic device and the second electronic device to exit the splicing display, the fifth interface being a display interface before the first electronic device detects the first input.

[0032] The second aspect provides a splicing display method, which is applied to a first electronic device in communication with a second electronic device through a close-range wireless connection. The method includes: displaying, by the first electronic device, a first interface; receiving, by the first electronic device, first indication information sent by the second electronic device, the first indication information being used to instruct the first electronic device and the second electronic device to perform splicing display; and in response to receiving the first indication information, displaying, by the first electronic device, first partial image information and sending, by the first electronic device, second partial image information to the second electronic device, the first partial image information and the second partial image information being associated with the first interface, according to position information of the first electronic device and the second electronic device.

[0033] With reference to the second aspect, in some implementations of the second aspect, the method further includes: receiving, by the first electronic device, first information sent by the second electronic device, the first information being used to indicate a size of a display screen of the second electronic device; and before the first electronic device displays the first partial image information and sends the second partial image information to the second electronic device, the method includes: according to a number of electronic devices performing splicing display, a size of a display screen of the first electronic device, and a size of a display screen of the second electronic device, expanding and dividing image information corresponding to the first interface to obtain the first partial image information and the second partial image information.

[0034] With reference to the second aspect, in some implementations of the second aspect, the display screen of the first electronic device has a first size in a first direction and a second size in a second direction, the display screen of the second electronic device has a third size in the first direction, the first direction and the second direction are perpendicular, and the first electronic device enlarges the image information corresponding to the first interface according to the number of electronic devices that are displayed in a spliced manner, the size of the display screen of the first electronic device, and the size of the display screen of the second electronic device, including: when the first electronic device and the second electronic device are displayed in the spliced manner along the first direction, the first electronic device enlarges the image information corresponding to the first interface to have a fourth size in the first direction and keeps the size of the image information corresponding to the first interface unchanged in the second direction, the fourth size being a sum of the first size and the third size.

[0035] With reference to the second aspect, in some implementations of the second aspect, the enlarged image information is further divided to obtain third partial image information, the third partial image information being located between the first partial image information and the second partial image information, and the third partial image information being determined by the first electronic device according to a distance between the first electronic device and the second electronic device.

[0036] With reference to the second aspect, in some implementations of the second aspect, the first interface displays first page content of a first file, and the first partial image information is displayed and the second partial image information is sent to the second electronic device, including: when it is determined that the first file includes multiple pages of content, the first page content is displayed and the second page content of the first file is sent to the second electronic device.

[0037] With reference to the second aspect, in some implementations of the second aspect, the first file further includes third page content and fourth page content, and the method further includes: in response to detecting a second input of a user, the first electronic device displays the third page content and sends the fourth page content to the second electronic device.

[0038] With reference to the second aspect, in some implementations of the second aspect, the first file further includes third page content and fourth page content, and the method further includes: the first electronic device receives second indication information sent by the second electronic device, the second indication information being used to indicate that the second electronic device detects a third input; and in response to receiving the second indication information, the first electronic device displays the third page content and sends the fourth page content to the second electronic device.

[0039] With reference to the second aspect, in some implementations of the second aspect, the first interface is a desktop of the first electronic device, and the displaying the first part of the image information and sending the second part of the image information to the second electronic device comprises: in response to receiving the first indication information, the first electronic device displays the desktop and sends image information corresponding to the desktop to the second electronic device.

[0040] With reference to the second aspect, in some implementations of the second aspect, the desktop comprises an icon of a first application, and the method further comprises: the first electronic device receiving third indication information sent by the second electronic device, the third indication information being used to indicate that the second electronic device detects a fourth input, the fourth input being an input to the icon of the first application; and in response to receiving the third indication information, the first electronic device sending image information corresponding to a display interface of the first application to the second electronic device.

[0041] With reference to the second aspect, in some implementations of the second aspect, the desktop further comprises an icon of a second application, and the method further comprises: in response to detecting a fifth input of a user to the icon of the second application, the first electronic device displaying a display interface of the second application.

[0042] With reference to the second aspect, in some implementations of the second aspect, the first interface is a display interface of a third application, and the display interface comprises a first interface element associated with a second interface of the third application, wherein the first part of the image information is image information corresponding to the first interface, and the second part of the image information is image information corresponding to the first interface; or the first part of the image information is image information corresponding to the first interface, and the second part of the image information is image information corresponding to the second interface.

[0043] With reference to the second aspect, in some implementations of the second aspect, the first interface further comprises a second interface element associated with a third interface of the third application, and the method further comprises: in response to detecting a sixth input of a user to the second interface element, the first electronic device displays the image information corresponding to the first interface and sends image information corresponding to the third interface to the second electronic device.

[0044] With reference to the second aspect, in some implementations of the second aspect, the third interface includes a third interface element, the third interface element is associated with a fourth interface of the third application, and the method further includes: receiving, by the first electronic device, fourth indication information sent by the second electronic device, the fourth indication information being used to indicate that the second electronic device detects the seventh input, the seventh input being an input to the third interface element; and in response to receiving the fourth indication information, displaying, by the first electronic device, the fourth interface and sending, by the first electronic device, image information corresponding to the third interface to the second electronic device.

[0045] With reference to the second aspect, in some implementations of the second aspect, the method further includes: receiving, by the first electronic device, fifth indication information sent by the second electronic device, the fifth indication information being used to indicate that the first electronic device and the second electronic device exit the splicing display.

[0046] In a third aspect, a method for splicing display is provided, the method is applied to a second electronic device, the second electronic device communicates with a first electronic device through a near field wireless connection, and the method includes: in response to detecting a first input of a user, sending first indication information to the first electronic device, the first indication information being used to indicate that the first electronic device and the second electronic device perform splicing display; receiving second part image information sent by the first electronic device, the second part image information being associated with a first interface, the first interface being a display interface of the first electronic device when the first electronic device receives the first indication information; and in response to receiving the second part image information, displaying the second part image information.

[0047] With reference to the third aspect, in some implementations of the third aspect, the second part image information is image information obtained by the first electronic device after enlarging and dividing image information corresponding to the first interface.

[0048] With reference to the third aspect, in some implementations of the third aspect, the first interface displays first page content of a first file, and the displaying, in response to receiving the second part image information, of the second part image information includes: in response to receiving the second page content, displaying, by the second electronic device, the second page content.

[0049] With reference to the third aspect, in some implementations of the third aspect, the first file further includes third page content and fourth page content, and the method further includes: in response to detecting a third input of a user, sending, by the second electronic device, second indication information to the first electronic device, the second indication information being used to indicate that the second electronic device detects the third input; receiving, by the second electronic device, the fourth page content sent by the first electronic device; and in response to receiving the fourth page content, displaying, by the second electronic device, the fourth page content.

[0050] With reference to the third aspect, in some implementations of the third aspect, the first interface is a desktop of the first electronic device, and in response to receiving the second partial image information, displaying the second partial image information includes: in response to receiving image information corresponding to the desktop, the second electronic device displays the desktop.

[0051] With reference to the third aspect, in some implementations of the third aspect, the desktop includes an icon of a first application, and the method further includes: in response to a fourth input of a user, the second electronic device sends third indication information to the first electronic device, the third indication information being used to indicate that the second electronic device detects the fourth input, the fourth input being an input to the icon of the first application; the second electronic device receives image information corresponding to a display interface of the first application; and in response to receiving the image information corresponding to the display interface of the first application, the second electronic device displays the display interface of the first application.

[0052] With reference to the third aspect, in some implementations of the third aspect, the first interface is a display interface of a third application, the display interface including a first interface element associated with a second interface of the third application, and the second partial image information is image information corresponding to the first interface or image information corresponding to the second interface.

[0053] With reference to the third aspect, in some implementations of the third aspect, the second partial image information further includes a second interface element associated with a third interface of the third application, and the method further includes: in response to detecting a fifth input of a user, the second electronic device sends fourth indication information to the first electronic device, the fourth indication information being used to indicate that the second electronic device detects the fifth input, the fifth input being an input to the second interface element; the second electronic device receives image information corresponding to the third interface sent by the first electronic device; and in response to receiving the image information corresponding to the third interface, the second electronic device displays the third interface.

[0054] With reference to the third aspect, in some implementations of the third aspect, the method further includes: the second electronic device displays a first control while displaying the second partial image information; and in response to detecting an eighth input of a user to the first control, the second electronic device displays a fourth interface and sends fifth indication information to the first electronic device, the fifth indication information being used to instruct the first electronic device and the second electronic device to exit the spliced display, the fourth interface being a display interface before the first electronic device detects the first input.

[0055] In a fourth aspect, an apparatus is provided, which includes: a display unit configured to display a first interface; a receiving unit configured to receive first indication information sent by a second electronic device, the first indication information being used to indicate that the apparatus and the second electronic device are to be displayed in a split mode; the display unit is further configured to, in response to receiving the first indication information, display first partial image information according to orientation information of the first electronic device and the second electronic device; and a sending unit configured to send second partial image information to the second electronic device, wherein the first partial image information and the second partial image information are associated with the first interface.

[0056] In a fifth aspect, an apparatus is provided, which includes: a detecting unit configured to detect a first input of a user; a sending unit configured to, in response to the first input, send first indication information to a first electronic device, the first indication information being used to indicate that the first electronic device and the apparatus are to be displayed in a split mode; a receiving unit configured to receive second partial image information sent by the first electronic device, the second partial image information being associated with a first interface, the first interface being a display interface of the first electronic device when the first electronic device receives the first indication information; and a display unit configured to, in response to receiving the second partial image information, display the second partial image information.

[0057] In a sixth aspect, an electronic device is provided, which includes: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and include instructions. When the instructions are executed by the electronic device, the electronic device performs the method in any possible implementation of the second aspect.

[0058] In a seventh aspect, an electronic device is provided, which includes: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and include instructions. When the instructions are executed by the electronic device, the electronic device performs the method in any possible implementation of the third aspect.

[0059] In an eighth aspect, a computer program product is provided, which includes instructions. When the computer program product is run on a first electronic device, the electronic device performs the method of the second aspect; or when the computer program product is run on a second electronic device, the electronic device performs the method of the third aspect.

[0060] In a ninth aspect, a computer-readable storage medium is provided, including instructions, when the instructions are run on a first electronic device, causing the electronic device to perform the method of the second aspect described above; or, when the instructions are run on a second electronic device, causing the electronic device to perform the method of the third aspect described above.

[0061] In a tenth aspect, a chip is provided for executing instructions, when the chip is run, the chip performs the method of the second aspect described above; or, the chip performs the method of the third aspect described above. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application.

[0063] Figure 2 is a software structure block diagram provided by an embodiment of the present application.

[0064] Figure 3 is a set of graphical user interfaces provided by an embodiment of the present application.

[0065] Figure 4 is another set of graphical user interfaces provided by an embodiment of the present application.

[0066] Figure 5 is another set of graphical user interfaces provided by an embodiment of the present application.

[0067] Figure 6 is another set of graphical user interfaces provided by an embodiment of the present application.

[0068] Figure 7 is another set of graphical user interfaces provided by an embodiment of the present application.

[0069] Figure 8 is another set of graphical user interfaces provided by an embodiment of the present application.

[0070] Figure 9 is another set of graphical user interfaces provided by an embodiment of the present application.

[0071] Figure 10 is another set of graphical user interfaces provided by an embodiment of the present application.

[0072] Figure 11 is another set of graphical user interfaces provided by an embodiment of the present application.

[0073] Figure 12 is another set of graphical user interfaces provided by an embodiment of the present application.

[0074] Figure 13 is another set of graphical user interfaces provided by an embodiment of the present application.

[0075] Figure 14 is another group of graphical user interfaces provided by embodiments of the present application.

[0076] Figure 15 is a schematic structural diagram of a source device and a sink device provided by embodiments of the present application.

[0077] Figure 16 is a schematic flowchart of a method for displaying by a source device and a sink device in a full-screen mode provided by embodiments of the present application.

[0078] Figure 17 is a process for cropping by a source device in a split mode and displaying the cropped canvas on a virtual screen provided by embodiments of the present application.

[0079] Figure 18 is a process for cropping by a source device in a mask mode and displaying the cropped canvas on a virtual screen provided by embodiments of the present application.

[0080] Figure 19 is another process for cropping by a source device in a split mode and displaying the cropped canvas on a virtual screen provided by embodiments of the present application.

[0081] Figure 20 is another process for cropping by a source device in a mask mode and displaying the cropped canvas on a virtual screen provided by embodiments of the present application.

[0082] Figure 21 is a schematic flowchart of a method for displaying by a source device and a sink device in a paging mode provided by embodiments of the present application.

[0083] Figure 22 is a schematic diagram of displaying a desktop of a source device in a paging mode provided by embodiments of the present application.

[0084] Figure 23 is a schematic flowchart of a method for displaying by a source device and a sink device in a parallel mode provided by embodiments of the present application.

[0085] Figure 24 is a process for opening a certain activity page in an application layer sequence in a parallel mode provided by embodiments of the present application.

[0086] Figure 25 is a schematic flowchart of a method for displaying by a source device and a sink device in a dual-application mode provided by embodiments of the present application.

[0087] Figure 26is a schematic flow chart of a method of spliced display provided by an embodiment of the present application.

[0088] Figure 27 is a schematic structural diagram of an apparatus provided by an embodiment of the present application.

[0089] Figure 28 is another schematic structural diagram of an apparatus provided by an embodiment of the present application.

[0090] Figure 29 is another structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0091] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; in this document, “and / or” only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, “plural” or “multiple” means two or more than two.

[0092] Hereinafter, the terms “first” and “second” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more features. In the description of the embodiments, unless otherwise specified, the meaning of “multiple” is two or more than two.

[0093] The method provided by the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not make any limitation on the specific type of electronic device.

[0094] Exemplary, Figure 1A structural diagram of the electronic device 100 is shown. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0095] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0096] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated into one or more processors.

[0097] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0098] The processor 110 can also have a memory that stores instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is using repeatedly. If the processor 110 needs to use the instructions or data again, it can call them directly from the memory. This avoids repeated access and reduces the latency of the processor 110, thus improving the efficiency of the system.

[0099] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0100] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100.

[0101] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled with the audio module 170 through the I2S buses to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface to enable the function of answering a phone call through a Bluetooth earphone.

[0102] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 can be coupled with the wireless communication module 160 through a PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface to enable the function of playing music through a Bluetooth earphone. Both the I2S interface and the PCM interface can be used for audio communication.

[0103] The UART interface is a universal serial bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to enable Bluetooth functionality. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the UART interface to enable the function of playing music through a Bluetooth earphone.

[0104] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to enable the camera function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to enable the display function of the electronic device 100.

[0105] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0106] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0107] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection modes or combinations of multiple interface connection modes in the above embodiments.

[0108] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 while also supplying power to the electronic device through the power management module 141.

[0109] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0110] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0111] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0112] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the same to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor, and radiate the same as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the same device as at least part of the modules of the processor 110.

[0113] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110, and disposed in the same device as the mobile communication module 150 or other functional modules.

[0114] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0115] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0116] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0117] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0118] The electronic device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0119] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0120] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0121] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0122] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0123] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognition applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0124] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0125] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0126] The electronic device 100 can realize audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0127] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functional modules of the audio module 170 can be disposed in the processor 110.

[0128] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0129] The receiver 170B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 receives a call or a voice message, the user can listen to the voice by holding the receiver 170B close to the ear.

[0130] The microphone 170C, also referred to as a "microphone", "sound transducer", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak into the microphone 170C by holding the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, the noise reduction function can also be realized. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, realize the function of directional recording, etc.

[0131] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0132] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The pressure sensor 180A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view short messages is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0133] The gyroscope sensor 180B can be configured to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake photography. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0134] The barometric pressure sensor 180C is configured to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude, assists positioning and navigation by using the air pressure value measured by the barometric pressure sensor 180C.

[0135] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the cover or the opening and closing state of the flip cover, the electronic device 100 can set a feature such as automatic unlocking of the flip cover.

[0136] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the acceleration sensor 180E can detect the magnitude and direction of gravity. The acceleration sensor 180E can also be used to identify the attitude of the electronic device, and can be applied to landscape / portrait screen switching, pedometers, etc.

[0137] Distance sensor 180F is configured to measure distance. Electronic device 100 can measure distance by infrared or laser. In some embodiments, electronic device 100 can utilize distance sensor 180F to measure distance for fast focusing when taking a picture.

[0138] Proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. Electronic device 100 emits infrared light outwardly through the light emitting diode. Electronic device 100 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, electronic device 100 can determine that there is an object near electronic device 100. When insufficient reflected light is detected, electronic device 100 can determine that there is no object near electronic device 100. Electronic device 100 can utilize proximity light sensor 180G to detect that a user is holding electronic device 100 close to the ear for a phone call, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used for automatic unlocking and locking of the screen in a holster mode or a pocket mode.

[0139] Ambient light sensor 180L is configured to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light brightness. Ambient light sensor 180L can also be used to automatically adjust white balance when taking a picture. Ambient light sensor 180L can also cooperate with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touch.

[0140] Fingerprint sensor 180H is configured to collect a fingerprint. Electronic device 100 can utilize the collected fingerprint characteristics to implement fingerprint unlocking, access application lock, fingerprint picture taking, fingerprint call answering, and the like.

[0141] Temperature sensor 180J is configured to detect temperature. In some embodiments, electronic device 100 utilizes the temperature detected by temperature sensor 180J to implement temperature processing strategies. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 implements performance reduction of a processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, electronic device 100 heats battery 142 to avoid abnormal shutdown of electronic device 100 caused by low temperature. In other embodiments, when the temperature is lower than yet another threshold, electronic device 100 implements voltage boosting of the output voltage of battery 142 to avoid abnormal shutdown caused by low temperature.

[0142] Touch sensor 180K, also referred to as "touch panel". Touch sensor 180K can be disposed on display screen 194, and touch sensor 180K and display screen 194 together form a touch screen, also referred to as "touch panel". Touch sensor 180K is configured to detect touch operations applied to or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K can also be disposed on the surface of electronic device 100, which is different from the position of display screen 194.

[0143] Bone conduction sensor 180M can obtain vibration signals. In some embodiments, bone conduction sensor 180M can obtain vibration signals of the human body's vocal vibration bone block. Bone conduction sensor 180M can also contact the human body pulse to receive blood pressure pulsation signals. In some embodiments, bone conduction sensor 180M can also be disposed in a headset to form a bone conduction headset. Audio module 170 can analyze voice signals based on the vibration signals of the vocal vibration bone block obtained by the bone conduction sensor 180M to realize voice functions. The application processor can analyze heart rate information based on the blood pressure pulsation signals obtained by the bone conduction sensor 180M to realize heart rate detection functions.

[0144] Keys 190 include power on / off keys, volume keys, and the like. Keys 190 can be mechanical keys. They can also be touch keys. Electronic device 100 can receive key input and generate key signal input related to user settings and function control of electronic device 100.

[0145] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations applied to different regions of display screen 194 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effects can also be customizable.

[0146] Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, and also to indicate messages, missed calls, notifications, and the like.

[0147] The SIM card interface 195 is configured to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by being inserted into or pulled out of the SIM card interface 195. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195. The types of the multiple cards can be the same or different. The SIM card interface 195 can be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with a network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 uses an embedded-SIM (eSIM) card, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0148] It should be understood that the phone card in the embodiments of the present application includes, but is not limited to, a SIM card, an eSIM card, a universal subscriber identity module (USIM), a universal integrated circuit card (UICC), and the like.

[0149] The software system of the electronic device 100 can use a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application exemplarily illustrate the software structure of the electronic device 100 by taking an Android system with a layered architecture as an example.

[0150] Figure 2 FIG. 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Android runtime and system library, and a kernel layer. The application layer can include a series of application packages.

[0151] As shown in FIG. 1, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and the like. Figure 2

[0152] ​The application framework layer provides an application programming interface (API) and programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0153] As shown in Figure 2 the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0154] The window manager is used to manage the window program. The window manager can acquire the display screen size, determine whether there is a status bar, lock the screen, and intercept the screen, etc.

[0155] The content provider is used to store and acquire data, and make the data accessible to the application program. The data can include video, image, audio, dialed and received phone, browsing history and bookmark, phone book, etc.

[0156] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, etc. The view system can be used to build an application program. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0157] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including connection, hang-up, etc.).

[0158] The resource manager provides various resources for the application program, such as localized strings, icons, pictures, layout files, video files, etc.

[0159] The notification manager makes the application program display notification information in the status bar, which can be used to convey a type of message that can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform the completion of downloading, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, the electronic device vibrating, the indicator light flashing, etc.

[0160] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0161] The core library includes two parts: one part is the function function required to be called by the java language, and the other part is the core library of Android.

[0162] The application program layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application program layer and the application framework layer into binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, management of security and exceptions, and garbage collection.

[0163] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (for example, OpenGL ES), a 2D graphics engine (for example, SGL), and the like.

[0164] The surface manager is used to manage a display subsystem and provides fusion of 2D and 3D layers for a plurality of application programs.

[0165] The media library supports playback and recording of a plurality of commonly used audio, video formats, and static image files. The media library can support a plurality of audio and video encoding formats, for example, MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like.

[0166] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, and the like.

[0167] The 2D graphics engine is a drawing engine for 2D drawing.

[0168] The kernel layer is a layer between hardware and software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0169] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Harmony, and the like.

[0170] Figure 3 is a group of graphical user interfaces (GUIs) provided by the embodiments of the present application.

[0171] Referring to the GUI shown in (a) of Figure 3 The mobile phone A displays a play interface of a video application, and the play interface displays a video play screen 301. At this time, the mobile phone B displays a desktop of the mobile phone B.

[0172] When the mobile phone B is close to the mobile phone A, the mobile phone A and the mobile phone B can form a network through a near field wireless connection mode. The network formation mode includes but is not limited to an access point (AP) network formation and a peer-to-peer (P2P) network formation. The AP network formation is that devices under the same AP (for example, a home Wi-Fi router) can communicate with each other through the AP device, thereby forming a many-to-many network formation. For example, the mobile phone A and the mobile phone B can be located under the same home router, and when the mobile phone B is close to the mobile phone A, the mobile phone A can use a received signal strength indication (RSSI) technology to calculate the distance between the device A and the device B according to the strength of the received signal. When the distance is less than or equal to a preset distance, the mobile phone A and the mobile phone B can form an AP network.

[0173] Wi-Fi direct (Wi-Fi direct): also known as Wi-Fi peer to peer (Wi-Fi P2P), is a point-to-point connection mode. It can enable multiple Wi-Fi devices to form a peer-to-peer network (P2P network) to communicate with each other without an access point (AP). One station (STA) can act as an AP in the traditional sense, called a group owner (GO); another STA can be called a group client (GC), which can be connected to the GO like connecting to the AP. Among them, one STA can play the role of GO (i.e., act as an AP), and other STAs play the role of GC. In the embodiments of the present application, when a device is close to another device, the device on the left side can be the GO by default, and the device on the right side can be the GC. For example, the mobile phone A can be the GO, and the mobile phone B can be the GC. Alternatively, when a device detects a user's right sliding operation, the device can be the GC and the device can select another device on the left side as the GO; or when a device detects a user's left sliding operation, the device can be the GC and the device can select another device on the right side as the GO.

[0174] To build a peer-to-peer (P2P) network, one electronic device must first be able to scan for and detect another electronic device using the P2P protocol; this process is called the discovery phase. Only after discovery can the establishment of a P2P connection be triggered. When phone B approaches phone A, phone A can use RSSI technology to calculate the distance between device A and device B based on the RSSI. When this distance is less than or equal to a preset distance, phone A and phone B can form a P2P network.

[0175] Wi-Fi P2P technology forms the foundation for upper-layer services. Currently, P2P applications built on top of P2P mainly include Miracast and WLAN Direct. In a Miracast application scenario, a P2P-enabled electronic device can scan for and connect to a P2P-enabled large-screen device, then directly display its video, images, and other resources on the large-screen device. Leveraging P2P technology greatly enriches the Wi-Fi experience.

[0176] When phone B detects that the user swipes right on the desktop, phone B sends an instruction message to phone A, which indicates that phone B wants to enter full-screen mode.

[0177] See Figure 3 The GUI is shown in (b). After receiving the instruction information sent by mobile phone B, mobile phone A can enlarge the size of the canvas displayed in the current playback interface by 1 time. Mobile phone A can crop the enlarged canvas to obtain two areas of the same size (area 302 and area 303). Mobile phone A can display the canvas shown in area 302 on its screen and project the canvas of area 303 onto mobile phone B, so that mobile phone B can display the canvas of area 303 on its screen.

[0178] In one embodiment, the display interface of mobile phone B also includes an exit control. When mobile phone B detects that the user clicks the exit control, mobile phone B can send an instruction message to mobile phone A, which instructs mobile phone A to exit full-screen mode.

[0179] In one embodiment, the exit control can be drawn by mobile phone B. When mobile phones A and B enter full-screen mode, the exit control can be drawn and displayed on the screen of mobile phone B.

[0180] See Figure 3 As shown in GUI (c), when phone C is close to phone A, phones A, B, and C can form an AP network or a P2P network. When phone C detects a user swiping to the right, phone C can send an instruction message to phone A, indicating that phone C wants to enter full-screen mode.

[0181] In one embodiment, when mobile phone C is close to mobile phone B, mobile phone B can determine that the distance between mobile phone C and mobile phone B is less than or equal to a preset distance by RSSI ranging. Mobile phone A, mobile phone B and mobile phone C can form an AP network, or mobile phone A, mobile phone B and mobile phone C can form a P2P network.

[0182] In one embodiment, when mobile phone C is close to mobile phone B, mobile phones A and B can determine the distance between mobile phone A and mobile phone C, and the distance between mobile phone B and mobile phone C, respectively, using RSSI ranging. When the distance between mobile phone A and mobile phone C is less than or equal to a first preset distance, and the distance between mobile phone B and mobile phone C is less than or equal to a second preset distance, mobile phones A, B, and C can form an AP network, or mobile phones A, B, and C can form a P2P network.

[0183] See Figure 3 As shown in (d) of the GUI, after receiving the instruction information sent by phone C, phone A can enlarge the size of the canvas displayed on the current playback interface by 2 times. Phone A can then crop the enlarged canvas to obtain three areas of the same size (area 304, area 305, and area 306). Phone A can display the canvas shown in area 304 on its screen. Phone A can also project the canvas shown in area 305 onto phone B and the canvas shown in area 306 onto phone C, so that phone B displays the canvas shown in area 305 on its screen, and phone C displays the canvas shown in area 306 on its screen.

[0184] In one embodiment, the display interface of mobile phone C also includes an exit control. When mobile phone C detects that the user clicks the exit control, mobile phone C can send an instruction message to mobile phone A, which instructs mobile phone A to exit full-screen mode.

[0185] In one embodiment, the exit control can be drawn by mobile phone C. When mobile phones A, B, and C enter full-screen mode, mobile phone C can draw the exit control and display it on its screen. At this time, mobile phones B and C can both include the exit control.

[0186] In this embodiment, multiple devices can be spliced ​​together to form a larger screen through near-field wireless connection without the need for additional hardware. The source device can dynamically modify the size of the display canvas and distribute it to various sink devices for display after cropping, which helps to improve the user experience.

[0187] The above combination Figure 3It is shown that the phone A expands the canvas, and then evenly divides the canvas size according to the number of devices, and then presents the canvas on each device completely. This way can also be considered as a splitting way. Considering that there is a distance between the phone A and the phone B or between the phone B and the phone C, the phone A regards the joint between the devices as a mask element after expanding the canvas, and the canvas is blocked by the mask element. The phone A can also process the expanded canvas by the mask way. The process of processing by the mask way is introduced below in combination with the GUI shown in Figure 4

[0188] Referring to the GUI shown in (a) of Figure 4 , the phone A displays a play interface of a video application, and the play interface displays a video playing picture. At this time, the phone B displays a desktop of the phone B.

[0189] When the phone B approaches the phone A, the phone A and the phone B can form a network by a near field wireless connection way. When the phone B detects a user's operation of swiping right on the desktop, the phone B sends indication information to the phone A, and the indication information is used to indicate that the phone B wants to enter a full screen mode.

[0190] Referring to the GUI shown in (b) of Figure 4 , after receiving the indication information sent by the phone B, the phone A can expand the displayed canvas size in the current play interface by 1 times. The phone A can crop the canvas expanded by 1 times according to the distance between the phone A and the phone B, so as to obtain the canvas shown in the regions 401, 402 and 403. The phone A can display the canvas shown in the region 401 through the display screen, and project the canvas shown in the region 403 to the phone B, so that the phone B displays the canvas shown in the region 403 through the display screen.

[0191] The pixel value of the region 402 is determined according to the distance between the phone A and the phone B. A possible implementation way is that, if the screen physical size of the phone A is X (6 cm), and the screen resolution is M (1080) * N (2340). Assuming that the physical size of the width of the region 402 is Y (for example, 0.5 cm), then the pixel value Z to be cropped is Z=(Y / X)*M=90, that is, 90 pixels need to be cropped. In the embodiment of the present application, the physical size of the width of the region 402 can be calculated by the phone A according to the RSSI to measure the distance between the phone A and the phone B; or the physical size of the width of the region 402 can also be the sum of the physical frame sizes of the phone A and the phone B when the phone A and the phone B are completely attached.

[0192] ​In one embodiment, if the distance between the mobile phone A and the mobile phone B is greater than a preset distance (for example, 3 cm), the mobile phone A can perform the cropping on the expanded canvas in the splitting manner; if the distance between the mobile phone A and the mobile phone B is less than or equal to the preset distance, the mobile phone A can perform the cropping on the expanded canvas in the masking manner.

[0193] Referring to the GUI shown as (c) in Figure 4 When the mobile phone C approaches the mobile phone B, the mobile phone A, the mobile phone B and the mobile phone C can perform AP networking or P2P networking. When the mobile phone C detects the user's operation of swiping to the right, the mobile phone C can send indication information to the mobile phone A, where the indication information is used to indicate that the mobile phone C wants to enter the full-screen mode.

[0194] In one embodiment, the mobile phone C can also indicate the distance between the mobile phone B and the mobile phone C to the mobile phone A.

[0195] Referring to the GUI shown as (d) in Figure 4 When the mobile phone A receives the indication information sent by the mobile phone C, the mobile phone A can expand the size of the displayed canvas in the current playing interface by 2 times. The mobile phone A can crop the canvas expanded by 2 times according to the distance between the mobile phone A and the mobile phone B and the distance between the mobile phone B and the mobile phone C. As shown in (d) in Figure 4 The mobile phone A can crop the canvas expanded by 2 times into the areas 404, 405, 406, 407 and 408. The pixel values of the areas 405 and 407 can be determined according to the distance between the mobile phone A and the mobile phone B and the distance between the mobile phone B and the mobile phone C, and the specific determination process can be referred to the description in the above embodiments, which will not be described here for brevity. The mobile phone A can display the canvas shown by the area 404 through the display screen, and project the canvas shown by the area 406 onto the mobile phone B and project the canvas shown by the area 408 onto the mobile phone C, so that the mobile phone B displays the canvas shown by the area 406 through the display screen, and the mobile phone C displays the canvas shown by the area 408 through the display screen.

[0196] The above describes the process of cropping the canvas in the splitting manner and the masking manner respectively. Figure 3 and Figure 4 The following describes the process of exiting the full-screen mode among multiple devices in combination with Figure 5 . Figure 5 Another set of GUIs provided by the embodiments of the present application is shown.

[0197] Referring to the GUI shown as (a) in Figure 5 , the mobile phone A, the mobile phone B and the mobile phone C are currently in the full-screen mode. When the mobile phone C detects the user's operation of clicking the exit control, the GUI shown as (b) in Figure 5 is displayed.

[0198] Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed. Figure 5 Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed. Figure 3 Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed. Figure 5 Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed.

[0199] Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed. Figure 5 Referring to the GUI shown in (b) of FIG. 13A, in response to detecting that the user clicks the exit control, the phone C can send indication information to the phone A, the indication information being used to instruct the phone C to exit the full-screen mode. After receiving the indication information, the phone A can adjust the canvas size expansion from 2 times to 1 time. The phone A can crop the canvas after the expansion by 1 time, thereby obtaining two regions of the same size (for example, regions 302 and 303 shown in (b) of FIG. 13B). The phone A can display the canvas shown by the region 302 through the display screen and cast the canvas shown by the region 303 to the phone B, thereby causing the phone B to display the canvas of the region 303 through the display screen. The phone C exits the full-screen mode and displays the desktop of the phone C. When the phone B detects that the user clicks the exit control, the GUI shown in (c) of FIG. 13B can be displayed.

[0200] It should also be understood that, Figure 5 Referring to the GUI shown in (a) of FIG. 13A, if the phone A processes the canvas after the expansion by 2 times in the splitting manner, after receiving the indication that the phone C exits the full-screen mode, the phone A can continue to process the canvas after the expansion by 1 time in the splitting manner. In an embodiment, if the phone A processes the canvas after the expansion by 2 times in the masking manner, after receiving the indication that the phone C exits the full-screen mode, the phone A can continue to process the canvas after the expansion by 1 time in the masking manner.

[0201] Figure 6 Another set of GUIs provided by embodiments of the present application is shown.

[0202] Referring to the GUI shown in (a) of FIG. 14A, the distance between the phone A and the phone B is greater than a preset distance, the phone A displays the desktop of the phone A, and the phone B displays the desktop of the phone B. Figure 6 When the phone A and the phone B are close and the distance between the phone A and the phone B is less than or equal to the preset distance, the phone A and the phone B display the GUI shown in (b) of FIG. 14B.

[0203] Figure 6 ​​

[0204] See Figure 6 As shown in (b) of the GUI, when the distance between phone A and phone B is less than or equal to a preset distance, control 601 is displayed on the desktop of phone A, and control 602 is displayed on the desktop of phone B. When phone B detects the user dragging control 602 to the right, it displays as shown in (b). Figure 6 The GUI shown in (c) is shown in the image.

[0205] See Figure 6 As shown in (c) of the diagram, in response to phone B detecting a user dragging control 602 to the right, phone B sends an instruction to phone A. This instruction indicates that phone B wishes to enter full-screen mode. Upon receiving this instruction, phone A can enlarge the canvas on its current screen (phone A's desktop) by a factor of 1. Phone A can then crop the enlarged canvas to obtain two areas of the same size (a left area and a right area). Phone A can display the canvas shown in the left area of ​​its screen (including network type, signal strength, and Wi-Fi connection status in the status bar, time (08:08) and date information (August 6th, Thursday), and icons of Apps 1, 2, 5, 6, 9, and 10) and project the canvas shown in the right area of ​​its screen (including battery level information, time information (08:08), weather information (35℃), location information (Xi'an), and icons of Apps 3, 4, 7, 8, 11, and 12) onto Phone B, thus allowing Phone B to view the canvas in the right area of ​​its screen. Phone B can also display icons for parallel mode (603), dual-application mode (604), pagination mode (605), and an exit control (606).

[0206] In one embodiment, the icon 603 for parallel mode, the icon 604 for dual application mode, the icon 605 for pagination mode, and the exit control 606 can be drawn by mobile phone B. After receiving the screen projection content from mobile phone A, mobile phone B can add the above-mentioned icons and controls to the screen projection content.

[0207] In one embodiment, the icon 603 for parallel mode, the icon 604 for dual-application mode, the icon 605 for pagination mode, and the exit control 606 can also be drawn by mobile phone A and displayed on mobile phone A.

[0208] In one embodiment, the icon 603 for parallel mode, the icon 604 for dual application mode, and the icon 605 for pagination mode can be drawn by mobile phone A and displayed on mobile phone A, and the exit control 606 can be drawn by mobile phone B and displayed on mobile phone B.

[0209] When phone A detects that the user has tapped the App 2 icon, it displays the following: Figure 6 The GUI is shown in (d) above. In response to phone A detecting a user tapping the App2 icon, phone A launches App2, simultaneously enlarging the canvas displayed by App2 by one time and processing the enlarged canvas. The display methods of phones A and B can be referred to the description in the above embodiments; for simplicity, they will not be repeated here.

[0210] In this embodiment of the application, App2 can be a video App.

[0211] like Figure 6 In the GUI shown in (e), when mobile phone B detects a user's click on control 607, mobile phone B can send a touch event and coordinate point information to mobile phone A. The touch event indicates that mobile phone B has detected the user's click operation, and the coordinate point information indicates the coordinate point at which the user clicked. Mobile phone A can determine that the user clicked the full-screen viewing control on mobile phone B based on the touch event and coordinate point information, thereby displaying the video window in full-screen mode on both mobile phone A and mobile phone B. It should be understood that the process of displaying the video window in full-screen mode on mobile phone A and mobile phone B can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0212] It should be understood that Figure 7 The mobile phone A shown can crop the canvas after it has been enlarged by 1 time through segmentation, or it can crop the canvas after it has been enlarged by 1 time through masking. The specific implementation process can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0213] Figure 7 Another set of GUIs provided in an embodiment of this application is shown. This GUI illustrates the process of switching from full-screen mode to dual-application mode.

[0214] See Figure 7 As shown in (a) of the GUI, both phones A and B are in full-screen mode. When phone B detects that the user has clicked on the dual-application mode operation, it displays as shown below. Figure 7 The GUI shown in (b) is shown in the image.

[0215] See Figure 7As shown in (b) of the diagram, in response to phone B detecting that the user has clicked the dual-application mode icon, phone B can send an instruction message to phone A. This instruction message indicates that phone B has detected that the user has clicked the dual-application mode icon. Upon receiving the instruction message from phone A, phone A displays its own desktop and sends the canvas corresponding to its desktop to phone B, so that phone B also displays phone A's desktop. When phone A detects that the user has clicked the App3 icon, it displays as shown... Figure 7 The GUI shown in (c) is shown in the image.

[0216] See Figure 7 The GUI shown in (c) responds to a user tapping the App3 icon on phone A. Phone A can then display the App3 interface, while phone B displays phone A's home screen. When phone B detects a user tapping the App4 icon, it displays... Figure 7 The GUI shown in (d) is shown in the image.

[0217] In this embodiment of the application, App3 can be a video app, and App4 can be a photo app.

[0218] See Figure 8 As shown in (d) of the diagram, in response to phone B detecting a user tapping the App4 icon, phone B can send a touch event and coordinate information to phone A. The touch event indicates that phone B detected the user's tap, and the coordinate information indicates the coordinates of the tap. Phone A can then determine that the user tapped the App4 icon on phone B based on this coordinate information. Therefore, phone A can launch App4 in the background and project its interface onto phone B, allowing phone B to display the App4 interface.

[0219] In this embodiment, when mobile phone A and mobile phone B are in dual-application mode, the user can launch different applications on mobile phone A on both mobile phone A and mobile phone B, which helps to improve the user experience. For example, the user can use mobile phone A and mobile phone B to open shopping app 1 and shopping app 2 respectively, so as to compare prices of goods; or, the user can use a social app to chat on mobile phone A while using a video app to watch videos on mobile phone B.

[0220] Figure 8 Another set of GUIs provided in an embodiment of this application is shown. This GUI illustrates the process of switching from full-screen mode to paginated mode.

[0221] See Figure 8As shown in (a) of the GUI, both phones A and B are in full-screen mode. When phone B detects that the user has clicked on the pagination mode, it displays as shown in [image / description]. Figure 8 The GUI shown in (b) is shown in the image.

[0222] See Figure 8 As shown in (b) of the GUI, in response to phone B detecting that the user has clicked the pagination mode icon, phone B can send an instruction message to phone A. This instruction message indicates that phone B has detected that the user has clicked the pagination mode icon. Phone A can then switch from full-screen mode to pagination mode based on this instruction message. Phone A displays its desktop, which includes three desktop pages; currently, phone A displays the first desktop page. Phone A projects the canvas corresponding to its second desktop page onto phone B, causing phone B to display phone A's second desktop page. When phone A detects that the user has clicked the App5 icon, it displays as shown... Figure 8 The GUI shown in (c) is shown in the image.

[0223] See Figure 9 The GUI shown in (c) responds to a user tapping the App5 icon on phone A. Phone A can then display the App5 interface, which is the interface for document 1, which consists of 8 pages. Phone A can display the content of page 1 of document 1, and simultaneously project the content of page 2 of document 1 onto phone B, causing phone B to display the content of page 2 of document 1.

[0224] In one embodiment, when mobile phone A detects a user's swipe operation on the display screen (e.g., swipe up), mobile phone A can display the content of page 3 of document 1, and at the same time, mobile phone A projects the content of page 4 of document 1 onto mobile phone B, so that mobile phone B displays the content of page 4 of document 1.

[0225] In this embodiment, when mobile phone A and mobile phone B are in pagination mode, it helps to improve the efficiency of users browsing interface content, thereby improving the user experience.

[0226] Figure 9 Another set of GUIs provided in an embodiment of this application is shown. This GUI illustrates the process of switching from full-screen mode to parallel mode.

[0227] See Figure 9 The GUI shown in (a) indicates that both phones A and B are in full-screen mode. When phone B detects that the user has clicked on the parallel mode operation, it displays as shown below. Figure 9 The GUI shown in (b) is shown in the image.

[0228] See Figure 9In response to detecting that the user clicks the icon of the parallel mode, the phone B can send indication information to the phone A, the indication information being used to indicate that the phone B detects that the user clicks the icon of the parallel mode. The phone A can determine to switch from the full screen mode to the parallel mode according to the indication information. The phone A displays the desktop of the phone A, and the phone B displays the desktop of the phone B. When the phone A detects that the user clicks the icon of the App 6, the phone A displays the GUI as shown in (c) of FIG. 6. Figure 9

[0229] It should be understood that, as shown in (a) of FIG. 6, when the phone A determines to switch from the full screen mode to the parallel mode, the content displayed by the phone B is not limited. The phone B can display the desktop of the phone B; or, when the phone A and the phone B enter the parallel mode, the phone A can cast the canvas corresponding to the desktop of the phone A to the phone B, so that the phone B displays the desktop of the phone A. Figure 9

[0230] In one embodiment, the phone A can display a first display interface of an application, and the phone B can display the desktop of the phone B. When the phone B detects that the user slides from left to right on the screen, the phone B can send indication information to the phone A, the indication information being used to indicate that the phone B and the phone A are displayed by the parallel mode. In response to receiving the indication information sent by the phone B, the phone A can continue to display the first display interface of the application, and the phone A can not send any image information to the phone B, and the phone B can continue to display the desktop of the phone B; or, in response to receiving the indication information sent by the phone B, the phone A can continue to display the first display interface of the application, and the phone A can send the image information corresponding to the first display interface to the phone B, so that the phone B can display the first display interface; or, the first display interface includes a first interface element, the first interface element being associated with another display interface of the application, in response to receiving the indication information sent by the phone B, the phone A can continue to display the first display interface of the application, and the phone A can send the image information corresponding to the second display interface to the phone B. It should be understood that, when receiving the indication information, the phone A can select an interface element on the display interface at random, or the phone A can select an interface element located at a certain position (for example, the upper left corner) on the display interface. After selecting the interface element, the phone A can send the display interface associated with the interface element to the phone B, so that the phone B displays the display interface corresponding to the interface element.

[0231] Referring to Figure 9 ​​GUI shown in (c) of FIG. 13A, in response to the phone A detecting the operation of the user clicking the icon of the App 6, the phone A can start the App 6. The App 6 is a social application App, and the phone A displays a chat list of the user and multiple contacts, while the phone B displays the desktop of the phone B. When the phone A detects the operation of the user clicking the chat record of the user and the contact Dad, the phone A displays a GUI as shown in (d) of FIG. 13A. Figure 9 GUI shown in (d) of FIG. 13A.

[0232] Referring to Figure 9 GUI shown in (d) of FIG. 13A, in response to the phone A detecting the operation of the user clicking the chat record of the user and the contact Dad, the phone A displays a chat list of the user and multiple contacts, and the phone A can cast the chat interface of the user and the contact Dad to the phone B, so that the phone B displays the chat interface of the user and the contact Dad. The chat interface includes a text input box, a sending control, a control for sending a local photo, a control for sending a photo taken in real time, and a video call control. When the phone B detects the operation of the user clicking the video call control, the phone B displays a GUI as shown in (e) of FIG. 13A. Figure 9 GUI shown in (e) of FIG. 13A.

[0233] Referring to Figure 9 GUI shown in (e) of FIG. 13A, in response to the phone B detecting the operation of the user clicking the video call control, the phone B sends a touch event and coordinate point information to the phone A, the touch event is used to indicate that the phone B detects the clicking operation of the user, and the coordinate point information is used to indicate the coordinate point when the phone B detects the clicking of the user. The phone A can determine that the user clicks the video call control on the phone B according to the coordinate point information. The phone A can display an interface for requesting the contact Dad to answer the video call.

[0234] Referring to Figure 10 GUI shown in (f) of FIG. 13A, when the contact Dad accepts the video call request, the phone A can display a video chat interface of the user and the contact Dad, while the phone B can display the chat interface of the user and the contact Dad. When the phone B detects the text content input by the user, the phone B can send the text content to the phone A, so as to complete the real reply to the message on the phone A. After completing the real reply to the message, the phone A can cast the updated chat interface of the user and the contact Dad to the phone B, so that the phone B displays the updated chat interface of the user and the contact Dad.

[0235] While displaying the video chat interface of the user and the contact Dad on the phone A, the user can chat with the contact Dad through the phone B. The user can send text information (for example, “show you the photo I took last night”) and pictures to the contact Dad through the chat interface.

[0236] This application provides a method for displaying application activity pages in parallel. In this mode, the application can adjust to display two activity pages simultaneously, thereby bringing a better user experience when multiple devices are connected.

[0237] Figure 10 Another set of GUIs provided in the embodiments of this application is shown.

[0238] See Figure 10 As shown in GUI (a), when the distance between phone A and phone B is less than or equal to a preset distance, control 1001 is displayed on the desktop of phone A, and control 1002 is displayed on the desktop of phone B. When phone A detects the user dragging control 1001 to the left, it displays as shown in [image 1]. Figure 10 The GUI shown in (b) is shown in the image.

[0239] See Figure 11 In the GUI shown in (b), in response to phone A detecting a user dragging control 1001 to the left, phone A sends an instruction to phone B indicating that phone A wishes to enter full-screen mode. Upon receiving this instruction, phone B can enlarge the canvas on its current interface (phone B's desktop) by a factor of 1. Phone B can then crop the enlarged canvas to obtain two areas of the same size (a left area and a right area). Phone B can display the canvas shown in the right area of ​​its screen (including battery level information, time (08:08), date information (August 6th, Thursday), location information (High-tech Zone), and icons of Apps 15, 16, 19, 20, 23, and 24), and project the canvas shown in the left area of ​​its screen (including network type, Wi-Fi connection status, weather information (35℃), and icons of Apps 13, 14, 17, 18, 21, and 22) onto Phone A, thus allowing Phone A to display the canvas shown in the left area on its screen. Phone A can also include icons for parallel mode (1003), dual-application mode (1004), pagination mode (1005), and an exit control (1006).

[0240] In one embodiment, the icon 1003 for parallel mode, the icon 1004 for dual-application mode, the icon 1005 for pagination mode, and the exit control 1006 may be drawn by mobile phone A and displayed on the screen of mobile phone A.

[0241] In one embodiment, when mobile phone A and mobile phone B enter full-screen mode, the icon 1003 of parallel mode, the icon 1004 of dual application mode, the icon 1005 of pagination mode, and the exit control 1006 can also be drawn by mobile phone B and displayed on the display screen of mobile phone B.

[0242] In one embodiment, when mobile phone A and mobile phone B enter full-screen mode, the icon 1003 of parallel mode, the icon 1004 of dual application mode and the icon 1005 of pagination mode can be displayed on mobile phone B, and the exit control 1006 can be displayed on mobile phone A.

[0243] Figure 11 Another set of GUIs is shown in an embodiment of this application.

[0244] See Figure 11 As shown in GUI (a), when the distance between phone A and phone B is less than or equal to a preset distance, control 1101 is displayed on the desktop of phone A, and control 1102 is displayed on the desktop of phone B. When phone B detects the user dragging control 1102 to the right, it displays as shown in [image missing]. Figure 11 The GUI shown in (b) is shown in the image.

[0245] See Figure 11 As shown in (b), in response to the user's dragging of control 1102 to the right, mobile phone B can display icons for full-screen mode, dual-application mode, parallel mode, and pagination mode on its screen. If the user wants both mobile phones A and B to enter full-screen mode, the user can drag control 1102 to the upper right.

[0246] See Figure 11 In the GUI shown in (c), when phone B detects that the user has dragged control 1102 to overlap or completely overlap with the icon corresponding to the full-screen mode, phone B can send an instruction message to phone A. This instruction message indicates that phone B wants to enter full-screen mode. After receiving the instruction message, phone A can enlarge the canvas on the current interface (phone A's desktop) by 1 time. Phone A can crop the enlarged canvas to obtain two areas of the same size (the left area and the right area). Phone A can display the canvas shown in the left area on its screen (including the network type, Wi-Fi connection status, time (08:08), and date information (August 6, Thursday) in the status bar, and the icons of App1, App2, App5, App6, App9, and App10) and project the canvas shown in the right area (including battery level information, weather information (35℃), and the icons of App3, App4, App7, App8, App11, and App12) onto phone B, so that phone B can view the canvas in the right area of ​​its screen. Additionally, phone B can also display icons for parallel mode, dual-application mode, pagination mode, and an exit control.

[0247] In one embodiment, such as Figure 11As shown in (b) of FIG. 12, when the phone B detects that the user drags the control 1102 to an area that is greater than or equal to a preset area, the phone B can send the indication information to the phone A.

[0248] It should be understood that, Figure 12 As shown in (a) of FIG. 12, the phone A can crop the canvas that is enlarged by 1 times in a split manner, or the phone A can crop the canvas that is enlarged by 1 times in a mask manner.

[0249] Figure 12 Another set of GUIs in the embodiments of the present application is shown.

[0250] Referring to Figure 12 As shown in (a) of FIG. 12, when the distance between the phone A and the phone B is less than or equal to a preset distance, the phone A displays the control 1201 on the desktop, and the phone B displays the control 1202 on the desktop. When the phone A detects that the user drags the control 1201 to the left, the phone A displays a GUI as shown in (b) of FIG. 12. Figure 12

[0251] Referring to Figure 12 As shown in (b) of FIG. 12, in response to the phone A detecting that the user drags the control 1201 to the left, the phone B can display an icon of the full screen mode, an icon of the dual application mode, an icon of the parallel mode, and an icon of the page mode on the display screen. If the user wants the phone A and the phone B to enter the full screen mode, the user can drag the control 1201 to the upper left.

[0252] Referring to Figure 13 ​In the GUI shown in (c), when phone A detects that the user has dragged control 1201 to overlap or completely overlap with the icon portion of the full-screen mode, phone A sends an instruction message to phone B. This instruction message indicates that phone A wishes to enter full-screen mode. After receiving this instruction message, phone B can enlarge the canvas on the current interface (phone B's desktop) by 1 time. Phone B can then crop the enlarged canvas to obtain two areas of the same size (the left area and the right area). Phone B can display the canvas shown in the right area of ​​its screen (including battery level information, time (08:08), date information (August 6th, Thursday), location information (High-tech Zone), and icons of Apps 15, 16, 19, 20, 23, and 24), and project the canvas shown in the left area of ​​its screen (including network type, Wi-Fi connection status, weather information (35℃), and icons of Apps 13, 14, 17, 18, 21, and 22) onto Phone A, allowing Phone A to display the canvas shown in the left area. Simultaneously, Phone A can also display icons for parallel mode, dual-application mode, pagination mode, and an exit control.

[0253] In one embodiment, when mobile phone A detects that the area of ​​the part of the control 1201 that overlaps with the icon in full-screen mode is greater than or equal to a preset area, mobile phone A can send the instruction information to mobile phone B.

[0254] Figure 14 and Figure 13 This shows the process of phone A and phone B automatically entering full-screen mode and automatically exiting full-screen mode.

[0255] See Figure 13 As shown in (a) of the GUI, when the distance between phone A and phone B is greater than a preset distance, phone A displays its own desktop, and phone B displays its own desktop. When phone A and phone B are close together, the following is displayed: Figure 13 The GUI shown in (b) is shown in the image.

[0256] See Figure 14 As shown in GUI (b), when the distance between mobile phone A and mobile phone B is less than or equal to a preset distance, mobile phone B sends an instruction message to mobile phone A. This instruction message indicates that mobile phone B wishes to enter full-screen mode. After receiving this instruction message, mobile phone A can enlarge the canvas on its current interface (mobile phone A's desktop) by 1 time. Mobile phone A can then crop the enlarged canvas, thereby enabling both mobile phone A and mobile phone B to display in full-screen mode. The process of mobile phone A cropping the enlarged canvas can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0257] In the embodiments of the present application, the phone A and the phone B can default to the leftmost device as the source device and the rightmost device as the sink device.

[0258] Referring to the GUI shown in (a) of FIG. 1, after the phone A and the phone B enter the full screen mode, the user moves the phone B away from the phone A. Figure 14

[0259] Referring to the GUI shown in (b) of FIG. 1, when the phone A detects that the distance between the phone A and the phone B is greater than the preset distance, the phone A and the phone B can automatically exit the full screen mode, or the phone A can stop sending the cropped canvas to the phone B. The phone A can display the desktop of the phone A, and the phone B displays the desktop of the phone B. Figures 3 to 14

[0260] In one embodiment, when the phone A detects that the distance between the phone A and the phone B is greater than the preset distance, the phone A and the phone B can automatically exit the full screen mode, and the phone A can also disconnect the wireless connection with the phone B.

[0261] The above describes several groups of GUIs provided by the embodiments of the present application in combination with the GUIs shown in FIG. 1. The following will introduce the interaction process between the source device and the sink device in the embodiments of the present application in combination with the accompanying drawings. Figure 15

[0262] Figure 3 FIG. 2 shows a schematic structural diagram of the source device and the sink device provided by the embodiments of the present application. The source device can be the phone A shown in FIG. 1, and the sink device can be the phone B shown in FIG. 1; or the source device can be the phone B shown in FIG. 1, and the sink device can be the phone A shown in FIG. 1. Figure 3 Figure 10 Figure 10 Figure 15

[0263] ​​​​​​​The source device includes an application (App) layer, an application framework (Framework) layer, a native (Native) layer, and a server (Server) layer. The App layer can include multiple applications. The Framework layer includes a layer management and an input subsystem. The layer management is used to manage layer information corresponding to an application interface, and the input subsystem is used to process a user input event or a reverse input event. The Native layer includes a layer synthesis module and a layer clipping module. The layer synthesis module is used to synthesize an image according to layer information, and the layer clipping module is used to clip a canvas according to layer information. The Server layer includes an audio / video stream grabbing module, a virtual screen management module, and a network module. The audio / video stream grabbing module is used to grab an audio stream or a video stream. The virtual screen management module is used to manage creation and release of a virtual display. The network module is used to transmit an audio stream or a video stream to a sink device and receive a reverse input event sent by the sink device.

[0264] It should be understood that Figure 2 The functions implemented by the modules in the native layer and the server layer of the source device can be dependent on Figure 2 system libraries and kernel layers or drive hardware. For example, the layer synthesis module and the layer clipping module in the native layer can be dependent on Figure 2 media libraries, three-dimensional graphics processing libraries, and image processing libraries in system libraries. The audio / video stream grabbing module and the virtual screen management module in the server layer can be dependent on Figure 15 display drivers and audio drivers in kernel layers and three-dimensional graphics processing libraries and image processing libraries in system libraries.

[0265] The sink device includes an application (App) layer, an application framework (Framework) layer, a native (Native) layer, and a server (Server) layer. The Framework layer includes a sound system, an input system, and a display system. The sound system is used to play sound after audio decoding. The display system is used to display an interface after video decoding. The input system is used to receive a touch operation of a user. The Native layer includes a video rendering module, an audio / video decoding module, and a reverse input event grabbing module. The audio / video decoding module is used to decode an audio stream or a video stream received from the source device. The video rendering module is used to render and display the decoded video stream. The reverse input event grabbing module is used to grab a reverse input event of a user. The Server layer includes a network module. The network module is used to receive an audio stream or a video stream sent by the source device and send a reverse input event to the source device.

[0266] It should be understood that Figure 2 The functionality implemented by modules in the local layer of the sink device can depend on and Figure 2 Modules or drivers in the system libraries and kernel layer. For example, a video rendering module might depend on... Figure 2 The system library includes 3D graphics processing libraries, image processing libraries, and display drivers in the kernel layer. Audio and video decoding modules may depend on... Figure 2 The system library includes 3D graphics processing libraries, image processing libraries, etc. The reverse input event capture module can depend on... Figure 16 Touch panel (TP) drivers and other hardware-level components.

[0267] Figure 3 A schematic flowchart of method 1600 for displaying source and sink devices in full-screen mode is shown. Method 1600 includes:

[0268] S1601, the source device establishes a wireless connection with sink device 1.

[0269] For example, such as Figure 3 As shown in (a), when mobile phone A detects that the distance between mobile phone A and mobile phone B is less than or equal to a preset distance, mobile phone A and mobile phone B can form a network through near-field wireless connection, thereby establishing a wireless connection between mobile phone A and mobile phone B. For example, mobile phone A and mobile phone B can form an AP network, or mobile phone A and mobile phone B can form a P2P network.

[0270] S1602, after sink device 1 detects the user's first operation, sink device 1 instructs source device to enter full-screen mode.

[0271] For example, such as Figure 11 As shown in (a), phone A is the source device and phone B is the sink device 1. When phone B detects a user swiping to the right, phone B can send an instruction message to phone A, which indicates that phone B wants to enter full-screen mode.

[0272] For example, such as Figure 10 As shown in (b), mobile phone A is the source device and mobile phone B is the sink device 1. When mobile phone B detects that the user drags the control 1102 and partially or completely overlaps with the icon in full-screen mode, mobile phone B can send an instruction message to mobile phone A. This instruction message is used to indicate that mobile phone B wants to enter full-screen mode.

[0273] For example, such as Figure 12As shown in (a) of FIG. 12, the mobile phone B is the source device, and the mobile phone A is the sink device 1. When the mobile phone A detects the user's leftward sliding operation, the mobile phone A can send indication information to the mobile phone B, the indication information being used to indicate that the mobile phone A wants to enter the full-screen mode.

[0274] For example, as shown in (b) of FIG. 12, the mobile phone B is the source device, and the mobile phone A is the sink device 1. When the mobile phone A detects that the control 1201 coincides with or fully coincides with the icon part of the full-screen mode after being dragged by the user, the mobile phone A can send indication information to the mobile phone B, the indication information being used to indicate that the mobile phone A wants to enter the full-screen mode. Figure 17 In the embodiments of the present application, when the sink device 1 detects the user's rightward sliding operation or the sink device 1 detects that the control is dragged rightward to coincide with or fully coincide with the icon part of a certain mode, the sink device can determine that the device on the left side thereof is the source device; when the sink device 1 detects the user's leftward sliding operation or the sink device 1 detects that the control is dragged leftward to coincide with or fully coincide with the icon part of a certain mode, the sink device can determine that the device on the right side thereof is the source device.

[0275] It should be understood that, in the embodiments of the present application, the source device can determine the distance between the source device and the sink device 1 and the orientation information between the source device and the sink device 1 through positioning technologies such as Bluetooth, ultra-wideband (UWB), ultrasonic waves, etc., and the specific implementation process can refer to the prior art, which will not be described herein for the sake of brevity.

[0276] For example, the source device is provided with a Bluetooth / Wi-Fi antenna array (or, the source device is provided with an angle of arrival (AOA) calculation capability), and the sink device 1 is provided with a Bluetooth / Wi-Fi antenna array (or, the sink device 1 is provided with an AOA calculation capability). The source device can calculate the orientation of the sink device 1, and the Bluetooth / Wi-Fi antenna array of the source device can receive the wireless signal of the sink device 1 to calculate the orientation of the sink device 1 according to formulas (1) and (2):

[0277]

[0278]

[0279]

[0280] ​wherein d is the distance between the Bluetooth / Wi-Fi antenna array of the source device and the Bluetooth / Wi-Fi antenna of the sink device 1, is the phase difference between the Bluetooth / Wi-Fi antenna array of the source device and the Bluetooth / Wi-Fi antenna of the sink device 1, λ is the wavelength of the Bluetooth signal sent by the sink device 1, and θ is the angle of arrival. It should be understood that in the embodiments of the present application, the source device calculates the orientation of the sink device 1, and it can also be understood that the source device can calculate the orientation of the line connecting the Bluetooth / Wi-Fi antenna array of the source device and the Bluetooth / Wi-Fi antenna of the sink device 1.

[0281] It should be understood that the sink device 1 can also use the above formulas (1) and (2) to calculate the orientation of the source device.

[0282] In one embodiment, when the sink device 1 detects the first operation of the user, the sink device 1 can send a user datagram protocol (UDP) packet to the source device, and the UDP packet can carry the indication information, which indicates that the sink device 1 wants to enter the full-screen mode. The UDP packet includes the data part of the IP datagram. The data part of the IP datagram can include extensible bits. The sink device 1 and the source device can agree on the content of a certain extensible bit. When a certain extensible bit is 1, the source device can know that the sink device 1 wants to enter the full-screen mode.

[0283] In one embodiment, when the sink device 1 detects the first operation of the user, the sink device 1 can send a transmission control protocol (TCP) packet to the source device, and the TCP packet can carry the indication information, which indicates that the sink device 1 wants to enter the full-screen mode. The TCP packet includes a TCP header and a TCP data part, wherein the TCP header includes a reserved field. The sink device 1 and the source device can agree on the content of a certain reserved field. When a certain reserved field bit is 1, the source device can know that the sink device 1 wants to enter the full-screen mode.

[0284] S1603, in response to receiving the indication of the sink device 1, the source device expands the canvas on the current interface by one time.

[0285] For example, the canvas size originally displayed by the source device is 1080x2340. After the source device receives the indication information from the sink device 1, the source device can determine that the sink device 1 wants to form a dual-screen splicing with the source device, and then the source device expands the canvas size of the main device to (1080x2)x2340, i.e., 2160x2340.

[0286] It should be understood that in the embodiments of the present application, if the source device and the sink device 1 are horizontally distributed (or distributed left and right) after the source device receives the indication information from the sink device 1, the source device can expand the canvas size of the main device to 2160x2340. If the source device determines that the source device and the sink device 1 are vertically distributed (or distributed top and bottom), the source device can also expand the canvas size of the main device to 1080x(2340x2), i.e., 1080x4680. The following will be described by taking the source device and the sink device 1 (and the sink device 2) as horizontally distributed as an example.

[0287] In S1604, the source device crops the doubled canvas, and displays a part of the cropped canvas on the source device and puts another part into the virtual screen (display 1) created by the source device.

[0288] In the embodiments of the present application, the way in which the source device crops the expanded canvas can include but is not limited to a splitting way and a masking way.

[0289] Figure 18This illustrates the process by which the source device segments and cropes the canvas, then displays the cropped canvas on a virtual screen. The layer information for each display is managed by the Android component SurfaceFlinger. Since no application is running on a display, it has no layer information. SurfaceFlinger composites each display sequentially. For example, when SurfaceFlinger composites display0, it crops the layers, with cropping region 1 being [0, 0, 1080, 2340]. The source device can then place the canvas shown in cropping region 1 into display0, allowing it to display. When SurfaceFlinger composites display1, it copies all layer information from display0, crops it again, with cropping region 2 being [1080, 0, 2160, 2340], and shifts the cropped region 2 1080 degrees to the left before placing it into display1. The source device can then project display1 onto the sink device 1.

[0290] Figure 4 This illustrates the process by which the source device crops the canvas using a masking method and displays the cropped canvas on a virtual screen. The source device first determines the area to be cropped based on the distance between the source device and sink device 1 (e.g., ...). Figure 3 (as shown in region 402 in (b)). It should be understood that the process by which the source device determines the pixel value of the region to be cropped can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity. Assume that the pixel value to be cropped determined by the source is x.

[0291] For example, when SurfaceFlinger composites display0, it clips the layers, with clipping region 3 being [0, 0, 1080, 2340]. The source device can place the canvas shown in clipping region 3 into display0, allowing the source device to display display0. When SurfaceFlinger composites display1, it copies all layer information from display0, then clips it, with clipping region 4 being [1080+x, 0, 2160, 2340]. The clipped region is then shifted 1080+x to the left and placed into display1. The source device can then project display1 onto the sink device 1.

[0292] S1605, the source end device projects display1 to sink end device 1.

[0293] In one embodiment, if the screen resolution of sink end device 1 is 1080x2340, when sink end device 1 receives the canvas information corresponding to the cropped region 4 sent by the source end device, the canvas can be displayed in the center of the screen with black borders of size x / 2 on both sides of the screen.

[0294] In one embodiment, when sink end device 1 receives the canvas information corresponding to the cropped region 4 sent by the source end device, the canvas can also be displayed on the left side of the screen with a black border of size x on the right side of the screen.

[0295] In one embodiment, when sink end device 1 receives the canvas information corresponding to the cropped region 4 sent by the source end device, the canvas can also be displayed on the right side of the screen with a black border of size x on the left side of the screen.

[0296] In one embodiment, when sink end device 1 indicates that the source end device wants to join the full screen mode, sink end device 1 can also send the screen resolution information to the source end device. After the source end device learns the screen resolution of sink end device 1 and the distance between the source end device and sink end device 1, the source end device can first expand the canvas size to 2160x2340. The source end device can stretch the expanded canvas horizontally according to the pixel value (e.g., x) that needs to be cropped, and stretch it to (2160+x)x2340. In this way, the cropped region 3 can be [0, 0, 1080, 2340], and the size of the cropped region 4 is [1080+x, 0, 2160+x, 2340]. The source end device can put the cropped region 4 into display1 after shifting it to the left by 1080+x. The source end device can project display1 to sink end device 1. Sink end device 1 does not need to leave black borders on the screen when displaying.

[0297] It should be understood that the process of projecting display1 from the source end device to sink end device 1 can refer to existing projection technology, and for the sake of brevity, will not be described here.

[0298] In one embodiment, in S1602, the sink device 1 may trigger both the source device and the sink device to enter full-screen mode. In this embodiment, the source device may also trigger both the source device and the sink device 1 to enter full-screen mode. For example, when the source device detects a user swiping from the left to the right of the screen, it may send a segmented portion of the image information and an instruction to the sink device 1. This instruction instructs both the source device and the sink device 1 to enter full-screen mode. Upon receiving the segmented portion of the image information and the instruction, the sink device 1 displays the image information.

[0299] S1606, the source device establishes a wireless connection with the sink device 2.

[0300] For example, such as Figure 3 As shown in (a), when mobile phone C is close to mobile phone B, mobile phone B can determine through RSSI ranging that the distance between mobile phone C and mobile phone B is less than or equal to a preset distance. Then, mobile phones A, B, and C can form a network via near-field wireless connection, thus establishing a wireless connection. For example, mobile phones A, B, and C can form an AP network, or they can form a P2P network.

[0301] S1607, when sink device 2 detects the user's second operation, sink device 2 instructs source device to enter full-screen mode.

[0302] For example, such as Figure 19 As shown in (d), phone A is the source device, phone B is the sink device 1, and phone C is the sink device 2. When phone C detects a user swiping to the right, phone C can send an instruction message to phone A, which indicates that phone C wants to enter full-screen mode.

[0303] It should be understood that the process of sink device 2 sending indication information to source device (indication information used to indicate that sink device 2 wants to enter full-screen mode) can be referred to the description of S1602 above, and will not be repeated here.

[0304] In the embodiments of the present application, when the sink device 2 detects the user's rightward sliding operation or the sink device 2 detects that the control is dragged to the right and overlaps or completely overlaps with the icon part of a certain mode, the sink device can determine that the device on its left side is the source device. If the sink device 2 detects that the left side thereof includes multiple devices, the sink device 2 can take the leftmost device as the source device.

[0305] When the sink device 2 detects the user's leftward sliding operation or the sink device 2 detects that the user drags the control to the left and overlaps or completely overlaps with the icon part of a certain mode, the sink device 2 can determine that the device on its right side is the source device. If the sink device 2 detects that the right side thereof includes multiple devices, the sink device 2 can take the rightmost device as the source device.

[0306] In one embodiment, the sink device 2 can also send the distance information between the sink device 1 and the sink device 2 to the source device.

[0307] For example, the sink device 2 can send a UDP data packet to the source device, and the UDP data packet can carry the distance information between the sink device 1 and the sink device 2. The UDP data packet includes the data part of the IP datagram. The data part of the IP datagram can include extensible bits. The sink device 2 and the source device can agree on the content of a certain extensible bit. For example, when a certain extensible bit is 000, the source device can know that the distance between the sink device 1 and the sink device 2 is 1 centimeter; when a certain extensible bit is 001, the source device can know that the distance between the sink device 1 and the sink device 2 is 2 centimeters; when a certain extensible bit is 011, the source device can know that the distance between the sink device 1 and the sink device 2 is 3 centimeters.

[0308] S1608, in response to receiving the indication of the sink device 2, the source device expands the canvas on the current interface by two times.

[0309] For example, the original canvas size displayed by the source device was 1080x2340. When the source device receives the instruction information from the sink device 2, the source device can determine that the sink devices 1 and 2 want to form a three-screen splicing with the source device. Then, the source device expands the main device's display canvas size to (1080x3)x2340, that is, 3240x2340.

[0310] S1609, the source device will crop the canvas after it has been enlarged by two times. The first cropped part is displayed on the source device, the second part is placed in the virtual screen (display1) created by the source device, and the third part is placed in another virtual screen (display2) created by the source device.

[0311] For segmentation methods, such as Figure 20 As shown, when SurfaceFlinger composites display0, it clips the layers, with clipping region 1 being [0, 0, 1080, 2340]. The source device can place the canvas shown in clipping region 1 into display0, allowing the source device to display the canvas in display0. When SurfaceFlinger composites display1, it copies all layer information from display0 and then clips it, with clipping region 2 being [1080, 0, 2160, 2340]. The source device can shift clipping region 2 1080 degrees to the left and place it into display1. The source device can then project the canvas in display1 onto the sink device 1.

[0312] When SurfaceFlinger composites display2, it copies all layer information from display0 and then performs cropping, where cropping region 3 is [2160, 0, 3240, 2340]. The source device can shift cropping region 3 2160 degrees to the left and place it into display2. The source device can then project the canvas from display2 onto the sink device 2.

[0313] For masking methods, such as Figure 4 As shown, the source device first determines the two areas to be clipped based on the distance between the source device and sink device 1, and the distance between sink device 1 and sink device 2 (e.g., Figure 20Region 405 and Region 407 shown in (d) in FIG. 4B). It should be understood that the process of determining the pixel values of the region that needs to be cropped by the source device can refer to the description in the above embodiments, which will not be repeated here for brevity. Assume that the pixel values x and y that need to be cropped are determined by the source device.

[0314] When SurfaceFlinger composites display0, the layer is cropped, where the cropped region 4 is [0, 0, 1080, 2340]. The source device can put the canvas shown in the cropped region 4 into display0, so that the source device can display the canvas in display0.

[0315] When SurfaceFlinger composites display1, all layer information is copied from display0 and then cropped, where the cropped region 5 is [1080+x, 0, 2160+x, 2340]. The source device can put the cropped region 5 into display1 after shifting left by 1080+x. The source device can cast the canvas in display1 to sink device 1.

[0316] When SurfaceFlinger composites display2, all layer information is copied from display0 and then cropped, where the cropped region 6 is [2160+x+y, 0, 3240, 2340]. The source device can put the cropped region 6 into display2 after shifting left by 2160+x+y. The source device can cast the canvas in display2 to sink device 2.

[0317] It should be understood that, Figure 5 The cropped region 5 can also be [1080+x, 0, 2160, 2340]. The source device can put the cropped region 5 into display1 after shifting left by 1080+x. The source device can cast the canvas in display1 to sink device 1.

[0318] The cropped region 6 can also be [2160+y, 0, 3240, 2340]. The source device can put the cropped region 6 into display2 after shifting left by 2160+y. The source device can cast the canvas in display2 to sink device 2.

[0319] S1610, the source device projects display1 onto sink device 1, and displays2 onto sink device 2.

[0320] It should be understood that the process of the source device projecting display1 to the sink device 1 and the process of projecting display2 to the sink device 2 can be referred to the process in S1605 above, and will not be repeated here.

[0321] It should be understood that the process of the source device casting display1 to the sink device 1 and the process of casting display2 to the sink device 2 can refer to existing casting technologies, and will not be elaborated here for the sake of simplicity.

[0322] S1611, sink device 2 detected that the user exited full-screen mode on sink device 2.

[0323] For example, such as Figure 5 As shown in (a), this operation can detect when the user clicks the exit control on mobile phone C.

[0324] S1612, in response to detecting the operation, sink device 2 sends an indication message to source device, which instructs sink device 2 to exit full-screen mode.

[0325] In one embodiment, such as Figure 5 As shown in (a), the exit control can be drawn by the source device and added to the enlarged canvas. When the sink device 2 detects this operation, it can obtain the user's touch event and the corresponding coordinate point. The sink device 2 can then send the touch event and the converted coordinate point to the source device.

[0326] For example, if the source device, sink device 1, and sink device 2 are used for full-screen display in a split manner, then when sink device 2 detects that the user has clicked at position (100, 100) on the screen, the sink device determines the event as a touch event, and the coordinates are converted to (100 + 2160, 100), i.e., (2260, 100). Sink device 2 can then send the touch event and the converted coordinates to the source device. After receiving the touch event and the converted coordinates, the source device can determine that the user clicked the exit control on sink device 2, thus proceeding to step S1613.

[0327] In one embodiment, if the source device, sink device 1, and sink device 2 are used for full-screen display via masking, and the cropping area 6 determined by the source device is [2160+x+y, 0, 3240, 2340], then if sink device 2 detects that the user has clicked at position (100, 100) on the screen, the sink device determines the event as a touch event, and the coordinates are converted to (100+2160+x+y, 100), i.e., (2260+x+y, 100).

[0328] For example, when sink device 2 detects the operation, it can send a UDP packet to source device. This UDP packet may carry the converted coordinate point information, which instructs sink device 2 to exit full-screen mode. The UDP packet includes the data portion of an IP datagram. The data portion of the IP datagram may include expandable bits. Sink device 2 and source device can agree on the content of a certain expandable bit. For example, when a certain expandable bit is 1, it indicates that the touch event is a click event. Alternatively, sink device 2 can use encoding methods such as GBK, ISO8859-1, or Unicode (e.g., UTF-8, UTF-16) to encode the converted coordinate point information. Source device can learn about the touch event detected by sink device 2 and the converted coordinate point information through the information in the expandable bits. Therefore, source device can determine that sink device 2 has detected the user's operation to exit full-screen mode.

[0329] In one embodiment, such as Figure 5 As shown in (a), the exit control can also be drawn by the sink device 2. When the sink device 2 detects that the user clicks the exit control, the sink device 2 can send an instruction message to the source device. This instruction message is used to instruct the sink device 2 to exit the full-screen mode.

[0330] For example, when sink device 2 detects this operation, it can send a UDP packet to source device. This UDP packet may carry the indication information, which instructs sink device 2 to exit full-screen mode. The UDP packet includes the data portion of an IP datagram. The data portion of the IP datagram may include expandable bits. Sink device 2 and source device can agree on the content of a certain expandable bit. When a certain expandable bit is 1, source device can know that sink device 2 has exited full-screen mode.

[0331] S1613, in response to receiving the instruction information, the source device adjusts the canvas size and re-assembles with the sink device 1.

[0332] It should be understood that after receiving this instruction, the source device can perform the operations described in S1603-S1605 above. For the sake of brevity, these will not be elaborated here.

[0333] S1614, sink device 1 detects that the user has exited full-screen mode on sink device 1.

[0334] For example, such as Figure 5 As shown in (b), this operation can detect when the user clicks the exit control on mobile phone B.

[0335] S1615, in response to detecting the operation, sink device 1 sends an indication message to source device, which instructs sink device 1 to exit full-screen mode.

[0336] In one embodiment, such as Figure 21 As shown in (a), the exit control can be drawn and added to the enlarged canvas by the source device. When the sink device 1 detects this operation, it can obtain the user's touch event and the corresponding coordinate point. The sink device 1 can send the touch event and the converted coordinate point to the source device. The source device can determine that the sink device 1 has detected that the user clicked the exit control based on the touch event and the converted coordinate point.

[0337] For example, if the source device, sink device 1, and sink device 2 are used for full-screen display in a split manner, then when sink device 1 detects a user click operation at position (100, 100) on the screen, the sink device determines the touch event as a user click event, and the coordinates are converted to (100+1080, 100), i.e., (1180, 100). Sink device 1 can then send the touch event and the converted coordinates to the source device. After receiving the touch event and the converted coordinates, the source device can determine that the user clicked the exit control on sink device 1, thus proceeding to step S1616.

[0338] In one embodiment, if the source device, sink device 1, and sink device 2 are used for full-screen display via masking, and the cropping area 5 determined by the source device is [1080+x, 0, 2160+x, 2340], then if sink device 2 detects that the user has clicked at position (100, 100) on the screen, the sink device determines the event as a touch event, and the coordinates are converted to (100+1080+x, 100), i.e., (1180+x, 100).

[0339] It should be understood that the process of the sink device 1 sending indication information to the source device can be referred to the description of S1612 above, and will not be repeated here.

[0340] S1616, In response to receiving the instruction information, the source device does not expand the canvas and restores the source device to single-device display.

[0341] In this embodiment, multiple devices can be wirelessly networked and their screens can be spliced ​​together by a preset gesture, thereby achieving a full-screen mode on multiple devices and improving the user experience.

[0342] Figure 8 A schematic flowchart of method 2100 for displaying information in a paginated mode on the source and sink devices is shown. Method 2100 includes:

[0343] S2101, the source device and the sink device establish a wireless connection.

[0344] It should be understood that S2101 can refer to the process of S1601 above, and for the sake of brevity, it will not be repeated here.

[0345] S2102, when the sink device detects the user's third operation, the sink device instructs the source device to enter pagination mode.

[0346] For example, such as Figure 11 As shown in (a), the icons for the pagination mode displayed on phone B can be drawn by phone A. Phone A can add icons for parallel mode, dual-application mode, pagination mode, and an exit control to the cropped canvas after placing it into display1.

[0347] In one embodiment, when mobile phone B detects a tap on the pagination mode, it can obtain the user's touch event and the corresponding coordinate point. Mobile phone B can then send the touch event and the converted coordinate point to mobile phone A. Mobile phone A can then switch from full-screen mode to pagination mode based on the touch event and the converted coordinate point.

[0348] For example, if phone B detects that the user clicked at position (100, 100) on the screen, then phone B determines the event as a touch event and converts the coordinates to (100+1080, 100), i.e., (1180, 100). Phone B can then send the touch event and the converted coordinates to phone A. Upon receiving the touch event and the converted coordinates, phone A can determine that the user clicked on the pagination mode on phone B, thus proceeding to step S2103.

[0349] In one embodiment, mobile phone B can also send information to mobile phone A about the coordinates (e.g., (100, 100)) of the click action detected by the user on mobile phone B's screen. After receiving the coordinate information from mobile phone B, mobile phone A can convert the coordinates to coordinates on a canvas that has been enlarged by a factor of 1 (e.g., (1180, 100)). Thus, mobile phone A can determine that the user clicked the pagination icon on mobile phone B.

[0350] It should be understood that the process of the sink device sending the touch event and the converted coordinates to the source device can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0351] In one embodiment, the icon for the pagination mode displayed on phone B can be drawn by phone B. After receiving the canvas of display1 sent by phone A, phone B can add icons for parallel mode, dual application mode, pagination mode, and an exit control to the canvas. When phone B detects that the user clicks on the pagination mode icon, phone B sends an instruction message to phone A, which indicates that phone B wishes to enter pagination mode.

[0352] For example, when mobile phone B detects that a user has clicked the pagination mode icon, mobile phone B can send a UDP packet to mobile phone A. This UDP packet can carry the indication information, which indicates that mobile phone B wishes to enter pagination mode. The UDP packet includes the data portion of the IP datagram. The data portion of the IP datagram can include expandable bits. Mobile phone B and mobile phone A can agree on the content of a certain expandable bit. When a certain expandable bit is 1, mobile phone A can know that mobile phone B wishes to join pagination mode.

[0353] In one embodiment, such as Figure 22 As shown in (b), when mobile phone B detects that the user drags control 1102 to partially or completely overlap with the control corresponding to the pagination mode, mobile phone B sends an instruction message to mobile phone A. This instruction message is used to indicate that mobile phone B wants to enter the pagination mode.

[0354] In one embodiment, after detecting that the user drags the control 1102 to coincide with the control part corresponding to the page mode or coincide with the whole control part, the mobile phone B can send a UDP packet to the mobile phone A, and the UDP packet can carry the indication information indicating that the mobile phone B wants to enter the page mode. The UDP packet includes a data part of an IP datagram. The data part of the IP datagram can include an extensible bit. The mobile phone A and the mobile phone B can agree on the content of the extensible bit. When the extensible bit is 1, the mobile phone A can know that the mobile phone B wants to enter the page mode.

[0355] S2103, in response to receiving the indication of the sink device, the source device and the sink device perform splicing display through the page mode.

[0356] Figure 8 A schematic diagram of displaying the desktop of the source through the page mode is shown. After the source device receives the indication of the sink device, the source device enlarges the size of the display canvas by one time, and the source adjusts the desktop layout. For example, taking the system of the source device as an Android system, the App of the source device displays the icons of the first page in the area 2201 and displays the icons of the second page in the area 2202 by overwriting the protected void onSizeChanged(int w, int h, int oldw, int oldh) method.

[0357] In the embodiment of the application, the page mode is different from mirroring one desktop and different from two independent desktops. In the page mode, the first page displays the first desktop page, and the second page is a natural extension of the first desktop page, and the second page can not include the function bar at the bottom. Each page in the page mode supports click, sliding and gesture operation, for example, each page can be slid, and the sliding between the two pages is linked.

[0358] For example, taking the system of the source device as an Android system, after the App of the source device listens to the change of the canvas size, the App of the source device judges the current mode by calling the interface getSplicingModeType() shown in Table 1. For example, when the return value is 2, it can be determined that the current mode is the page mode, and the page mode is switched to.

[0359] For example, Table 1 shows the correspondence between the package name, the interface prototype and the return value.

[0360] Table 1

[0361]

[0362] When the app determines that it is currently in pagination mode, it overrides the `protected void onSizeChanged(int w, int h, int oldw, int oldh)` method to display the icons for the first page in area 2201 and the icons for the second page in area 2202. The source device can then shift the canvas in the cropped area 2202 1080 degrees to the left and place it into `display1`, thereby projecting `display1` onto the sink device.

[0363] S2104, the source device will crop half of the canvas after it has been enlarged by one time and display it, and create a virtual screen (display1) to place the cropped half in display1.

[0364] S2105, the source device projects display1 onto the sink device.

[0365] See Figure 8 As shown in (b), the source device (phone A) can display the icon of the first desktop page, and the sink device (phone B) can display the icon of the second desktop page.

[0366] S2106, the source device detects the user's operation of launching the App and loads the content of the second page on the canvas that has been enlarged by one time.

[0367] S2107, the source device will crop half of the canvas after it has been enlarged by one time and display the other half on display1.

[0368] For example, such as Figure 23 As shown in (c), phone A can display the first and second pages (PDF1 / 8 and PDF2 / 8) of App 5 on a canvas that has been enlarged by a factor of two. Phone A can crop the content of the first page (PDF1 / 8) and display it on its screen, and then shift the cropped second page (PDF2 / 8) canvas 1080 degrees to the left and place it in display1. Phone A can then project display1 onto phone B.

[0369] It should be understood that the specific implementation process of S2106-S2107 can refer to the process of S2103-S2104 above. For the sake of brevity, it will not be repeated here.

[0370] S2108, the source device projects display1 onto the sink device.

[0371] For example, the sink device (phone B) can display the content of the second page (PDF2 / 8).

[0372] This application provides a way to enter a pagination mode, in which the application can adjust the layout, which helps to improve the user experience when splicing multiple devices.

[0373] Figure 9 A schematic flowchart of method 2300, which displays source and sink devices in parallel mode, is shown. Method 2300 includes:

[0374] S2301, the source device and the sink device establish a wireless connection.

[0375] It should be understood that S2301 can refer to the process of S1601 above, and for the sake of brevity, it will not be repeated here.

[0376] S2302, when the sink device detects the user's fourth operation, the sink device instructs the source device to enter parallel mode.

[0377] For example, such as Figure 11 As shown in (a), the parallel mode icon displayed on phone B can be drawn by phone A. Phone A can add the parallel mode icon, dual application mode icon, pagination mode icon, and exit control to the cropped canvas after placing it into display1.

[0378] In one embodiment, when mobile phone B detects a tap on the pagination mode, it can obtain the user's touch event and the corresponding coordinate point information. Mobile phone B can then send the touch event and coordinate point information to mobile phone A. Mobile phone A can then switch from full-screen mode to parallel mode based on the touch event and coordinate point information.

[0379] For example, if phone B detects that the user clicked at position (100, 100) on the screen, phone B can determine that the event is a touch event, and the coordinates are converted to (100+1080, 100), i.e. (1180, 100). Phone B can then send the touch event and the converted coordinates to phone A. After receiving the touch event and the converted coordinates, phone A can determine that the user clicked on the parallel mode on phone B, thus proceeding to step S2303.

[0380] In one embodiment, mobile phone B can also send information to mobile phone A about the coordinates (e.g., (100, 100)) of the click action detected by the user on mobile phone B's screen. After receiving the coordinate information from mobile phone B, mobile phone A can convert the coordinates to coordinates on a canvas that has been enlarged by a factor of 1 (e.g., (1180, 100)). Thus, mobile phone A can determine that the user clicked the pagination icon on mobile phone B.

[0381] It should be understood that the process of mobile phone B sending touch events and converted coordinate point information to mobile phone A can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0382] In one embodiment, the icon for the pagination mode displayed on phone B can be drawn by phone B. After receiving the canvas of display1 sent by phone A, phone B can add icons for parallel mode, dual application mode, pagination mode, and an exit control to the canvas. When phone B detects that the user clicks on the parallel mode icon, phone B sends an instruction message to phone A, which indicates that phone B wishes to enter parallel mode.

[0383] For example, when mobile phone B detects that a user has clicked the parallel mode icon, mobile phone B can send a UDP packet to mobile phone A. This UDP packet can carry the indication information, which indicates that mobile phone B wishes to enter parallel mode. The UDP packet includes the data portion of the IP datagram. The data portion of the IP datagram can include expandable bits. Mobile phone B and mobile phone A can agree on the content of a certain expandable bit. When a certain expandable bit is 1, mobile phone A can know that mobile phone B wishes to enter parallel mode.

[0384] For example, such as Figure 9 As shown in (b), when mobile phone B detects that the user drags control 1102 to partially or completely overlap with the control corresponding to the parallel mode, mobile phone B can send an instruction message to mobile phone A. This instruction message is used to indicate that mobile phone B wants to enter the parallel mode.

[0385] S2303, in response to receiving an instruction from the sink device, the source device will double the size of the display canvas.

[0386] S2304, the user has launched the App. The source side displays the App's homepage on half of the canvas, which is now twice the size.

[0387] For example, such as Figure 24As shown in (c), phone A can display the homepage of App 6 on the left half of a canvas that is doubled in size. Phone A can also crop the left half and place it in display0, thus displaying the canvas shown in display0. At this point, since phone A has not yet detected any user interaction with an activity page, the content displayed on the right half is not limited. Phone A can also display its desktop on the right half of the doubled-sized canvas. Phone A can also crop the right half and place it in display1, thus projecting display1 onto phone B. Alternatively, phone A can choose not to project to phone B, and phone B can display its own desktop.

[0388] S2305 When it is detected that a user clicked on an activity page on the home page of the App, the source device displays the activity page on the other half of a canvas that is twice the size.

[0389] Figure 9 This demonstrates the process of opening an active page within an application hierarchy in parallel mode. Figure 9 Taking the GUI shown as an example, when the source device detects that the user clicked on an activity page on the App's homepage, it can display the App's homepage content in area 2401 (which is doubled in size) and the activity page content in area 2402. The source device can crop the doubled canvas, placing the canvas in area 2401 into display0 and displaying the canvas in display0 on the screen. The source device can also shift the canvas in area 2402 1080 degrees to the left and place it into display1, then project display1 onto the sink device, allowing the sink device to display the activity page.

[0390] In this embodiment, when the source device detects that a user has launched an app, it can by default display the app's homepage on the left half of a canvas that is twice the size of its original size. When the source device detects that a user has clicked on an activity page, it can call an API to display that activity page on the right half.

[0391] For example, Table 2 shows the correspondence between package name, interface prototype and return value.

[0392] Table 2

[0393]

[0394] The source device determines its current mode by calling the `getSplicingModeType()` interface shown in Table 1. For example, a return value of 3 indicates that it is in parallel mode. When the source device detects that a user clicks on an active page in the app, the app can call the `startActivity()` interface shown in Table 2, setting the position of the active page in the Intent's extended field "position". For example, if the current device detects that a user clicks on active page 1 in the app and wants it to launch on another device (located to the right of the current device), it can write `intent.putExtra("position",RIGHT)` in the Intent and then call `startActivity(Intent)` to display active page 1 on the other device.

[0395] For example, such as Figure 25 As shown in (c), when phone A launches App6, App6 can call the getSplicingModeType() interface to determine if it is currently in parallel mode. When phone A detects that the user clicks on the chat history between the user and the contact "Dad", App6 determines that it needs to launch the activity page corresponding to the chat interface between the user and "Dad". Phone A determines that phone B is located to the right of phone A, so phone A can write intent.putExtra("position",RIGHT) in an Intent, and then call startActivity(Intent) to display the activity page corresponding to the chat interface between the user and "Dad" on device B. Phone A can crop the canvas after it has been enlarged by one time, and display the canvas corresponding to the chat list between the user and multiple contacts on the display screen; phone A can put the cropped activity page corresponding to the chat interface between the user and "Dad" into display1, and project display1 onto phone B. This allows phone B to display the activity page.

[0396] S2306, the source device crops the canvas after it has been enlarged by one time, displays one half of the cropped half on the display screen, and displays the other half on display1.

[0397] S2307, the source device projects the content of display1 to the sink device.

[0398] This application provides a method for displaying the hierarchical activity pages of an App in parallel mode. In parallel mode, multiple activity pages are displayed on multiple devices, resulting in a better user experience when multiple devices are connected.

[0399] Figure 7 A schematic flowchart of a method 2500 for displaying a dual-application mode by a source device and a sink device is shown. The method 2500 comprises:

[0400] S2501, the source device and the sink device establish a wireless connection.

[0401] It should be understood that S2501 can refer to the process of S1601 described above, and for brevity, will not be repeated here.

[0402] S2502, after the sink device detects the fifth operation of the user, the sink device indicates to the source device to enter the dual-application mode.

[0403] For example, as shown in (a) of FIG. 10, the icon of the dual-application mode displayed on the phone B can be drawn by the phone A. The phone A can add the icon of the parallel mode, the icon of the dual-application mode, the icon of the paging mode and the exit control on the cropped canvas after putting the cropped canvas into the display 1. Figure 7

[0404] In one embodiment, when the phone B detects the operation of clicking the dual-application mode, the phone B can obtain the touch event of the user and the corresponding coordinate point. The phone B can send the touch event and the converted coordinate point to the phone A. The phone A can switch from the full-screen mode to the dual-application mode according to the touch event and the coordinate point information.

[0405] For example, the phone B detects that the user clicks the operation at the position (100, 100) on the screen, and the phone B can determine that the touch event is a click event and convert the coordinate point to (100+1080, 100), i.e. (1180, 100). Thus, the phone B can send the touch event and the converted coordinate point to the phone A. The phone A can determine that the user clicks the icon of the dual-application mode on the phone B after receiving the touch event and the converted coordinate point, and then proceed to S2503.

[0406] In one embodiment, the phone B can also send the coordinate point (e.g. (100, 100)) information of the click operation detected by the user on the screen of the phone B to the phone A. The phone A can convert the coordinate point to the coordinate point on the canvas enlarged by 1 times (e.g. (1180, 100)) after receiving the coordinate information sent by the phone B. Thus, the phone A can determine that the user clicks the icon of the dual-application mode on the phone B.

[0407] It should be understood that the process of the phone B sending the touch event and the coordinate point information to the phone A can refer to the description in the above embodiments, and for brevity, will not be repeated here. ​

[0408] In one embodiment, the icon of the dual-application mode displayed on the phone B can be drawn by the phone B. After receiving the canvas of displayl sent by the phone A, the phone B can add the icon of the parallel mode, the icon of the dual-application mode, the icon of the page mode and the exit control on the canvas. When the phone B detects the operation of the user clicking the icon of the dual-application mode, the phone B sends the indication information to the phone A, which indicates that the phone B wants to enter the dual-application mode.

[0409] For example, when the phone B detects the operation of the user clicking the icon of the dual-application mode, the phone B can send a UDP packet to the phone A, which carries the indication information indicating that the phone B wants to enter the dual-application mode. The UDP packet includes the data part of the IP datagram. The data part of the IP datagram can include extensible bits. The phone B and the phone A can agree on the content of a certain extensible bit. When the certain extensible bit is 1, the phone A can know that the phone B wants to enter the dual-application mode.

[0410] S2503, in response to receiving the indication of the sink device, the source device creates a virtual screen (display) and puts the canvas corresponding to the desktop of the source device into the display.

[0411] S2504, the source device screens the display to the sink device.

[0412] In one embodiment, taking the source device as an Android system for example, after receiving the indication of the sink device wanting to enter the dual-application mode, the source device can create a display. The source device can use the second desktop (second Launcher) mechanism of Android to start the second Launcher on the display, and screen the display to the sink device, so that the sink device can display the desktop of the source device.

[0413] For example, as shown in (b) of FIG. 10, Figure 7 After receiving the indication of the phone B wanting to enter the dual-application mode, the phone A can start the desktop of the phone A (including the network mode in the status bar, the Wi-Fi connection state, the battery capacity, the weather information, the date information and the icons of App1-App12) on the display, and screen the display to the phone B, so that the phone B can also display the desktop of the phone A.

[0414] S2505: When the sink device detects that the user has clicked the icon of the first application, it sends the touch event and coordinate point information to the source device.

[0415] It should be understood that the process of the sink device sending touch events and coordinate point information to the source device can be referred to the description in the above embodiments, and will not be repeated here for the sake of brevity.

[0416] S2506, in response to receiving the touch event and the coordinate point information, the source device can launch the first application and run the first application in the display.

[0417] For example, such as Figure 7 As shown in (c), when phone B detects that the user clicked the App4 icon, phone B can send the touch event and the coordinate information of the user's click point collected by phone B to phone A. After receiving the touch event and coordinate information, phone A can determine that phone B detected that the user clicked the screen and the corresponding coordinate information. Thus, phone A can determine that the user clicked the App4 icon on phone B. Phone A can then launch App4 and run it in the display, and then project the display onto phone B.

[0418] S2507, the source device projects the display onto the sink device.

[0419] like Figure 26 As described in (d), mobile phone B can display the interface of App4.

[0420] It should be understood that the display process of the source device when it detects the user's operation can be referred to the description in the existing technology, and will not be repeated here for the sake of brevity.

[0421] This application provides a method for displaying different applications of the source device on the source device and the sink device respectively through a dual application mode, avoiding the process of users switching back and forth between different applications on the source device and bringing a better user experience when splicing multiple devices.

[0422] Figure 26 A schematic flowchart of a splicing display method 2600 provided in an embodiment of this application is shown. This method can be executed by a first electronic device and a second electronic device. The first electronic device can be the aforementioned source device, and the second electronic device can be the aforementioned sink device. The first electronic device can communicate with the second electronic device via a short-range wireless connection. Figure 3 As shown, the method 2600 includes:

[0423] S2601, the first electronic device displays a first interface.

[0424] In one embodiment, the first interface can be a desktop of the first electronic device, or a display interface of an application in the first electronic device, or a lock screen interface of the first electronic device. The type of the first interface is not limited in the embodiments of the present application.

[0425] For example, as shown in (a) of FIG. 13, the first interface can be a display interface of a video application on the phone A. Figure 18

[0426] S2602, the second electronic device sends first indication information to the first electronic device in response to detecting the first input of the user, the first indication information being used to instruct the first electronic device and the second electronic device to perform splicing display.

[0427] In one embodiment, the splicing display includes the splicing display by the full screen mode, the splicing display by the page mode, the splicing display by the parallel mode, or the splicing display by the dual application mode.

[0428] It should be understood that the implementation process of the second electronic device sending the first indication information to the first electronic device can refer to the description in the above embodiments, which will not be repeated here.

[0429] S2603, the first electronic device displays first part image information and sends second part image information to the second electronic device in response to receiving the first indication information, the first part image information and the second part image information being associated with the first interface according to the orientation information of the first electronic device and the second electronic device.

[0430] In one embodiment, if the first electronic device determines to perform splicing display with the second electronic device by the full screen mode, the first electronic device can expand the image information corresponding to the first interface in response to receiving the first indication information; the first electronic device can split the expanded image information to obtain the first part image information and the second part image information. In other words, the first part image information and the second part image information are obtained by splitting the first interface after being expanded by the first electronic device, so the first part image information and the second part image information are associated with the first interface.

[0431] ​In one embodiment, the first electronic device receives first information sent by the second electronic device before zooming the image information corresponding to the first interface, the first information being used to indicate the size of the display screen of the second electronic device; and the first electronic device zooms the image information corresponding to the first interface according to the number of electronic devices for display in mosaic, the size of the display screen of the first electronic device, and the size of the display screen of the second electronic device.

[0432] For example, if only the first electronic device and the second electronic device are used for display in mosaic, the first electronic device can determine to zoom the image information corresponding to the first interface by 1 times. If the size of the first electronic device in the first direction is A, the size of the first electronic device in the second direction is B, and the size of the second electronic device in the first direction is A, the size of the second electronic device in the second direction is B. When the first electronic device and the second electronic device are arranged along the first direction, the first electronic device can zoom the image information corresponding to the first interface in the first direction by 2A, and keep the size of the image information corresponding to the first interface in the second direction unchanged.

[0433] It should be understood that the size of the first electronic device in the first direction can be the size of the display screen of the first electronic device in the first direction, or the size of the first interface in the first direction; and the size of the first electronic device in the second direction can be the size of the display screen of the first electronic device in the second direction, or the size of the first interface in the second direction.

[0434] It should also be understood that the first direction and the second direction can be perpendicular. If the first direction is a horizontal direction, the second direction can be a vertical direction; if the first direction is a vertical direction, the second direction can be a horizontal direction.

[0435] In one embodiment, if the first electronic device has a size of A in a first direction and a size of B in a second direction, and the second electronic device has a size of C in the first direction and a size of D in the second direction, where A is greater than C and B is greater than D, then the first electronic device can expand the image information corresponding to the first interface in the first direction to a size of 2A, and keep the size of the image information corresponding to the first interface in the second direction unchanged. The first electronic device can split the expanded canvas (e.g., into two regions of equal size), and the first electronic device can reduce the region to be sent to the second electronic device by a uniform ratio. For example, if A / B is less than C / D, then the first electronic device can reduce the region to be sent to the second electronic device to a size of A*D / B in the first direction and a size of D in the second direction. Thus, the first electronic device can send the reduced region to the second electronic device, so that the second electronic device displays the canvas in the reduced region. For another example, if A / B is greater than C / D, then the first electronic device can reduce the region to be sent to the second electronic device to a size of C in the first direction and a size of B*C / A in the second direction. Thus, the first electronic device can send the reduced region to the second electronic device, so that the second electronic device displays the canvas in the reduced region.

[0436] In one embodiment, the display screen of the first electronic device has a size of a first size in a first direction and a size of a second size in a second direction, the display screen of the second electronic device has a size of a third size in the first direction, the first direction and the second direction are perpendicular, and when the first electronic device and the second electronic device are displayed in a spliced manner along the first direction, the first electronic device expands the image information corresponding to the first interface in the first direction to a fourth size and keeps the size of the image information corresponding to the first interface in the second direction unchanged, the fourth size being the sum of the first size and the third size.

[0437] In one embodiment, the expanded image information is further split into third image information, the third image information being located between the first image information and the second image information, and the third image information being determined by the first electronic device according to the distance between the first electronic device and the second electronic device.

[0438] For example, the third image information can be a region that needs to be cropped. Figure 8 For example, the third image information can be a region that needs to be cropped.

[0439] In one embodiment, the first interface displays the first page of a first document. When the first electronic device determines that the first document includes multiple pages of content, it can display the first page of content and send the second page of content of the first document to the second electronic device. The second electronic device displays the second page of content in response to receiving it.

[0440] In this embodiment, if the first electronic device determines that the second electronic device is to display the content in a paginated manner, the first electronic device can send the second page content of the first file to the second electronic device while displaying the first page content of the first file, thereby causing the second electronic device to display the second page content. At this time, the first portion of image information can be the first page content, and the second portion of image information can be the second page content; both the first portion of image information and the second portion of image information are associated with the first file displayed on the first interface.

[0441] For example, if a first electronic device is displaying the first page of a document, and the document comprises multiple pages, a second electronic device displays its desktop. When the second electronic device detects a first input from the user, it can send a first instruction to the first electronic device. This first instruction can instruct the first and second electronic devices to display the document in a paginated manner. If the first electronic device determines that the document and the second electronic device are to be displayed in a paginated manner, then the first electronic device can continue displaying the first page of the document and can send the second page of the document to the second electronic device, thereby causing the second electronic device to display the second page.

[0442] In one embodiment, the first document further includes a third page and a fourth page; the first electronic device displays the third page and sends the fourth page to the second electronic device in response to detecting a second input from a user; the second electronic device displays the fourth page in response to receiving the fourth page.

[0443] For example, such as Figure 8 As shown in (c), the PDF comprises 8 pages. Currently, the first electronic device and the second electronic device display the first page and the second page respectively in pagination mode. When mobile phone A detects a user's page-turning operation (e.g., mobile phone A detects the user swiping up from the bottom of the screen, or detecting the user swiping from the right side of the screen to the left side), mobile phone A can display the third page of the PDF and can send the fourth page of the PDF to mobile phone B, thereby causing mobile phone B to display the fourth page.

[0444] In one embodiment, the first document further includes a third page and a fourth page. In response to detecting a third input from a user, the second electronic device sends a second instruction to the first electronic device, the second instruction indicating that the second electronic device has detected the third input. In response to receiving the second instruction, the first electronic device displays the third page and sends the fourth page to the second electronic device. In response to receiving the fourth page, the second electronic device displays the fourth page.

[0445] For example, such as Figure 7 As shown in (c), the PDF comprises 8 pages. Currently, the first electronic device and the second electronic device display the first page and the second page respectively in pagination mode. When mobile phone B detects a user's page-turning operation (e.g., mobile phone B detects a user swiping up from the bottom of the screen, or a user swiping from the right side of the screen to the left), mobile phone B can send a corresponding touch event (e.g., a swipe event) to mobile phone A. Mobile phone A can determine from the touch event sent by mobile phone B that mobile phone B has detected a user's page-turning operation. At this time, mobile phone A can be triggered to display the third page of the PDF and send the fourth page of the PDF to mobile phone B, thereby causing mobile phone B to display the fourth page.

[0446] In one embodiment, the first interface is the desktop of the first electronic device. In response to receiving the first instruction information, the first electronic device displays the desktop and sends the image information corresponding to the desktop to the second electronic device. In response to receiving the image information corresponding to the desktop, the second electronic device displays the desktop.

[0447] For example, the first electronic device can display its desktop. When the second electronic device detects the user's first input, it can send a first instruction to the first electronic device, instructing the first and second electronic devices to display their desktops together. If the first electronic device determines that it and the second electronic device are displaying their desktops together in a dual-application mode, the first electronic device can continue to display its desktop and send image information corresponding to its desktop to the second electronic device, thereby causing the second electronic device to display the desktop of the first electronic device.

[0448] In one embodiment, the desktop includes an icon of a first application program, the second electronic device sends third indication information to the first electronic device in response to a fourth input of a user, the third indication information being used to indicate that the second electronic device detects the fourth input, the fourth input being an input to the icon of the first application program, the first electronic device sends image information corresponding to a display interface of the first application program to the second electronic device in response to receiving the third indication information, and the second electronic device displays the display interface of the first application program in response to receiving the image information corresponding to the display interface of the first application program.

[0449] For example, as shown in (c) of FIG. 1, when the phone B detects that the user clicks the icon of App4, the phone B can send a corresponding touch event (e.g., a click event) to the phone A. In response to receiving the touch event sent by the phone B, the phone A can determine that the phone B detects that the user clicks the icon of App4, and thus the phone A can send image information corresponding to the home page of App4 to the phone B, so that the phone B displays the home page of App4. Figure 9

[0450] In one embodiment, if the phone A does not start App4 in the background when the phone A receives the touch event sent by the phone B, the phone A can send image information corresponding to the home page of App4 to the phone B; if the phone A has started App4 in the background and is located at a certain display interface of App4 when the phone A receives the touch event sent by the phone B, the phone A can send image information corresponding to the display interface of App4 to the phone B, so that the phone B displays the display interface of App4.

[0451] In one embodiment, the desktop further includes an icon of a second application program, and the first electronic device displays a display interface of the second application program in response to detecting a fifth input of a user to the icon of the second application program.

[0452] In one embodiment, the first interface is a display interface of a third application program, the display interface includes a first interface element, the first interface element is associated with a second interface of the third application program, the first part of image information is image information corresponding to the first interface, and the second part of image information is image information corresponding to the first interface; or the first part of image information is image information corresponding to the first interface, and the second part of image information is image information corresponding to the second interface.

[0453] ​In an embodiment of the present application, if the first electronic device determines that the second electronic device is to be in parallel mode, the first electronic device can send image information corresponding to a current display interface of the first electronic device to the second electronic device, or the first electronic device can select an interface element on the current display interface, the interface element corresponding to another display interface, and the first electronic device can send image information corresponding to the other display interface to the second electronic device. The current display interface of the first electronic device can be a first-level page, and the other display interface can be a second-level page, and the first-level page and the second-level page are associated.

[0454] For example, the first electronic device can display a display interface 1 of an application, the display interface 1 including an interface element 1, the interface element 1 being associated with a display interface 2 of the application. When the second electronic device detects a first input of a user, the second electronic device can send first indication information to the first electronic device, the first indication information indicating that the first electronic device and the second electronic device are to be in parallel mode. If the first electronic device determines that the second electronic device is to be in parallel mode, the first electronic device can continue to display the display interface 1 and send image information of the display interface 1 to the second electronic device, so that the second electronic device displays the display interface 1. Alternatively, the first electronic device can continue to display the display interface 1 and send image information of the display interface 2 to the second electronic device, so that the second electronic device displays the display interface 2.

[0455] In an embodiment, the interface element 1 can be any interface element on the display interface 1 that is associated with another display interface, or the interface element 1 can be an interface element in a preset direction (for example, the upper left corner).

[0456] For example, the first electronic device can display a desktop of the first electronic device. When the second electronic device detects a first input of a user, the second electronic device can send first indication information to the first electronic device, the first indication information indicating that the first electronic device and the second electronic device are to be in parallel mode. If the first electronic device determines that the second electronic device is to be in parallel mode, the first electronic device can continue to display the desktop of the first electronic device and send image information of the desktop of the first electronic device to the second electronic device, so that the second electronic device displays the desktop of the first electronic device. Alternatively, the first electronic device can continue to display the desktop of the first electronic device and not send any image information to the second electronic device, and the second electronic device can continue to display a current interface of the second electronic device.

[0457] In one embodiment, the first interface further includes a second interface element associated with the third interface of the third application. The first electronic device is further configured to, in response to detecting a sixth input from the user to the second interface element, display image information corresponding to the first interface and send image information corresponding to the third interface to the second electronic device; the second electronic device is further configured to, in response to receiving image information corresponding to the third interface, display the third interface.

[0458] For example, such as Figure 9 As shown in (c), the second interface element can be the chat history between the user and the contact's father, and this second interface element is associated with the chat interface between the user and the contact's father. When phone A detects that the user clicks on the chat history with the contact's father, phone A can continue to display the current interface and can send the chat interface between the user and the contact's father to phone B, thereby causing phone B to display the chat interface between the user and the contact's father.

[0459] In one embodiment, the third interface includes a third interface element associated with a fourth interface of the third application. In response to detecting a seventh input from a user, the second electronic device sends a fourth indication message to the first electronic device, indicating that the second electronic device has detected the seventh input, which is an input to the third interface element. In response to receiving the fourth indication message, the first electronic device displays the fourth interface and sends image information corresponding to the third interface to the second electronic device. In response to receiving the image information corresponding to the third interface, the second electronic device displays the third interface.

[0460] For example, such as Figure 27 As shown in (d), when mobile phone B detects a user's click operation, mobile phone B can send a corresponding touch event (e.g., a click event) to mobile phone A. Upon receiving this touch event, mobile phone A can determine that mobile phone B has detected that the user clicked the video call control, thereby triggering mobile phone A to display the video call interface between the user and their contact, "Dad." Simultaneously, mobile phone A can continue to send the chat interface between the user and their contact, "Dad," to mobile phone B, causing mobile phone B to continue displaying the chat interface between the user and their contact, "Dad."

[0461] S2604, the second electronic device displays the second part of the image information in response to receiving the second part of the image information.

[0462] In one embodiment, the second electronic device displays the first control while displaying the second portion of the image information;

[0463] In response to detecting the eighth input of the user for the first control, a fifth interface is displayed and fifth indication information is sent to the first electronic device, the fifth indication information being used to instruct the first electronic device and the second electronic device to exit the spliced display, the fifth interface being a display interface before the first input is detected by the second electronic device.

[0464] Figure 26 A schematic block diagram of the apparatus 2700 provided by the embodiments of the present application is shown. The apparatus 2700 can be arranged in the first electronic device in the above Figure 28 The apparatus 2700 includes: a display unit 2710, configured to display a first interface; a receiving unit 2720, configured to receive first indication information sent by a second electronic device, the first indication information being used to instruct the apparatus and the second electronic device to perform spliced display; the display unit 2710 is further configured to, in response to receiving the first indication information, display first partial image information according to orientation information of the first electronic device and the second electronic device; and a sending unit 2730, configured to send second partial image information to the second electronic device, wherein the first partial image information and the second partial image information are associated with the first interface.

[0465] Figure 26 A schematic block diagram of the apparatus 2800 provided by the embodiments of the present application is shown. The apparatus 2800 can be arranged in the second electronic device in the above Figure 29 The apparatus 2800 includes: a detecting unit 2810, configured to detect a first input of a user; a sending unit 2820, configured to, in response to the first input, send first indication information to a first electronic device, the first indication information being used to instruct the first electronic device and the apparatus to perform spliced display; a receiving unit 2830, configured to receive second partial image information sent by the first electronic device, the second partial image information being associated with a first interface, the first interface being a display interface when the first indication information is received by the first electronic device; and a display unit 2840, configured to, in response to receiving the second partial image information, display the second partial image information.

[0466] Figure 29 A schematic structural diagram of an electronic device 2900 provided by the embodiments of the present application is shown. As shown in ​ The electronic device includes: one or more processors 2910, one or more memories 2920, the one or more memories 2920 storing one or more computer programs, the one or more computer programs including instructions. When the instructions are run by the one or more processors 2910, the first electronic device or the second electronic device performs the technical solutions in the above embodiments.

[0467] The embodiment of the present application provides a system, comprising a first electronic device and a second electronic device, and the system is used for executing the technical scheme in the above embodiment. The implementation principle and technical effect are similar to the above method-related embodiment, and details are not repeated here.

[0468] The embodiment of the present application provides a computer program product, when the computer program product runs in the first electronic device (or, source end device), so that the first electronic device executes the technical scheme in the above embodiment. The implementation principle and technical effect are similar to the above method-related embodiment, and details are not repeated here.

[0469] The embodiment of the present application provides a computer program product, when the computer program product runs in the second electronic device (or, sink end device), so that the second electronic device executes the technical scheme in the above embodiment. The implementation principle and technical effect are similar to the above method-related embodiment, and details are not repeated here.

[0470] The embodiment of the present application provides a readable storage medium, the readable storage medium contains instructions, when the instructions run in the first electronic device (or, source end device), so that the first electronic device executes the technical scheme in the above embodiment. The implementation principle and technical effect are similar, and details are not repeated here.

[0471] The embodiment of the present application provides a readable storage medium, the readable storage medium contains instructions, when the instructions run in the second electronic device (or, sink end device), so that the second electronic device executes the technical scheme in the above embodiment. The implementation principle and technical effect are similar, and details are not repeated here.

[0472] The embodiment of the present application provides a chip, the chip is used for executing instructions, when the chip runs, executes the technical scheme in the above embodiment. The implementation principle and technical effect are similar, and details are not repeated here.

[0473] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical scheme. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0474] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, and details are not repeated here.

[0475] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0476] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0477] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into one unit.

[0478] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0479] The above is merely specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A splicing display system, characterized in that, The system includes a first electronic device and a second electronic device, wherein the first electronic device communicates with the second electronic device via a short-range wireless connection. The first electronic device is used to display the first interface; The second electronic device is configured to send a first instruction message to the first electronic device in response to detecting a first input from a user, the first instruction message being used to instruct the first electronic device and the second electronic device to perform splicing display; The first electronic device is further configured to, in response to receiving the first instruction information, display a first portion of image information and send a second portion of image information to the second electronic device based on the orientation information of the first electronic device and the second electronic device, wherein the first portion of image information and the second portion of image information are associated with the first interface; The second electronic device is further configured to display the second portion of the image information in response to receiving the second portion of the image information; In the case where the first electronic device and the second electronic device are spliced ​​together in a first mode, the first interface is the display interface of the third application. The display interface includes a first interface element, which is associated with the second interface of the third application. The first interface and the second interface are hierarchical pages to each other. The first part of the image information is the image information corresponding to the first interface, and the second part of the image information is the image information corresponding to the second interface. The first mode is activated based on receiving a selection operation on the first icon, and at least one of the first electronic device and the second electronic device displays the first icon.

2. The system according to claim 1, characterized in that, When the first electronic device and the second electronic device are spliced ​​together in a second mode, the first electronic device is further configured to: receive first information sent by the second electronic device, wherein the first information is used to indicate the size of the display screen of the second electronic device; The first electronic device is specifically used to: enlarge and segment the image information corresponding to the first interface according to the number of electronic devices for splicing display, the size of the display screen of the first electronic device and the size of the display screen of the second electronic device, to obtain the first part of image information and the second part of image information; The second mode is activated based on receiving a selection operation for the second icon, and at least one of the first electronic device and the second electronic device displays the second icon.

3. The system according to claim 2, characterized in that, The first electronic device has a display screen with a first dimension in a first direction and a second dimension in a second direction; the second electronic device has a display screen with a third dimension in a first direction; the first direction and the second direction are perpendicular; and the first electronic device is specifically used for: When the first electronic device and the second electronic device are spliced ​​together along the first direction, the first electronic device expands the size of the image information corresponding to the first interface in the first direction to a fourth size while keeping the size of the image information corresponding to the first interface in the second direction unchanged. The fourth size is the sum of the first size and the third size.

4. The system according to claim 2 or 3, characterized in that, After segmenting the enlarged image information, a third part of image information is obtained. The third part of image information is located between the first part of image information and the second part of image information. The third part of image information is determined by the first electronic device based on the distance between the first electronic device and the second electronic device.

5. The system according to claim 1, characterized in that, The first interface displays the first page of the first file. When the first electronic device and the second electronic device are displayed together via a third mode, the first electronic device is specifically used for: When it is determined that the first file contains multiple pages of content, the first page of content is displayed and the second page of content of the first file is sent to the second electronic device; The second electronic device is specifically used to: display the content of the second page in response to receiving the content of the second page; The third mode is activated upon receiving a selection operation for a third icon, and at least one of the first electronic device and the second electronic device displays the third icon.

6. The system according to claim 5, characterized in that, The first document also includes the contents of the third page and the fourth page. The first electronic device is further configured to, in response to detecting a second input from a user, display the third page of content and send the fourth page of content to the second electronic device; The second electronic device is further configured to display the fourth page content in response to receiving the fourth page content.

7. The system according to claim 5, characterized in that, The first document also includes the contents of the third page and the fourth page. The second electronic device is further configured to send a second indication message to the first electronic device in response to detecting a third input from the user, the second indication message being used to indicate that the second electronic device has detected the third input; The first electronic device is further configured to, in response to receiving the second instruction information, display the third page content and send the fourth page content to the second electronic device; The second electronic device is further configured to display the fourth page content in response to receiving the fourth page content.

8. The system according to claim 1, characterized in that, The first interface is the desktop of the first electronic device. When the first electronic device and the second electronic device are displayed together in a fourth mode, the first electronic device is specifically used for: In response to receiving the first instruction information, the desktop is displayed and image information corresponding to the desktop is sent to the second electronic device; The second electronic device is specifically used to: display the desktop in response to receiving image information corresponding to the desktop; The fourth mode is activated based on receiving a selection operation for the fourth icon, and at least one of the first electronic device and the second electronic device displays the fourth icon.

9. The system according to claim 8, characterized in that, The desktop includes an icon for the first application. The second electronic device is further configured to send a third indication message to the first electronic device in response to a fourth input from the user, the third indication message being used to indicate that the second electronic device has detected the fourth input, the fourth input being an input to the icon of the first application; The first electronic device is further configured to send image information corresponding to the display interface of the first application to the second electronic device in response to receiving the third indication information; The second electronic device is further configured to display the display interface of the first application in response to receiving image information corresponding to the display interface of the first application.

10. The system according to claim 8 or 9, characterized in that, The desktop also includes icons for a second application. The first electronic device is further configured to display the display interface of the second application in response to detecting a fifth input from the user to the icon of the second application.

11. The system according to any one of claims 1 to 3, characterized in that, The first interface also includes a second interface element, which is associated with the third interface of the third application. The first electronic device is further configured to, in response to detecting a sixth input from the user to the second interface element, display image information corresponding to the first interface and send image information corresponding to the third interface to the second electronic device; The second electronic device is further configured to display the third interface in response to receiving image information corresponding to the third interface.

12. The system according to claim 11, characterized in that, The third interface includes third interface elements, and these third interface elements are associated with the fourth interface of the third application. The second electronic device is further configured to send a fourth indication message to the first electronic device in response to detecting a seventh input from the user, the fourth indication message being used to instruct the second electronic device to detect the seventh input, the seventh input being an input to the third interface element; The first electronic device is further configured to, in response to receiving the fourth instruction information, display the fourth interface and send the image information corresponding to the third interface to the second electronic device; The second electronic device is further configured to display the third interface in response to receiving image information corresponding to the third interface.

13. The system according to any one of claims 1 to 3, characterized in that, The second electronic device is also used to display the first control while displaying the second portion of the image information; In response to detecting an eighth input from the user to the first control, a fifth interface is displayed and a fifth instruction message is sent to the first electronic device. The fifth instruction message is used to instruct the first electronic device and the second electronic device to exit the splicing display. The fifth interface is the display interface of the second electronic device before the first input is detected.

14. A method for splicing displays, characterized in that, The method is applied to a first electronic device, which communicates with a second electronic device via a near-field wireless connection, and the method includes: The first electronic device displays the first interface; The first electronic device receives a first instruction information sent by the second electronic device, the first instruction information being used to instruct the first electronic device and the second electronic device to perform splicing display; In response to receiving the first instruction information, the first electronic device displays a first portion of image information and sends a second portion of image information to the second electronic device based on the location information of the first electronic device and the second electronic device. The first portion of image information and the second portion of image information are associated with the first interface. Wherein, when the first electronic device and the second electronic device are spliced ​​together in a first mode, the first interface is the display interface of the third application. The display interface includes a first interface element, which is associated with the second interface of the third application. The first interface and the second interface are hierarchical pages to each other. The first part of the image information is the image information corresponding to the first interface, and the second part of the image information is the image information corresponding to the second interface; the first mode is activated based on receiving a selection operation on the first icon, and at least one of the first electronic device and the second electronic device displays the first icon.

15. The method according to claim 14, characterized in that, The method further includes: When the first electronic device and the second electronic device are spliced ​​together in the second mode, the first electronic device receives first information sent by the second electronic device, the first information being used to indicate the size of the display screen of the second electronic device; Before displaying the first portion of image information and sending the second portion of image information to the second electronic device, the method includes: The first electronic device enlarges and segments the image information corresponding to the first interface based on the number of electronic devices used for splicing display, the size of the display screen of the first electronic device, and the size of the display screen of the second electronic device, to obtain the first part of image information and the second part of image information. The second mode is activated based on receiving a selection operation for the second icon, and at least one of the first electronic device and the second electronic device displays the second icon.

16. The method according to claim 15, characterized in that, The display screen of the first electronic device has a first size in a first direction and a second size in a second direction; the display screen of the second electronic device has a third size in a first direction; the first direction and the second direction are perpendicular; the first electronic device enlarges the image information corresponding to the first interface according to the number of electronic devices used for splicing display, the size of the display screen of the first electronic device, and the size of the display screen of the second electronic device, including: When the first electronic device and the second electronic device are spliced ​​together along the first direction, the first electronic device expands the size of the image information corresponding to the first interface in the first direction to a fourth size while keeping the size of the image information corresponding to the first interface in the second direction unchanged. The fourth size is the sum of the first size and the third size.

17. The method according to claim 15 or 16, characterized in that, After segmenting the enlarged image information, a third part of image information is obtained. The third part of image information is located between the first part of image information and the second part of image information. The third part of image information is determined by the first electronic device based on the distance between the first electronic device and the second electronic device.

18. The method according to claim 14, characterized in that, The first interface displays the first page of the first file. When the first electronic device and the second electronic device are displayed in a spliced ​​manner using a third mode, the process of displaying the first portion of image information and sending the second portion of image information to the second electronic device includes: When it is determined that the first file includes multiple pages of content, the first page of content is displayed and the second page of content of the first file is sent to the second electronic device. The third mode is activated based on receiving a selection operation on a third icon, and at least one of the first electronic device and the second electronic device displays the third icon.

19. The method according to claim 18, characterized in that, The first document also includes content on the third page and the fourth page, and the method further includes: In response to detecting a second input from the user, the first electronic device displays the third page of content and sends the fourth page of content to the second electronic device.

20. The method according to claim 18, characterized in that, The first document also includes content on the third page and the fourth page, and the method further includes: The first electronic device receives a second indication message sent by the second electronic device, the second indication message being used to indicate that the second electronic device has detected a third input; In response to receiving the second instruction information, the third page content is displayed and the fourth page content is sent to the second electronic device.

21. The method according to claim 14, characterized in that, The first interface is the desktop of the first electronic device. When the first electronic device and the second electronic device are displayed together in a fourth mode, the step of displaying the first part of the image information and sending the second part of the image information to the second electronic device includes: In response to receiving the first instruction information, the first electronic device displays the desktop and sends the image information corresponding to the desktop to the second electronic device. The fourth mode is activated based on receiving a selection operation for the fourth icon, and at least one of the first electronic device and the second electronic device displays the fourth icon.

22. The method according to claim 21, characterized in that, The desktop includes an icon for a first application, and the method further includes: The first electronic device receives a third indication message sent by the second electronic device, the third indication message being used to indicate that the second electronic device has detected a fourth input, the fourth input being an input to the icon of the first application; In response to receiving the third instruction information, the first electronic device sends image information corresponding to the display interface of the first application to the second electronic device.

23. The method according to claim 21 or 22, characterized in that, The desktop also includes an icon for a second application, and the method further includes: In response to detecting a fifth input from the user to the icon of the second application, the first electronic device displays the display interface of the second application.

24. The method according to any one of claims 14 to 16, characterized in that, The first interface also includes a second interface element, which is associated with the third interface of the third application. The method further includes: In response to detecting a sixth input from the user to the second interface element, the first electronic device displays image information corresponding to the first interface and sends image information corresponding to the third interface to the second electronic device.

25. The method according to claim 24, characterized in that, The third interface includes third interface elements, which are associated with the fourth interface of the third application. The method further includes: The first electronic device receives a fourth indication message sent by the second electronic device, the fourth indication message being used to indicate that the second electronic device has detected a seventh input, the seventh input being an input to the third interface element; In response to receiving the fourth instruction information, the first electronic device displays the fourth interface and sends the image information corresponding to the third interface to the second electronic device.

26. The method according to any one of claims 14 to 16, characterized in that, The method further includes: The first electronic device receives a fifth instruction message sent by the second electronic device, the fifth instruction message being used to instruct the first electronic device and the second electronic device to exit the splicing display.

27. A method for splicing displays, characterized in that, The method is applied to a second electronic device, which communicates with a first electronic device via a near-field wireless connection, and the method includes: In response to detecting a user’s first input, a first instruction message is sent to the first electronic device, the first instruction message being used to instruct the first electronic device and the second electronic device to perform splicing display; The device receives a second part of image information sent by the first electronic device. The second part of image information is associated with a first interface, which is the display interface when the first electronic device receives the first instruction information. In response to receiving the second part of the image information, the second part of the image information is displayed; Wherein, when the first electronic device and the second electronic device are spliced ​​together in a first mode, the first interface is the display interface of the third application. The display interface includes a first interface element, which is associated with the second interface of the third application. The first interface and the second interface are hierarchical pages to each other. The second part of the image information is the image information corresponding to the second interface; the first mode is activated based on receiving a selection operation on the first icon, and at least one of the first electronic device and the second electronic device displays the first icon.

28. The method according to claim 27, characterized in that, When the first electronic device and the second electronic device are spliced ​​together in the second mode, the second part of the image information is the image information obtained by the first electronic device after enlarging and segmenting the image information corresponding to the first interface. The second mode is activated based on receiving a selection operation for the second icon, and at least one of the first electronic device and the second electronic device displays the second icon.

29. The method according to claim 27, characterized in that, The first interface displays the first page of the first file. When the first electronic device and the second electronic device are displayed in a spliced ​​display mode, the step of responding to receiving the second portion of image information and displaying the second portion of image information includes: In response to receiving the second page content, the second electronic device displays the second page content, and the third mode is activated based on receiving a selection operation on a third icon, wherein at least one of the first electronic device and the second electronic device displays the third icon.

30. The method according to claim 29, characterized in that, The first document also includes content on the third page and the fourth page, and the method further includes: In response to detecting a third input from the user, the second electronic device sends a second indication message to the first electronic device, the second indication message indicating that the second electronic device has detected the third input; The second electronic device receives the fourth page content sent by the first electronic device; In response to receiving the fourth page content, the second electronic device displays the fourth page content.

31. The method according to claim 27, characterized in that, The first interface is the desktop of the first electronic device. When the first electronic device and the second electronic device are displayed together in a fourth mode, the step of responding to receiving the second part of the image information and displaying the second part of the image information includes: In response to receiving image information corresponding to the desktop, the second electronic device displays the desktop, and the fourth mode is activated based on receiving a selection operation for the fourth icon, wherein at least one of the first electronic device and the second electronic device displays the fourth icon.

32. The method according to claim 31, characterized in that, The desktop includes an icon for a first application, and the method further includes: In response to a fourth input from the user, the second electronic device sends a third indication message to the first electronic device, the third indication message indicating that the second electronic device has detected the fourth input, which is an input to the icon of the first application; The second electronic device receives image information corresponding to the display interface of the first application; In response to receiving image information corresponding to the display interface of the first application, the second electronic device displays the display interface of the first application.

33. The method according to any one of claims 27 to 31, characterized in that, The second part of the image information also includes a second interface element, which is associated with the third interface of the third application. The method further includes: In response to detecting a fifth input from the user, the second electronic device sends a fourth indication message to the first electronic device, the fourth indication message indicating that the second electronic device has detected the fifth input, the fifth input being an input to the second interface element; The second electronic device receives the image information corresponding to the third interface sent by the first electronic device; In response to receiving the image information corresponding to the third interface, the third interface is displayed.

34. The method according to any one of claims 27 to 31, characterized in that, The method further includes: The second electronic device displays the first control while displaying the second part of the image information; In response to detecting an eighth input from the user to the first control, the second electronic device displays a fourth interface and sends a fifth instruction message to the first electronic device. The fifth instruction message is used to instruct the first electronic device and the second electronic device to exit the splicing display. The fourth interface is the display interface of the second electronic device before detecting the first input.

35. An electronic device, characterized in that, include: One or more processors; One or more memory units; The one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 14 to 26; or, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 27 to 34.

36. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 14 to 26; or, When the computer instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 27 to 34.

Images