A display method, electronic device and system

Abstract

Embodiments of the present application provide a display method, an electronic device and a system. The method comprises: a first electronic device, in response to receiving a first indication, displaying a first interface and sending first information to a second electronic device, the first information comprising one or more interface elements on the first interface and label information of the one or more interface elements; the second electronic device, in response to receiving the first information, displaying a second interface according to the label information of the one or more interface elements, the second interface displaying at least part of the one or more interface elements. In the embodiments of the present application, the source device labels the interface elements on the user interface with label information, and the client device can restore the user interface suitable for the client device through the label information of the interface elements, without relying on the video decoding capability of the client device.

Description

[0001] This application claims priority to the Chinese Patent Application No. 202110350080.3, filed on March 31, 2021, and entitled "A Display Method, Electronic Device and System", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of terminals, and more particularly, to a display method, electronic device and system. BACKGROUND

[0003] Currently, the technology for restoring the user interface (UI) between devices is mainly video projection type restoration technology (e.g., Miracast). The basic principle is to encode the source device picture as a video and play the video on the client device, thereby realizing the restoration of the user interface of the source device by the client device. Such UI restoration technology requires the ability of the client device (e.g., video decoding ability), and cannot be used for devices without video decoding ability. SUMMARY

[0004] Embodiments of the present application provide a display method, electronic device and system. By tagging information on interface elements on the user interface by the source device, the client device can restore the user interface suitable for the client device through the tag information of the interface elements, without relying on the video decoding ability of the client device.

[0005] In a first aspect, a system is provided, the system comprising a first electronic device and a second electronic device, the first electronic device being in communication with the second electronic device through a short-range wireless connection, wherein the first electronic device is configured to, in response to receiving a first indication, display a first interface and send first information to the second electronic device, the first information comprising one or more interface elements on the first interface and tag information of the one or more interface elements; and the second electronic device is configured to, in response to receiving the first information, display a second interface according to the tag information of the one or more interface elements, the second interface displaying at least part of the one or more interface elements.

[0006] In the embodiments of the present application, the first electronic device (source device) can send one or more interface elements on the first interface and the tag information of the one or more interface elements to the second electronic device (client device) after receiving the first indication, so that the second electronic device can restore the interface suitable for the second electronic device according to the tag information, the user can experience different interface styles on different devices, which helps to improve the user experience; at the same time, the first electronic device does not need to rely on its video encoding capability when sending the interface elements and their tag information, and the second electronic device does not need to rely on its video decoding capability when restoring the interface.

[0007] In some possible implementation manners, each of the one or more interface elements in the first information corresponds to the tag information of each interface element.

[0008] In some possible implementation manners, the first electronic device is further configured to obtain a plurality of interface elements on the first interface and label the plurality of interface elements with tag information before sending the first information to the second electronic device.

[0009] In some possible implementation manners, the tag information can be the name of the interface element.

[0010] With reference to the first aspect, in some implementation manners of the first aspect, the second electronic device is specifically configured to display the second interface according to the tag information of the one or more interface elements and a first mapping relationship, the first mapping relationship being a mapping relationship between the display area of the at least part of the interface elements and the tag information of the at least part of the interface elements.

[0011] In some possible implementation manners, the first electronic device stores the first mapping relationship, and when the first electronic device receives the first indication, the first electronic device can send the first mapping relationship to the second electronic device in addition to sending the interface elements and their tag information to the second electronic device.

[0012] In some possible implementation manners, the first electronic device can store a plurality of mapping relationships in the same scene. For example, when the first electronic device (for example, a mobile phone) sends the first mapping relationship to the second electronic device (for example, a notebook computer), the first electronic device (for example, a mobile phone) can send a second mapping relationship to a third electronic device (for example, a smart watch).

[0013] In some possible implementation manners, the first mapping relationship can be stored in the second electronic device. When the second electronic device receives the interface elements and the tag information sent by the first electronic device, the second electronic device can display the second interface according to the first mapping relationship.

[0014] In the embodiments of this application, different display interfaces can be restored according to different mapping relationships for different client devices. For example, different styles of display interfaces can be restored for a notebook computer and a smart watch in the same scene, so that the user can experience different interface styles on different types of devices, and the use experience of the user can be improved.

[0015] With reference to the first aspect, in some implementations of the first aspect, the first electronic device is specifically configured to: in response to detecting that the user initiates the operation of the first application program, display the first interface and send the first information to the second electronic device.

[0016] In the embodiments of this application, when the first electronic device detects that the user initiates the operation of the first application program, the first electronic device can display the display interface of the first application program and send the interface elements on the display interface and the label information of the interface elements to the second electronic device, so that the display interface of the first application program can be restored on the second electronic device without other cumbersome operations of the user, and the use experience of the user can be improved.

[0017] With reference to the first aspect, in some implementations of the first aspect, the first electronic device is specifically configured to: in response to detecting that the user initiates the operation of the first application program, display the first interface and prompt the user whether to send one or more interface elements on the first interface to the second electronic device for display; and in response to detecting that the user determines to send one or more interface elements on the first interface to the second electronic device for display, send the first information to the second electronic device.

[0018] In the embodiments of this application, the first electronic device can also prompt the user whether to send the interface elements to the second electronic device for display when detecting that the user initiates the first application program, so that the user can clearly know which device to display the interface of the first application program.

[0019] In some possible implementation manners of the first aspect, the first electronic device is further configured to: in response to detecting that the user initiates the operation of the first application program, display the first interface and send first indication information to the second electronic device, the first indication information being used to indicate that the first electronic device has initiated the first application program; the second electronic device is further configured to: in response to receiving the first indication information, prompt the user whether to display the display interface of the first application program on the second electronic device; the second electronic device is further configured to: in response to detecting that the user determines to display the display interface of the first application program on the second electronic device, send first request information to the first electronic device, the first request information being used to request to display the interface element on the first interface; and the first electronic device is specifically configured to: in response to receiving the first request information, send the first information to the second electronic device.

[0020] In the embodiments of the present application, when the first electronic device initiates the first application program, the first electronic device can indicate the second electronic device that the first electronic device has initiated the first application program. The second electronic device can prompt the user whether to display the display interface of the first application program on the second electronic device, which helps the user to explicitly determine which device needs to display the interface of the first application program.

[0021] In some possible implementation manners of the first aspect, the second electronic device is further configured to: in response to detecting the operation of the user, send second request information to the first electronic device, the second request information being used to request to display the interface element on the first interface; and the first electronic device is specifically configured to: in response to receiving the second request information sent by the second electronic device, display the first interface and send the first information to the second electronic device.

[0022] In the embodiments of the present application, when the second electronic device detects the operation of the user, the second electronic device can determine that the current focus of the user is on the second electronic device. In this way, the second electronic device can request the interface element on the first interface from the first electronic device, so as to restore the display interface on the first electronic device.

[0023] In some possible implementation manners, the operation of the user can be an operation of the user moving or clicking a mouse, or the operation of the user clicking a physical key on a keyboard.

[0024] In some possible implementation manners, the operation of the user can be an operation of the user clicking an icon associated with the first application program.

[0025] With reference to the first aspect, in some implementations of the first aspect, the first electronic device is further configured to determine that a distance between the first electronic device and the second electronic device is less than or equal to a preset distance before sending the first information to the second electronic device.

[0026] With reference to the first aspect, in some implementations of the first aspect, the at least part of the interface elements includes a first interface element, and the first electronic device is further configured to, in response to the first interface element being updated, send second information to the second electronic device, the second information including the updated first interface element and label information of the first interface element; and the second electronic device is further configured to, in response to receiving the second information, display the updated first interface element on the second interface.

[0027] In the embodiments of the present application, when the first electronic device detects that the first interface element is updated, the first electronic device can send the updated first interface element to the second electronic device in real time for display, so that the interface elements displayed on the first electronic device and the second electronic device can be synchronized.

[0028] In some possible implementations, if the first interface is a video playing interface, the first interface element can be a video playing progress bar or image information in a video display window.

[0029] In some possible implementations, if the first interface is a music playing interface, the first interface element can be lyrics information corresponding to audio being currently played.

[0030] In some possible implementations, if the first interface element is real-time changing, the first electronic device can send the first interface element to the second electronic device at a certain period.

[0031] With reference to the first aspect, in some implementations of the first aspect, the at least part of the interface elements includes a second interface element, and the second electronic device is further configured to, in response to detecting user input to the second interface element, send second indication information to the first electronic device, the second indication information being used to indicate that the second electronic device detects the user input to the second interface element; and the first electronic device is further configured to, in response to receiving the second indication information, perform an operation corresponding to the input.

[0032] In the embodiments of the present application, when the second electronic device detects the user input to the second interface element, the second electronic device can instruct the first electronic device to detect the user input to the second interface element, so that the first electronic device can perform a corresponding operation.

[0033] For example, the second interface element is a download control. When the second electronic device detects a click event of the download control by the user, the second electronic device can indicate to the first electronic device that it detects the click event of the download control by the user. After receiving the indication, the first electronic device can complete the download of the file (a song, a document, or a video, etc.) in the background of the first electronic device.

[0034] With reference to the first aspect, in some implementations of the first aspect, the first electronic device is further configured to, in response to performing the operation, update a third interface element, the at least part of the interface elements including the third interface element; the first electronic device is further configured to send third information to the second electronic device, the third information including the updated third interface element and label information of the third interface element; and the second electronic device is further configured to, in response to receiving the third information, display the updated third interface element on the second interface.

[0035] In the embodiments of the present application, if the first electronic device updates the third interface element or adds a new interface element on the first interface after performing the corresponding operation, the first electronic device can send the information of the updated third interface element or the added new interface element to the second electronic device for display, so that the interface elements displayed on the first electronic device and the second electronic device can be synchronized.

[0036] With reference to the first aspect, in some implementations of the first aspect, the first information further includes information of a scenario corresponding to the first interface.

[0037] In a second aspect, a display method is provided, which is applied to a first electronic device in communication with a second electronic device through a close-range wireless connection. The method includes: receiving, by the first electronic device, a first indication; and in response to receiving the first indication, displaying, by the first electronic device, a first interface and sending, by the first electronic device, first information to the second electronic device, the first information including one or more interface elements on the first interface and label information of the one or more interface elements.

[0038] With reference to the second aspect, in some implementations of the second aspect, the method further includes: sending, by the first electronic device, information of a first mapping relationship to the second electronic device, the first mapping relationship being a mapping relationship between a display area of an interface element and label information of the interface element.

[0039] With reference to the second aspect, in some implementations of the second aspect, receiving, by the first electronic device, the first indication includes: detecting, by the first electronic device, an operation of starting a first application by a user.

[0040] With reference to the second aspect, in some implementations of the second aspect, in response to receiving the first indication, the first electronic device displays the first interface and sends the first information to the second electronic device, including: in response to detecting that a user initiates an operation of the first application, displaying the first interface and prompting the user whether to send one or more interface elements on the first interface to the second electronic device for display; and in response to detecting that the user determines to send the one or more interface elements on the first interface to the second electronic device for display, sending the first information to the second electronic device.

[0041] With reference to the second aspect, in some implementations of the second aspect, in response to receiving the first indication, the first electronic device sends the first information to the second electronic device, including: the first electronic device receiving first request information sent by the second electronic device, the first request information being used to request display of an interface element on the first interface; and the first electronic device sending the first information to the second electronic device in response to receiving the first request information.

[0042] With reference to the second aspect, in some implementations of the second aspect, the method further includes: before sending the first information to the second electronic device, the first electronic device determining that a distance between the first electronic device and the second electronic device is less than or equal to a preset distance.

[0043] With reference to the second aspect, in some implementations of the second aspect, the one or more interface elements include a first interface element, and the method further includes: in response to the first interface element being updated, the first electronic device sending second information to the second electronic device, the second information including the updated first interface element and label information of the first interface element.

[0044] With reference to the second aspect, in some implementations of the second aspect, the one or more interface elements include a second interface element, and the method further includes: the first electronic device receiving second indication information sent by the second electronic device, the second indication information being used to indicate that the second electronic device detects a user input to the second interface element; and in response to receiving the second indication information, the first electronic device performing an operation for the input.

[0045] With reference to the second aspect, in some implementations of the second aspect, the method further includes: in response to performing the operation, the first electronic device updating a third interface element, the one or more interface elements including the third interface element; and the first electronic device sending third information to the second electronic device, the third information including the updated third interface element and label information of the third interface element.

[0046] In a third aspect, a display method is provided. The method is applied to a second electronic device that communicates with a first electronic device through a short-range wireless connection. The method includes: receiving, by the second electronic device, first information sent by the first electronic device, the first information including one or more interface elements on a first interface displayed by the first electronic device and label information of the one or more interface elements; and in response to receiving the label information, displaying, by the second electronic device, a second interface according to the label information of the one or more interface elements, the second interface displaying at least part of the one or more interface elements.

[0047] With reference to the third aspect, in some implementations of the third aspect, displaying, by the second electronic device, the second interface according to the label information of the one or more interface elements includes: displaying, by the second electronic device, the second interface according to the label information of the one or more interface elements and a first mapping relationship, the first mapping relationship being a mapping relationship between a display area of the at least part of the interface elements and the label information of the at least part of the interface elements.

[0048] With reference to the third aspect, in some implementations of the third aspect, before receiving, by the second electronic device, the first information sent by the first electronic device, the method further includes: receiving, by the second electronic device, first indication information sent by the first electronic device, the first indication information indicating that the first electronic device has started the first application; in response to receiving the first indication information, prompting, by the second electronic device, a user whether to display a display interface of the first application by the second electronic device; and in response to detecting an operation of the user determining to display the display interface of the first application by the second electronic device, sending, by the second electronic device, first request information to the first electronic device, the first request information being used to request to display interface elements on the first interface.

[0049] With reference to the third aspect, in some implementations of the third aspect, before receiving, by the second electronic device, the first information sent by the first electronic device, the method further includes: in response to detecting an operation of the user, sending, by the second electronic device, second request information to the first electronic device, the second request information being used to request to display interface elements on the first interface.

[0050] With reference to the third aspect, in some implementations of the third aspect, the at least part of the interface elements includes a first interface element, and the method further includes: receiving, by the second electronic device, second information sent by the first electronic device, the second information including an updated first interface element and label information of the first interface element; and in response to receiving the second information, displaying, by the second electronic device, the updated first interface element on the second interface.

[0051] In some implementations of the third aspect, in conjunction with the third aspect, the at least part of the interface elements includes a second interface element, and in response to detecting the input of the user on the second interface element, the second electronic device sends second indication information to the first electronic device, the second indication information being used to indicate that the second electronic device detects the input of the user on the second interface element.

[0052] In some implementations of the third aspect, in conjunction with the third aspect, the at least part of the interface elements includes a third interface element, and the method further includes: if the first electronic device updates the third interface element after receiving the second indication information, the second electronic device receives third information sent by the first electronic device, the third information including the updated third interface element and label information of the third interface element; and in response to receiving the third information, the third electronic device displays the updated third interface element on the second interface.

[0053] In the fourth aspect, an apparatus is provided, which includes: a receiving unit configured to receive a first indication; a display unit configured to display a first interface in response to receiving the first indication; and a sending unit configured to send first information to a second electronic device, the first information including one or more interface elements on the first interface and label information of the one or more interface elements.

[0054] In the fifth aspect, an apparatus is provided, which includes: a receiving unit configured to receive first information sent by a first electronic device, the first information including one or more interface elements on a first interface displayed by the first electronic device and label information of the one or more interface elements; and a display unit configured to display a second interface according to the label information of the one or more interface elements in response to receiving the label information, the second interface displaying at least part of the one or more interface elements.

[0055] In the sixth aspect, an electronic device is provided, which includes: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and include instructions. When the instructions are executed by the electronic device, the electronic device performs the method in any possible implementation of the second aspect.

[0056] In the seventh aspect, an electronic device is provided, which includes: one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and include instructions. When the instructions are executed by the electronic device, the electronic device performs the method in any possible implementation of the third aspect.

[0057] In an eighth aspect, a computer program product including instructions, which, when executed on a first electronic device, cause the electronic device to perform the method of the second aspect described above; or, when executed on a second electronic device, cause the electronic device to perform the method of the third aspect described above.

[0058] In a ninth aspect, a computer-readable storage medium includes instructions, which, when executed on a first electronic device, cause the electronic device to perform the method of the second aspect described above; or, when executed on a second electronic device, cause the electronic device to perform the method of the third aspect described above.

[0059] In a tenth aspect, a chip is provided for executing instructions, when the chip is executed, the chip performs the method of the second aspect described above; or, the chip performs the method of the third aspect described above. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 FIG. 1 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.

[0061] Figure 2 FIG. 2 is a software structure block diagram provided by an embodiment of the present application.

[0062] Figure 3 FIG. 3 is a set of graphical user interfaces provided by an embodiment of the present application.

[0063] Figure 4 FIG. 4 is another set of graphical user interfaces provided by an embodiment of the present application.

[0064] Figure 5 FIG. 5 is another set of graphical user interfaces provided by an embodiment of the present application.

[0065] Figure 6 FIG. 6 is a schematic block diagram of a system architecture provided by an embodiment of the present application.

[0066] Figure 7 FIG. 7 is a schematic diagram of a drawing instruction provided by an embodiment of the present application.

[0067] Figure 8 FIG. 8 is a schematic diagram of a distribution of interface element regions in a music scene provided by an embodiment of the present application.

[0068] Figure 9 FIG. 9 is a schematic diagram of a distribution of interface element regions in a chat scene provided by an embodiment of the present application.

[0069] Figure 10 FIG. 10 is a schematic flowchart of a display method provided by an embodiment of the present application.

[0070] Figure 11 is a schematic structural diagram of an apparatus provided by an embodiment of the present application.

[0071] Figure 12 is another schematic structural diagram of an apparatus provided by an embodiment of the present application.

[0072] Figure 13 is another structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, “ / ” represents the meaning of or, for example, A / B can represent A or B; in this document, “and / or” only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, “plural” or “multiple” means two or more than two.

[0074] Hereinafter, the terms “first” and “second” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more features. In the description of the embodiments, unless otherwise specified, the meaning of “multiple” is two or more than two.

[0075] The method provided by the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not make any limitation on the specific type of electronic device.

[0076] Exemplary, Figure 1A structural diagram of the electronic device 100 is shown. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0077] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0078] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0079] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching instructions and executing instructions.

[0080] The processor 110 can also have a memory that stores instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is using repeatedly. If the processor 110 needs to use the instructions or data again, it can call them directly from the memory. This avoids repeated access and reduces the latency of the processor 110, thus improving the efficiency of the system.

[0081] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0082] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100.

[0083] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled with the audio module 170 through the I2S buses to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface to enable the function of answering a phone call through a Bluetooth earphone.

[0084] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 can be coupled with the wireless communication module 160 through a PCM bus interface. In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface to enable the function of playing music through a Bluetooth earphone. Both the I2S interface and the PCM interface can be used for audio communication.

[0085] The UART interface is a universal serial bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to enable Bluetooth functionality. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the UART interface to enable the function of playing music through a Bluetooth earphone.

[0086] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), and the like. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to enable the camera function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to enable the display function of the electronic device 100.

[0087] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

[0088] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and a peripheral device. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0089] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation on the electronic device 100. In some other embodiments of the present application, the electronic device 100 can also use different interface connection modes or combinations of multiple interface connection modes in the above embodiments.

[0090] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 while also supplying power to the electronic device through the power management module 141.

[0091] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0092] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0093] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0094] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer the same to the modem processor for demodulation. The mobile communication module 150 can also amplify signals modulated by the modem processor, and radiate the same as electromagnetic waves through the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the same device as at least part of the modules of the processor 110.

[0095] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110, and disposed in the same device as the mobile communication module 150 or other functional modules.

[0096] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and radiate it as an electromagnetic wave via the antenna 2.

[0097] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0098] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0099] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0100] The electronic device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0101] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0102] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0103] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0104] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0105] The NPU is a neural-network (NN) calculation processor, which can quickly process input information by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognition applications such as image recognition, face recognition, voice recognition, text understanding, etc.

[0106] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to realize data storage functions. For example, music, video, etc. Files are saved in the external memory card.

[0107] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0108] The electronic device 100 can realize audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0109] The audio module 170 is configured to convert digital audio information into an analog audio signal output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some of the functional modules of the audio module 170 can be disposed in the processor 110.

[0110] The speaker 170A, also referred to as a "loudspeaker", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0111] The receiver 170B, also referred to as a "earpiece", is configured to convert an audio electrical signal into a sound signal. When the electronic device 100 receives a call or a voice message, the user can listen to the voice by holding the receiver 170B close to the ear.

[0112] The microphone 170C, also referred to as a "microphone", "transducer", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak into the microphone 170C by holding the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, the noise reduction function can also be realized. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of the sound, realize the function of directional recording, etc.

[0113] The earphone interface 170D is configured to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0114] The pressure sensor 180A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The pressure sensor 180A can be of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, etc. The capacitive pressure sensor can include at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to a short message application icon, an instruction to view a short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold is applied to the short message application icon, an instruction to create a new short message is executed.

[0115] The gyroscope sensor 180B can be configured to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake photography. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of shaking of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and lets the lens offset the shaking of the electronic device 100 by reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0116] The barometric pressure sensor 180C is configured to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the barometric pressure sensor 180C, and assists in positioning and navigation.

[0117] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Further, based on the detected opening and closing state of the cover or the flip cover, the electronic device 100 can set a feature such as automatic unlocking of the flip cover.

[0118] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). When the electronic device 100 is stationary, the acceleration sensor 180E can detect the magnitude and direction of gravity. The acceleration sensor 180E can also be used to identify the attitude of the electronic device, and can be applied to landscape / portrait screen switching and pedometer applications.

[0119] Distance sensor 180F is configured to measure distance. Electronic device 100 can measure distance by infrared or laser. In some embodiments, electronic device 100 can utilize distance sensor 180F to measure distance for fast focusing when taking a picture.

[0120] Proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode. Electronic device 100 emits infrared light outwardly through the light emitting diode. Electronic device 100 detects infrared reflected light from nearby objects using the photodiode. When sufficient reflected light is detected, electronic device 100 can determine that there is an object near electronic device 100. When insufficient reflected light is detected, electronic device 100 can determine that there is no object near electronic device 100. Electronic device 100 can utilize proximity light sensor 180G to detect that a user is holding electronic device 100 close to the ear for a phone call, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used for automatic unlocking and locking of the screen in a holster mode or a pocket mode.

[0121] Ambient light sensor 180L is configured to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light brightness. Ambient light sensor 180L can also be used to automatically adjust white balance when taking a picture. Ambient light sensor 180L can also cooperate with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touch.

[0122] Fingerprint sensor 180H is configured to collect a fingerprint. Electronic device 100 can utilize the collected fingerprint characteristics to implement fingerprint unlocking, access application lock, fingerprint picture taking, fingerprint call answering, and the like.

[0123] Temperature sensor 180J is configured to detect temperature. In some embodiments, electronic device 100 utilizes the temperature detected by temperature sensor 180J to implement temperature processing strategies. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 implements performance reduction of a processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, electronic device 100 heats battery 142 to avoid abnormal shutdown of electronic device 100 caused by low temperature. In other embodiments, when the temperature is lower than yet another threshold, electronic device 100 implements voltage boosting of the output voltage of battery 142 to avoid abnormal shutdown caused by low temperature.

[0124] Touch sensor 180K, also referred to as "touch panel". Touch sensor 180K can be disposed on display screen 194, and touch sensor 180K and display screen 194 together form a touch screen, also referred to as "touch panel". Touch sensor 180K is configured to detect touch operations applied to or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the touch event type. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K can also be disposed on the surface of electronic device 100, which is different from the position of display screen 194.

[0125] Bone conduction sensor 180M can obtain vibration signals. In some embodiments, bone conduction sensor 180M can obtain vibration signals of the human body's vocal vibration bone block. Bone conduction sensor 180M can also contact the human body pulse to receive blood pressure pulsation signals. In some embodiments, bone conduction sensor 180M can also be disposed in a headset to form a bone conduction headset. Audio module 170 can analyze voice signals based on the vibration signals of the vocal vibration bone block obtained by the bone conduction sensor 180M to realize voice functions. The application processor can analyze heart rate information based on the blood pressure pulsation signals obtained by the bone conduction sensor 180M to realize heart rate detection functions.

[0126] Keys 190 include power on / off keys, volume keys, and the like. Keys 190 can be mechanical keys. They can also be touch keys. Electronic device 100 can receive key input and generate key signal input related to user settings and function control of electronic device 100.

[0127] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations applied to different regions of display screen 194 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. Touch vibration feedback effects can also be customizable.

[0128] Indicator 192 can be an indicator light, which can be used to indicate charging status, power changes, and also to indicate messages, missed calls, notifications, and the like.

[0129] The SIM card interface 195 is configured to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by being inserted into or pulled out of the SIM card interface 195. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195. The types of the multiple cards can be the same or different. The SIM card interface 195 can be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with a network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 uses an embedded-SIM (eSIM) card, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0130] It should be understood that the phone card in the embodiments of the present application includes, but is not limited to, a SIM card, an eSIM card, a universal subscriber identity module (USIM), a universal integrated circuit card (UICC), and the like.

[0131] The software system of the electronic device 100 can use a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application take the Android system with a layered architecture as an example to exemplarily illustrate the software structure of the electronic device 100.

[0132] Figure 2 FIG. 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Android runtime and system library, and a kernel layer. The application layer can include a series of application packages.

[0133] As shown in FIG. 1, the application packages can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, and the like. Figure 2

[0134] ​The application framework layer provides an application programming interface (API) and programming framework for applications of the application layer. The application framework layer includes some pre-defined functions.

[0135] As shown in Figure 2 the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0136] The window manager is used to manage window programs. The window manager can acquire the size of the display screen, determine whether there is a status bar, lock the screen, and intercept the screen, etc.

[0137] The content provider is used to store and acquire data, and make the data accessible to the application. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0138] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0139] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including call connection, call hang-up, etc.).

[0140] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc.

[0141] The notification manager enables the application to display notification information in the status bar, which can be used to convey a type of message that can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform the completion of the download, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, or a notification in the form of a dialogue window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, the electronic device vibrating, the indicator light flashing, etc.

[0142] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0143] The core library includes two parts: one part is the function function required by the java language to call, and the other part is the core library of Android.

[0144] The application program layer and the application framework layer run in a virtual machine. The virtual machine executes the java files of the application program layer and the application framework layer into binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, management of security and exceptions, and garbage collection.

[0145] The system library can include a plurality of functional modules. For example, a surface manager, media libraries, a three-dimensional graphics processing library (for example, OpenGL ES), a 2D graphics engine (for example, SGL), and the like.

[0146] The surface manager is used to manage a display subsystem and provides fusion of 2D and 3D layers for a plurality of application programs.

[0147] The media libraries support playback and recording of a plurality of commonly used audio, video formats, and static image files. The media libraries can support a plurality of audio and video encoding formats, for example, MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, and the like.

[0148] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, and the like.

[0149] The 2D graphics engine is a drawing engine for 2D drawing.

[0150] The kernel layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, and sensor drivers.

[0151] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Harmony, and the like.

[0152] Figure 3 is a set of graphical user interfaces (GUIs) provided by the embodiments of the present application.

[0153] As shown in (a) in Figure 3 The phone displays a desktop of the phone. The notebook computer is located near the phone, and the notebook computer can display a desktop of the notebook computer.

[0154] When the notebook computer is close to the mobile phone, the mobile phone and the notebook computer can form a network through a near field wireless connection. The network formation manner includes, but is not limited to, an access point (AP) network formation and a peer-to-peer (P2P) network formation. The AP network formation is that devices under the same AP (for example, a home Wi-Fi router) can communicate with each other through the AP device, thereby forming a many-to-many network. For example, the mobile phone and the notebook computer can be under the same home router. When the notebook computer is close to the mobile phone, the mobile phone can use a received signal strength indication (RSSI) technology to calculate the distance between the mobile phone and the notebook computer according to the strength of the received signal. When the distance is less than or equal to a preset distance, the mobile phone and the notebook computer can form an AP network.

[0155] Wi-Fi direct (Wi-Fi direct): also known as Wi-Fi peer to peer (Wi-Fi P2P), is a point-to-point connection manner. It can enable multiple Wi-Fi devices to form a point-to-point network (P2P network) to communicate with each other without an access point (AP). One station (STA) can act as an AP in the traditional sense, called a group owner (GO); another STA can be called a group client (GC), which can be connected to the GO like connecting to the AP. Among them, one STA can play the role of GO (i.e., act as an AP), and other STAs play the role of GC. In the embodiments of the present application, when one device is close to another device, the device on the left side can be the GO by default, and the device on the right side can be the GC. For example, the mobile phone can be the GO, and the notebook computer can be the GC. Alternatively, when a device detects a user's rightward sliding operation, the device can be the GC and the device can select another device on the left side as the GO; or when a device detects a user's leftward sliding operation, the device can be the GC and the device can select another device on the right side as the GO.

[0156] To build a peer-to-peer (P2P) network, one electronic device must first be able to scan for and detect another electronic device using the P2P protocol; this process is called the discovery phase. Only after discovery can the establishment of a P2P connection be triggered. When a laptop is close to a mobile phone, the phone can use RSSI technology to calculate the distance between the phone and the laptop. When this distance is less than or equal to a preset distance, the phone and laptop can form a P2P network.

[0157] Wi-Fi P2P technology forms the foundation for upper-layer services. Currently, P2P applications built on top of P2P mainly include Miracast and WLAN Direct. In a Miracast application scenario, a P2P-enabled electronic device can scan for and connect to a P2P-enabled large-screen device, then directly display its video, images, and other resources on the large-screen device. Leveraging P2P technology greatly enriches the Wi-Fi experience.

[0158] When a phone detects that a user has tapped the music app icon, the phone and laptop can display something like this. Figure 3 The GUI shown in (b) is shown in the image.

[0159] like Figure 3 As shown in (b), in response to detecting a user clicking the icon of a music application, the mobile phone can display a music playback interface, which includes multiple interface elements such as a music album cover image 301, a song title 302 (e.g., "Song 1"), an artist name 303, a lyrics display area 304, currently playing lyrics 305 (e.g., the lyrics played from 00:12 to 00:15 are "ExxxxxxxF"), a download control 306, a share control 307, a volume adjustment control 308, a more function selection control 309, a song playback progress bar 310, a mode selection control 311 (this mode selection control can be used to select sequential playback, single loop, shuffle playback, etc.), a play previous song control 312, a pause / start control 313, a play next song control 314, and a like control 315.

[0160] In one embodiment, when the mobile phone determines that the distance between the laptop and the mobile phone is less than or equal to a preset distance, the mobile phone can send information about multiple interface elements on the currently displayed music playback interface, as well as the audio content of the song, to the laptop. For example... Figure 3As shown in (b) of FIG. 3, in response to receiving the information of the plurality of interface elements sent by the mobile phone, the notebook computer can display a music playing window 316, wherein the music playing window 316 includes a cover picture 301 of a music album, a song name 302, a singer name 303, a currently playing lyric 305, a like control 315, a previous song playing control 312, a pause / start control 313, and a next song playing control 314.

[0161] In one embodiment, the mobile phone, the notebook computer, and the smart watch are networked. When the mobile phone detects that the user clicks the icon of the music application, the mobile phone can compare the distance between the mobile phone and the notebook computer and the distance between the mobile phone and the smart watch. When the distance between the mobile phone and the notebook computer is less than the distance between the mobile phone and the smart watch, the mobile phone can send the information of the plurality of interface elements on the currently displayed music playing interface and the audio content of the song to the notebook computer.

[0162] In one embodiment, when the mobile phone detects that the user starts the music application, if the mobile phone and the notebook computer have been networked, the mobile phone can prompt the user whether to send the display interface of the music application to the notebook computer. When the mobile phone detects that the user determines to send the display interface of the music application to the notebook computer, the mobile phone can send the information of the plurality of interface elements on the currently displayed music playing interface and the audio content of the song to the notebook computer.

[0163] In one embodiment, if the mobile phone, the notebook computer, and the smart watch are networked, the mobile phone can prompt the user to send the display interface of the music application to the notebook computer or the smart watch. For example, in response to detecting that the user determines to send the display interface of the music application to the notebook computer, the mobile phone can send the information of the plurality of interface elements on the currently displayed music playing interface and the audio content of the song to the notebook computer; for another example, in response to detecting that the user determines to send the display interface of the music application to the smart watch, the mobile phone can send the information of the plurality of interface elements on the currently displayed music playing interface and the audio content of the song to the smart watch.

[0164] In one embodiment, the display interface of the notebook computer can display an icon associated with the music application. When the notebook computer detects that the user clicks the icon, the notebook computer can send a request information to the mobile phone, the request information being used to request the mobile phone to send the information of the plurality of interface elements on the music playing interface and the audio content of the song to the notebook computer.

[0165] In one embodiment, when the laptop detects a user's first action (e.g., clicking a mouse or pressing a key on the keyboard), the laptop can send an indication message to the mobile phone, indicating that the laptop has detected the user's first action. In response to receiving this indication message and the mobile phone currently displaying a music playback interface, the mobile phone can send information about multiple interface elements and the audio content of the song to the laptop.

[0166] In one embodiment, the mobile phone can also send the phone's device name (e.g., Mate 40) and username (e.g., Lily) to the laptop. The laptop can also prompt the user on the music playback window 316 that the music playback interface is from Lily's Mate 40.

[0167] In one embodiment, when the mobile phone sends information about the multiple interface elements and audio content to the laptop computer, the mobile phone can continue to display the music playback interface and may not play the audio content of the song.

[0168] In this embodiment, the mobile phone can extend multiple interface elements from the currently running application's display screen to the laptop, allowing users to see the mobile application's display screen on the laptop and improving the user experience. Simultaneously, the music playback interface displayed on the laptop differs from that on the mobile phone; the laptop's music playback interface is more suited to the laptop's playback style, helping users experience a display method appropriate for their current device.

[0169] Figure 4 This is a set of GUIs provided in the embodiments of this application.

[0170] like Figure 4 As shown in (a), the phone displays its home screen. The laptop is located near the phone, and can display its home screen. When the phone detects that the user has tapped the icon of a social media app, both the phone and laptop can display... Figure 4 The GUI shown in (b) is shown in the image.

[0171] It should be understood that when the distance between the mobile phone and the laptop is less than or equal to the preset distance, the mobile phone and the laptop can form a network. The networking process can be referred to the description in the above embodiments, and will not be repeated here.

[0172] like Figure 4As shown in (b) of FIG. 4, in response to detecting the operation of the user clicking the icon of the social application, the phone can display the home page of the social application, where the home page of the social application includes a plurality of interface elements, for example, the chat record 400 with the contact Dad, the chat record 405 with the contact Mom, the chat record 406 with the contact Sam, and the chat record 407 with the contact Lucy. Taking the chat record 400 with the contact Dad as an example, the chat record includes the avatar information 401 of the contact Dad, the name 402 of the contact Dad, the latest chat record 403 with the contact Dad, and the occurrence time 404 of the latest chat record.

[0173] When the phone determines that the distance between the laptop and the phone is less than or equal to the preset distance, the phone can send the information of the plurality of interface elements on the currently displayed home page of the social application to the laptop. As shown in (b) of FIG. 5, in response to receiving the information of the plurality of interface elements sent by the phone, the laptop can display a home page display window 408 of another social application, where the window 408 includes the chat record 400 with the contact Dad, the chat record 405 with the contact Mom, the chat record 406 with the contact Sam, and the chat record 407 with the contact Lucy. Figure 4

[0174] In an embodiment, the phone can also send the device name (for example, Mate 40) of the phone and the user name (for example, Lily) to the laptop. The laptop can also prompt the user on the window 408 that the display interface of the social application is from the Mate 40 of Lily.

[0175] In an embodiment, in response to receiving the information of the plurality of interface elements sent by the phone, the laptop can also draw the minimize window control 409 and the close window control 410 in addition to restoring the home page of the social application displayed on the phone.

[0176] In an embodiment, when the laptop detects the operation of the user zooming in or out of the window 408, the laptop can also adjust the size of the window 408.

[0177] As shown in (b) of FIG. 4, in response to detecting the operation of the user clicking the icon of the social application, the phone can display the home page of the social application, where the home page of the social application includes a plurality of interface elements, for example, the chat record 400 with the contact Dad, the chat record 405 with the contact Mom, the chat record 406 with the contact Sam, and the chat record 407 with the contact Lucy. Taking the chat record 400 with the contact Dad as an example, the chat record includes the avatar information 401 of the contact Dad, the name 402 of the contact Dad, the latest chat record 403 with the contact Dad, and the occurrence time 404 of the latest chat record. Figure 4 As shown in (b) of FIG. 4, in response to detecting the operation of the user clicking the icon of the social application, the phone can display the home page of the social application, where the home page of the social application includes a plurality of interface elements, for example, the chat record 400 with the contact Dad, the chat record 405 with the contact Mom, the chat record 406 with the contact Sam, and the chat record 407 with the contact Lucy. Taking the chat record 400 with the contact Dad as an example, the chat record includes the avatar information 401 of the contact Dad, the name 402 of the contact Dad, the latest chat record 403 with the contact Dad, and the occurrence time 404 of the latest chat record. Figure 4

[0178] As shown in (b) of FIG. 4, in response to detecting the operation of the user clicking the icon of the social application, the phone can display the home page of the social application, where the home page of the social application includes a plurality of interface elements, for example, the chat record 400 with the contact Dad, the chat record 405 with the contact Mom, the chat record 406 with the contact Sam, and the chat record 407 with the contact Lucy. Taking the chat record 400 with the contact Dad as an example, the chat record includes the avatar information 401 of the contact Dad, the name 402 of the contact Dad, the latest chat record 403 with the contact Dad, and the occurrence time 404 of the latest chat record. Figure 4 ​​As shown in (c), in response to detecting that the user clicked on the chat history with the contact's father, the laptop can send a touch event and the tag information of the chat history 400 to the mobile phone. In response to receiving the touch event and tag information, the mobile phone can determine that the user clicked on the chat history with the contact's father on the laptop, and thus the mobile phone can display the chat interface between the user and the contact's father. This chat interface includes multiple interface elements, such as a return control 411, the contact's father's name 412, the time information of the user's most recent chat history with the contact's father 413, the user's avatar 414, and the content of the user's most recent chat history with the contact's father 415. Simultaneously, the mobile phone can also send information about the multiple interface elements on the chat interface to the laptop. In response to receiving the information about the multiple interface elements sent by the mobile phone, the laptop can update the content in window 408. Figure 4 In (c), the laptop can display a return control 411, the contact's father's name 412, the time information of the user's most recent chat history with the contact's father 413, the user's avatar 414, and the content of the user's most recent chat history with the contact's father 415 in window 408. In response to receiving information from these multiple interface elements, the laptop can also draw a text input box 416 in window 408, which can be used for text input and also includes a voice control 417 and a send control 418.

[0179] like Figure 4 As shown in (d), after the user receives the message "Okay, let's schedule a visit at home" from the contact person's father, the phone can display another chat interface between the user and the contact person's father. Compared to... Figure 4 (c) shows the chat interface displayed on the mobile phone, which also includes several added interface elements, such as the time information 419 (e.g., "08:00"), the contact's father's avatar information 420, and the message content 421. In response to receiving the message, the mobile phone can also send information about the newly added interface elements after the interface update to the laptop, such as the time information 419, the contact's father's avatar information 420, and the message content 421. In response to receiving information about the multiple interface elements sent by the mobile phone, the laptop can update the content in window 408. Figure 4 In (d), the laptop can display the return control 411, the name of the contact father 412, the time information 413, the user's avatar 414, the content of the chat history 415, and newly added interface elements (time information 419, the contact father's avatar information 420, and the message content 421) in window 408.

[0180] When the laptop detects that the user has entered the text "OK" in text input box 416, the laptop can send the text content to the mobile phone. In response to receiving this text content, the mobile phone can display the text "OK" in text input box 422. When the laptop detects that the user has clicked the send control 418, the laptop can send a touch event and the label information of control 418 to the mobile phone. In response to receiving this touch event and the label information of control 418, the mobile phone can determine that the user clicked the send control on the laptop. Therefore, the mobile phone can send the content "OK" from the text input box to the contact "Dad".

[0181] like Figure 4 As shown in (e), in response to sending the text content to the contact "Dad," the phone can display another chat interface between the user and the contact "Dad." Compared to Figure 5 (d) shows the chat interface displayed on the mobile phone, which also includes the user's avatar 423 and the user's reply 424. In response to replying to the message, the mobile phone can also send information about the newly added interface elements after the interface update to the laptop, such as the user's avatar 423 and the user's reply 424. In response to receiving information about multiple interface elements sent by the mobile phone, the laptop can update the content in window 408. For example... Figure 5 In (e), the laptop can display the return control 411, the name of the contact father 412, the time information 413, the user's avatar 414, the content of the chat history 415, the time information 419, the avatar information of the contact father 420, the content of the message 421, and the newly added interface elements (the user's avatar 423 and the content of the user's reply 424) in window 408.

[0182] In this embodiment, the mobile phone can send multiple interface elements from the social application's display screen to the laptop, thereby displaying window 408 on the laptop. Window 408 is adapted to the widescreen of the laptop. For example, the text input box 416 in window 408 has a larger area than the text input box 422 on the mobile phone, making it more convenient for users to input text and freeing them from the limitations of the small text input box on the mobile phone. Furthermore, when replying to messages on the laptop, users can use the laptop's input method and text input box, allowing them to use the keyboard for text input and quickly complete message replies.

[0183] Figure 5 This is a set of GUIs provided in the embodiments of this application.

[0184] like Figure 5As shown in (a) of FIG. 5, the mobile phone displays a desktop of the mobile phone. The notebook computer is located in the vicinity of the mobile phone, and the notebook computer can display a desktop of the notebook computer. When the mobile phone detects that a user clicks an icon of the gallery application, the mobile phone and the notebook computer can display a GUI as shown in (b) of FIG. 5. Figure 5

[0185] It should be understood that when the distance between the mobile phone and the notebook computer is less than or equal to the preset distance, the mobile phone and the notebook computer can be networked, and the networking process can refer to the description in the above embodiments, which will not be described here.

[0186] As shown in (a) of FIG. 5, the mobile phone displays a desktop of the mobile phone. The notebook computer is located in the vicinity of the mobile phone, and the notebook computer can display a desktop of the notebook computer. When the mobile phone detects that a user clicks an icon of the gallery application, the mobile phone and the notebook computer can display a GUI as shown in (b) of FIG. 5. Figure 5

[0187] When the mobile phone determines that the distance between the notebook computer and the mobile phone is less than or equal to the preset distance, the mobile phone can send information of the plurality of interface elements on the currently displayed home page of the gallery application to the notebook computer. As shown in (b) of FIG. 5, in response to receiving the information of the plurality of interface elements sent by the mobile phone, the notebook computer can display a window 516, wherein the window 516 includes the plurality of thumbnail photos. Figure 5

[0188] In one embodiment, the mobile phone can further send a device name (for example, Mate 40) of the mobile phone and a user name (for example, Lily) to the notebook computer. The notebook computer can further prompt the user on the window 516 that the display interface of the gallery application is from the Mate 40 of Lily.

[0189] As shown in (c) of FIG. 5, when the notebook computer detects that a user selects the thumbnail photos 501-509 and drags them into a folder, the notebook computer can send a request information to the mobile phone, the request information being used to request image information of the photos 501-509, and the request information further including tag information of the thumbnail photos 501-509. In response to receiving the request information, the mobile phone can send the image information of the photos 501-509 to the notebook computer. In response to receiving the image information of the photos 501-509 sent by the mobile phone, the notebook computer can save the photos 501-509 in the folder. Figures 3 to 5 As shown in (d) of FIG. 5, when the notebook computer detects that a user double-clicks the folder, the notebook computer can display a display window 517 of the folder, and the display window 517 of the folder includes information of the photos 501-509.

[0190] Figure 6 As shown in (d) of FIG. 5, when the notebook computer detects that a user double-clicks the folder, the notebook computer can display a display window 517 of the folder, and the display window 517 of the folder includes information of the photos 501-509. ​​​​

[0191] In this embodiment, the mobile phone can send multiple interface elements from the gallery application's display screen to the laptop computer, thereby displaying window 516 on the laptop computer. When a user wants to save multiple photos from the gallery to the local laptop computer, they only need to select multiple photos with the mouse and drag them to the desired folder, eliminating the need for the user to send photos from the mobile phone to the laptop computer and improving the user experience.

[0192] The above combination Figure 2 Several GUIs in the embodiments of this application have been introduced. The implementation process of the technical solutions in the embodiments of this application is described below with reference to the accompanying drawings.

[0193] Figure 3 A schematic block diagram of the system architecture provided in this application embodiment is shown. This system architecture includes a source device and a sink device. The source device includes an application layer and a service layer. The application layer includes a tagged view module 601, a content fetcher module 602, and an action dispatcher module 603. The service layer includes a data transfer module 604 and a remote action receiver module 605.

[0194] It should be understood that the functions implemented by the marker view module 601, the content extraction module 602, and the action scheduling module 603 can be achieved through [the following mechanism / mechanism]: Figure 4 The view system is implemented in the application framework layer.

[0195] The sink device includes a view content parser module 606 and an action translator module 607. The following sections provide a detailed description of each module in both the source and sink devices.

[0196] The marker view module 601 is used to mark scene and interface elements. Figure 3 Taking the music playback interface displayed on the mobile phone in (b) as an example, the mobile phone can mark the entire interface as a music scene. Multiple interface elements on this interface are also individually labeled. The source device can assign corresponding tags to these multiple interface elements, and these tags include information such as the attributes and names of the interface elements. For example, Table 1 shows a schematic diagram illustrating the correspondence between interface elements and their corresponding tags.

[0197] Table 1

[0198]

[0199]

[0200] In one embodiment, for some scenarios (e.g., chat scenario), the source device can also group multiple interface elements and label each group of interface elements after grouping. For example, the GUI displayed on the mobile phone in (b) of FIG. 6 shows a chat scenario. The mobile phone can list each chat record. After listing, the mobile phone can label each interface element in each list. For example, Table 2 shows an exemplary correspondence between interface elements and their corresponding labels. Figure 7

[0201] Table 2

[0202]

[0203] It should be understood that the above-described Table 1 and Table 2 are only exemplary, and there can be other ways of corresponding between interface elements and their corresponding label information, and the embodiments of the present application are not limited thereto.

[0204] The content extraction module 602 is configured to extract interface elements and send the extracted interface elements to the data transmission module 604 of the service layer. For example, the content extraction module 602 can extract interface elements from the music scenario shown in (b) of FIG. 6. Figure 6 The content extraction module 602 can extract the interface element 301 corresponding to the cover picture, the interface element 302 corresponding to the song name, the interface element 303 corresponding to the singer, the interface element 305 corresponding to the currently playing lyrics, the interface element 315 corresponding to the like control, the interface element 312 corresponding to the previous song playing control, the pause / play control 313, and the next song playing control 314.

[0205] The content extraction module 602 can extract interface elements in the following ways, but is not limited thereto:

[0206] Method 1

[0207] Extract interface elements from instructions of the rendering layer. The instructions of the rendering layer include all information for restoring each interface element. Figure 3 As shown in the diagram of the rendering instructions, there are more than 40 instructions, of which the drawText and drawImage instructions are the only instructions with explicit content. The source device can extract the content of the interface elements from these two types of instructions.

[0208] Method 2

[0209] ​The interface elements are extracted from the view layer (or the control layer). Taking the control TextView in Android as an example, the developer can call the interface setText of the TextView to set the text to be displayed. The source device can record the set text, thereby completing the extraction of the interface element with the attribute TextView. Similarly, the developer can call the interface setImage of the ImageView to set the image information to be displayed, and the source device can record the set image information, thereby completing the extraction of the interface element with the attribute ImageView.

[0210] It should be understood that, as shown in Figure 8 The drawing layer and the view layer (or the control layer) can be included in the application layer, and the drawing layer and the view layer can realize the interface display of the source. The view layer can provide the interface elements such as TextView or ImageView for the developer to build the interface. The developer can set the text style, text size, text thickness, etc. through the interface (for example, setText) provided by the TextView. The content extraction module 602 can directly complete the extraction of the interface elements in the view layer. Alternatively, the interface elements provided by the view layer can be converted into drawText and drawImage instructions in the drawing layer, and the content extraction module 602 can also extract the interface elements through the drawText and drawImage instructions in the drawing layer.

[0211] Method three

[0212] The developer of the application can customize the content. For example, the developer can set the content in the format of <key, value> for the interface elements on the interface, so that the source device can extract the interface elements with the content in the format.

[0213] The data transmission module 604 can send the scene name (for example, the scenes of music and chat) of the display interface of the source and the information of the marked interface elements (including the tag information of the interface elements and the interface elements extracted through the content extraction module 602) to the content analysis module 606 of the sink device.

[0214] In an embodiment, the source device can send a BLE data packet to the sink device, and the BLE data packet can carry the interface element and the tag information of the interface element. The BLE data packet includes a protocol data unit (PDU), and the interface element and the tag information of the interface element can be carried in a service data field in the PDU or can be carried in a manufacturer specific data field in the PDU. For example, the payload of the service data field can include a plurality of bits, and the plurality of bits include extensible bits. The source device and the sink device can agree on the content of a certain extensible bit.

[0215] For example, for a graphical interface element (ImageView), the source device can encode the graphical interface element by using an image encoding technology, and carry the encoded data on a first bit (the first bit can include one or more bits); meanwhile, the source device can encode the tag information of the graphical interface element. Taking the interface element 301 in Table 1 as an example, the tag information of the interface element 301 can be the name “cover picture”. The source device can encode the tag information by using an encoding method such as GBK, ISO8859-1, or Unicode (for example, UTF-8, UTF-16), and carry the encoded data on a second bit (the second bit can include one or more bits); the source device can associate the first bit and the second bit. After receiving the BLE data packet, the sink device can decode the data on the first bit by using an image decoding technology to obtain the interface element 301; meanwhile, the sink device can also decode the information on the second bit to obtain the tag information “cover picture” of the interface element 301.

[0216] For example, for a text interface element (TextView), the source device can encode the text interface element by using GBK, ISO8859-1, or Unicode (for example, UTF-8, UTF-16), and carry the encoded data of the image on the third bit (the third bit can include one or more bits); meanwhile, the source device can encode the label information of the text interface element. Taking the interface element 303 in Table 1 as an example, the label information of the interface element 303 can be the name "singer". The source device can encode the label information by using GBK, ISO8859-1, or Unicode (for example, UTF-8, UTF-16), and carry the encoded data on the fourth bit (the fourth bit can include one or more bits); the source device can associate the third bit and the fourth bit. After receiving the BLE data packet, the sink device can decode the data on the third bit by using a corresponding decoding technology, and thus obtain the interface element 303; meanwhile, the sink device can decode the information on the fourth bit, and thus obtain the label information "singer" of the interface element 303.

[0217] It should be understood that, for interface elements such as a music playing progress bar and a switch control, which do not belong to the graphic interface elements and do not belong to the text interface elements, the source device can convert these interface elements and the switch control into a string, so that the source device can also encode these interface elements and the label information thereof by using GBK, ISO8859-1, or Unicode (for example, UTF-8, UTF-16).

[0218] In one embodiment, in addition to sending the interface element and the label information of the interface element to the sink device, the source device can also send the scene information corresponding to the interface element to the sink device. For example, for the GUI shown in Table 1, the mobile phone can also carry the scene information "music" in the BLE data packet sent to the notebook computer. The source device can refer to the process of carrying the label information in the BLE data packet for implementation of carrying the scene information in the BLE data packet, which is not described herein again. Figure 9

[0219] In one embodiment, the label information can be a name or an ID. For example, for the interface element 301, the source device and the sink device can agree that the interface element with an ID of 1 corresponds to the name "cover picture", and the interface element with an ID of 2 corresponds to the name "song name", and so on.​

[0220] In one embodiment, the source device can send a user datagram protocol (UDP) packet to the sink device, and the UDP packet can carry the interface element and the label information of the interface element. The UDP packet includes a data portion of an IP datagram. The data portion of the IP datagram can include extensible bits. The source device and the sink device can agree on the extensible bits. Thus, the source device can carry the interface element and the label information of the interface element on the extensible bits.

[0221] In one embodiment, the source device can send a transmission control protocol (TCP) packet to the sink device, and the TCP packet can carry the interface element and the label information of the interface element. The TCP packet includes a TCP header and a TCP data portion, and the notification information can be carried in the TCP data portion. For example, the TCP data portion can include a plurality of bits, and the plurality of bits can include extensible bits. Thus, the source device can carry the interface element and the label information of the interface element on the extensible bits. For the information transmission process of the interface elements such as graphical interface elements, text interface elements, progress bars, and switch controls, reference can be made to the description in the above embodiments, which will not be described herein again.

[0222] The view content parser module 606 of the sink device is configured to parse the received packet after receiving the packet sent by the data transmission module 604 of the source device, so as to obtain one or more interface elements and the label information of the interface elements. In one embodiment, the sink device can also obtain the current scene information.

[0223] The sink device stores a corresponding relationship between the display area of the interface element and the label information of the interface element in different scenes. Figure 8 and Figure 9 The schematic diagrams of the distribution of the interface element area in the music scene and the chat scene are output respectively.

[0224] Exemplarily, Figure 3A schematic diagram of the distribution of interface element regions of a music scene saved by a sink device is shown. A cover picture can be displayed in region 801, a song name can be displayed in region 802, a singer can be displayed in region 803, a currently playing lyric can be displayed in region 804, a like control can be displayed in region 805, a control for playing a previous song can be displayed in region 806, a pause / play control can be displayed in region 807, and a control for playing a next song can be displayed in region 808.

[0225] The content analysis module can place each interface element into a designated region according to the parsed interface element and the label information corresponding to the interface element. For example, when the sink device parses interface element 301 and the name in the label information of interface element 301 is a cover picture, the sink device can place interface element 301 in region 801; when the sink device parses interface element 302 and the name in the label information of interface element 302 is a song name, the sink device can place interface element 302 in region 802.

[0226] For example, Figure 3 A schematic diagram of the distribution of interface element regions of a chat scene saved by a sink device is shown. The name of an application can be displayed in region 901; a first group of chat records can be displayed in region 902, which can further include a plurality of sub-regions, for example, a contact's avatar can be displayed in region 9021, a contact's name can be displayed in region 9022, the content of the most recent chat between the user and the contact can be displayed in region 9023, and the implementation information of the most recent chat between the user and the contact can be displayed in region 9024. By analogy, a second group of chat records, a third group of chat records, and a fourth group of chat records can be displayed in regions 903-905.

[0227] The content analysis module can put each interface element into a designated area according to each interface element and the label information corresponding to each interface element. For example, the sink device analyzes interface elements 401-404 as interface elements in the first group of chat records. If the name in the label information of interface element 401 is the name of a contact, the sink device can put interface element 401 into area 9021; if the name in the label information of interface element 402 is the name of a contact, the sink device can put interface element 402 into area 9022; if the name in the label information of interface element 403 is the content of the last chat, the sink device can put interface element 403 into area 9023; if the name in the label information of interface element 404 is the time information of the last chat, the sink device can put interface element 404 into area 9024. Similarly, the sink device can put other interface elements into areas 902-904, respectively.

[0228] The action translation module 607 is configured to send the corresponding event and the label information of the interface element corresponding to the event to the remote action receiving module 605 of the source device when the sink device detects a touch event (for example, a click event on a certain control).

[0229] In one embodiment, when the motion translation module 607 on the sink side detects a user click event on a first interface element, the sink device can send a BLE data packet to the source device. This BLE data packet includes event type information and tag information for the first interface element. The BLE data packet includes a Protocol Data Unit (PDU). The interface element and its tag information can be carried in the service data field of the PDU, or in a vendor-specific data field of the PDU. For example, the payload of the service data field can include multiple bits, including expandable bits. The source device and the sink device can agree on the content of a certain expandable bit. For example, taking a laptop computer as the sink device, when some expandable bits are 00, it indicates that the event type is selecting the first interface element; when some expandable bits are 01, it indicates that the event type is right-clicking after selecting the first interface element. For example, the sink device can encode the tag information corresponding to the first interface element using encoding methods such as GBK, ISO8859-1, or Unicode (e.g., UTF-8, UTF-16), and carry the encoded data on one or more expandable bits. After parsing the BLE data packet, the source device can determine the type of the event and the corresponding tag information of the interface element, thereby knowing that the sink device has detected the corresponding event on the first interface element.

[0230] After receiving an event from the action translation module 607, the remote action receiving module 605 can send the received information to the action scheduling module 603. The action scheduling module 603 can locate the corresponding interface element based on its label information and determine the specific operation performed by the user on the sink device based on the event type. Therefore, the action scheduling module 603 can update the interface elements on the currently displayed screen.

[0231] For example, such as Figure 4 As shown in (b), when the laptop detects user input to the pause / play control, it can send an event type (click event) and label information for interface element 313 to the mobile phone. In response to receiving this information, the mobile phone can determine that the user has clicked the pause / play control on the laptop, thus pausing the currently playing music. The mobile phone updates interface element 313 from a playing state to a paused state. Simultaneously, the mobile phone stops sending the audio content corresponding to the song to the laptop and can also send the updated interface element 313 and its label information to the laptop, allowing the laptop to display the updated interface element 313.

[0232] In one embodiment, if the second interface element is included in the one or more interface elements and the state of the second interface element is updated in real time, after the first electronic device sends the one or more interface elements and the label information thereof to the second electronic device, the first electronic device can send the updated second interface element and the label information thereof to the second electronic device in real time after detecting the update of the state of the second interface element. For example, Figure 10 Taking the interface element 305 shown in (b) of FIG. 3B as an example, the currently playing lyrics are updated in real time. For example, the lyrics "ExxxxxxxF" are played at 00:12-00:15 and the lyrics "GxxxxxxxH" are played at 00:16-00:18. Therefore, at the moment when the song is played to 00:16, the mobile phone can send the updated interface element 305 ("GxxxxxxxH") and the label information thereof to the notebook computer. In this way, the notebook computer can update the content of the currently playing lyrics after receiving the updated interface element 305 and the label information thereof.

[0233] In one embodiment, for the interface element shown in (d) of FIG. 3B, Figure 3 When the notebook computer detects that the user inputs the text content "good" in the text input box 416, the notebook computer can send the text content and the label information of the text input box to the mobile phone. For example, the notebook computer can send a BLE data packet to the mobile phone. The text content and the label information of the text input box can be carried in the service data field in the PDU. The notebook computer can encode the text content and the label information of the text input box by using GBK, ISO8859-1, or Unicode (for example, UTF-8, UTF-16), and carry the encoded data on the extensible bit of the service data field. After receiving the BLE data packet, the mobile phone can decode the data on the extensible bit to obtain the text content and the label information of the text input box, so that the mobile phone can display the text content in the text input box 422 of the mobile phone.

[0234] When the notebook computer detects that the user clicks the sending control 418, the notebook computer can send the corresponding event type and the label information of the control 418 to the mobile phone. The specific sending process can refer to the description in the above embodiments, which will not be described here. After receiving the event type and the label information of the control 418, the mobile phone can determine that the notebook computer detects that the user clicks the sending control. Therefore, the mobile phone can complete the sending of the content in the text input box 422.

[0235] It should be understood that in the embodiments of the present application, the text input box 416 can be drawn by the notebook computer. The notebook computer and the mobile phone can agree on the label information of the text input box 416, the voice control 417 and the sending control 418 in advance. In this way, the mobile phone can know the operation of the user on the notebook computer after receiving the content sent by the notebook computer.

[0236] Figure 3 A schematic flowchart of a display method 1000 provided by the embodiments of the present application is shown, which can be executed by a first electronic device and a second electronic device. The first electronic device can be the source device, and the second electronic device can be the sink device. The method 1000 includes the following steps.

[0237] In step S1001, the first electronic device receives a first indication.

[0238] In one embodiment, the first indication can be that the first electronic device detects that the user starts an operation of the first application.

[0239] In one embodiment, the first indication can be that the first electronic device receives indication information sent by the second electronic device, and the indication information is used to instruct the first electronic device to send the interface elements on the first interface to the second electronic device. For example, the second electronic device can send the indication information to the first electronic device when detecting that the user clicks the icon associated with the first application.

[0240] In step S1002, in response to receiving the first indication, the first electronic device displays the first interface and sends one or more interface elements and label information of the one or more interface elements to the second electronic device, and the first interface includes the one or more interface elements.

[0241] For example, as shown in (b) of FIG. 11, the first electronic device can be a mobile phone, and the first interface can be a music playing interface, and the music playing interface includes the interface elements 301-315. Figure 8

[0242] It should be understood that the process that the first electronic device sends the one or more interface elements and the label information of the one or more interface elements to the second electronic device can refer to the description in the above embodiments, which will not be described here.

[0243] ​In one embodiment, when the distance between the first electronic device and the second electronic device is less than or equal to the preset distance, the first electronic device can prompt the user to send one or more interface elements on the first interface to the second electronic device for display. In response to the user determining to send the interface elements on the first interface to the second electronic device, the first electronic device can send the one or more interface elements and the label information of the one or more interface elements to the second electronic device.

[0244] In one embodiment, when the first electronic device determines that the device in the network with the first electronic device includes an electronic device supporting display of one or more interface elements, the first electronic device can send the one or more interface elements and the label information of the one or more interface elements to the second electronic device. For example, after displaying the first interface, the first electronic device can send a query request to the device in the network with the first electronic device, the query request being used to query whether the device receiving the query request supports display of one or more interface elements. If the second electronic device is included in the device in the network with the first electronic device and the second electronic device supports display of one or more interface elements, the second electronic device can send an acknowledgement (ACK) to the first electronic device, so that the first electronic device knows that the second electronic device supports display of one or more interface elements.

[0245] For example, the first electronic device can send a BLE data packet to the second electronic device, the BLE data packet carrying the query request. The BLE data packet includes a PDU, the query request being carried in a service data field in the PDU or a vendor-specific data field in the PDU. For example, the load of the service data field can include a plurality of bits, wherein the plurality of bits include extensible bits. The first electronic device and the second electronic device can agree on the content of a certain extensible bit. For example, when a certain extensible bit is 1, the second electronic device can know that the first electronic device requests to query whether the second electronic device supports display of one or more interface elements.

[0246] If the second electronic device supports display of one or more interface elements, the second electronic device can send a BLE data packet to the first electronic device. For example, when a certain extensible bit in the BLE data packet is 1, the first electronic device can know that the second electronic device supports display of one or more interface elements.

[0247] In one embodiment, if the plurality of devices networked with the first electronic device all support displaying the one or more interface elements, the first electronic device can further prompt the user to send the one or more interface elements to which device for display. For example, if the second electronic device and the third electronic device in the plurality of devices networked with the first electronic device both support displaying the one or more interface elements, the first electronic device can prompt the user to send the one or more interface elements to the second electronic device or the third electronic device for display. In response to the user selecting the second electronic device, the first electronic device can send the one or more interface elements and the label information of the one or more interface elements to the second electronic device.

[0248] In one embodiment, the first indication can also be request information from the second electronic device. For example, the display interface of the second electronic device can include an icon of an application. When the second electronic device detects the user clicking the icon, the second electronic device can send request information to the first electronic device, where the request information is used to request the information of the one or more interface elements on the display interface of the application.

[0249] It should be understood that the display interface of the second electronic device including the icon of the application does not mean that the second electronic device has installed the application. The second electronic device detecting the user clicking the icon can trigger the second electronic device sending request information to the first electronic device, where the request information is used to request the first electronic device sending the icon of the one or more applications on the display interface of the application to the second electronic device.

[0250] For example, in response to detecting the user clicking the icon of the first application (e.g., a music application or a chat application), the second electronic device can send a BLE data packet to the first electronic device. For example, when a certain extendable bit in the BLE data packet is 1, the first electronic device can know that the second electronic device requests the information of the one or more interface elements on the display interface of the first application from the first electronic device.

[0251] In one embodiment, after the first electronic device and the second electronic device are networked, the second electronic device can send a query request to the first electronic device, the query request being used to query whether the first electronic device has installed the first application. For example, the second electronic device can send a BLE data packet to the first electronic device, and the query request can be carried in the BLE data packet. The payload of the service data field in the BLE data packet can include a plurality of bits, and the plurality of bits can include extensible bits. The first electronic device and the second electronic device can agree on the content of a certain extensible bit. For example, when the certain extensible bit is 1, the first electronic device can know that the second electronic device requests to query whether the first electronic device has installed the first application. If the first electronic device has installed the first application, the first electronic device can send a response (ACK) to the second electronic device, so that the second electronic device can know that the first electronic device has installed the first application. Thereafter, if the second electronic device detects that the user clicks the icon of the first application, the second electronic device can send the request information to the first electronic device.

[0252] In S1003, the second electronic device can display a second interface according to the label information of the one or more interface elements, and the second interface can display at least part of the one or more interface elements.

[0253] In one embodiment, after the second electronic device receives the one or more interface elements and the label information thereof, the second electronic device can display a second interface according to the label information of the one or more interface elements. For example, the second electronic device can select the interface elements corresponding to the label information with higher priority to display. Figure 9 For example, the GUI shown in (b) of FIG. 3, the mobile phone can send the interface elements 301-315 and the label information thereof to the second electronic device, and the second electronic device can store the priority information of the label information (for example, the priority from high to low is song name, singer, currently playing lyrics, previous song playing control, pause / play control, and next song playing control). Then, the second electronic device can display the interface elements 302, 303, 305, 312, 313, and 314 according to the priority information and the label information of the interface elements 301-315.

[0254] In one embodiment, the second electronic device displays at least part of the one or more interface elements according to the first mapping relationship and the label information of the one or more interface elements, and the first mapping relationship is a mapping relationship between the display area of the one or more interface elements and the label information of the one or more interface elements.

[0255] In one embodiment, the first mapping relationship can be stored in the first electronic device. The first electronic device can further send the first mapping relationship to the second electronic device when sending the interface elements and the tag information to the second electronic device.

[0256] In one embodiment, for the same scenario (e.g., music scenario), the first electronic device can store different mapping relationships for different devices. For example, for a mobile phone as the first electronic device, the mobile phone can store a first mapping relationship and a second mapping relationship, the first mapping relationship being a mapping relationship between display regions of interface elements in a notebook computer and tag information of the interface elements, and the second mapping relationship being a mapping relationship between display regions of interface elements in a smart watch and tag information of the interface elements. For different sink devices, the mobile phone can send different interface elements and tag information of the interface elements. For example, for the notebook computer, the mobile phone can send the interface elements 301-303, 305, 312-315 and corresponding tag information to the notebook computer; for example, for the smart watch, the mobile phone can send the interface elements 302-303, 305, 312-314 and corresponding tag information to the smart watch.

[0257] In one embodiment, the first mapping relationship can also be stored in the second electronic device. When the second electronic device receives the interface elements and the tag information from the first electronic device, the second electronic device can display a second interface according to the tag information of the interface elements and the first mapping relationship.

[0258] In one embodiment, the second electronic device can store mapping relationships between display regions of interface elements and tag information of the interface elements in different scenarios. For example, as shown in Figure 3 and Figure 8 shown, the sink device can store mapping relationships between different regions and tag information of different interface elements in a music scenario and a chat scenario. When the second electronic device receives one or more interface elements and tag information of the interface elements from the first electronic device, the sink device can select at least part of the one or more interface elements to display according to the tag information of the one or more interface elements.

[0259] In one embodiment, the first electronic device can send all tagged interface elements and tag information of the interface elements in the first interface to the second electronic device. After receiving all the tagged interface elements and tag information of the interface elements, the second electronic device can select part or all of the interface elements to display according to the first mapping relationship.

[0260] For example, as shown in Figure 11As shown in (b) of FIG. 3, the mobile phone can send the tagged interface elements 301-315 and the corresponding tag information to the notebook computer. After receiving the interface elements and the tag information, the notebook computer can display the interface elements 301, 302, 303, 305 and 312-315 according to the corresponding relationship between the display area of the interface elements and the tag information. Figure 10 As shown in (b) of FIG. 3, the mobile phone can send the tagged interface elements 301-315 and the corresponding tag information to the notebook computer. After receiving the interface elements and the tag information, the notebook computer can display the interface elements 301, 302, 303, 305 and 312-315 according to the corresponding relationship between the display area of the interface elements and the tag information.

[0261] In one embodiment, the first electronic device can store the information of the interface elements required in a certain scenario. Thus, the first electronic device can send the interface elements and the tag information required by the second electronic device to the second electronic device. For example, after the first electronic device and the second electronic device are connected, the second electronic device can send the information of the interface elements required in the first scenario to the first electronic device. The second electronic device can send a BLE data packet to the first electronic device, in which the information of the first scenario and the information of the interface elements required in the first scenario are included. The payload of the service data field in the BLE data packet can include a plurality of bits, in which the plurality of bits include extensible bits. The first electronic device and the second electronic device can agree on the content of the extensible bits. For example, the second electronic device can encode the information of the first scenario (e.g., music) by using the encoding mode of GBK, ISO8859-1 or Unicode (e.g., UTF-8, UTF-16), and encode the tag information of the interface elements required in the first scenario by using the encoding mode of GBK, ISO8859-1 or Unicode (e.g., UTF-8, UTF-16), so as to carry the encoded data on one or more extensible bits. After receiving the BLE data packet, the first electronic device can decode the BLE data packet to obtain the tag information of the interface elements required by the second electronic device in the first scenario. Thus, after receiving the first indication, the first electronic device can send the interface elements and the tag information required by the second electronic device to the second electronic device.

[0262] Figure 12 A schematic block diagram of the apparatus 1100 provided by the embodiments of the present application is shown. The apparatus 1100 can be arranged in the first electronic device in the above Figure 10 The apparatus 1100 includes a receiving unit 1110, a display unit 1120 and a sending unit 1130. The receiving unit 1110 is configured to receive a first indication. The display unit 1120 is configured to display a first interface in response to receiving the first indication. The sending unit 1130 is configured to send first information to the second electronic device, the first information including one or more interface elements on the first interface and the tag information of the one or more interface elements.

[0263] Figure 13 A schematic block diagram of the apparatus 1200 provided by the embodiments of the present application is shown. The apparatus 1200 can be arranged in the second electronic device in the above-mentioned Figure 13 embodiments of the present application. The apparatus 1200 includes a receiving unit 1210, configured to receive first information sent by the first electronic device, the first information including one or more interface elements on a first interface displayed by the first electronic device and label information of the one or more interface elements; and a display unit 1220, configured to, in response to receiving the label information, display a second interface according to the label information of the one or more interface elements, the second interface displaying at least some of the one or more interface elements.

[0264] ​ A schematic structural diagram of an electronic device 1300 provided by the embodiments of the present application is shown. As shown in the figure, ​ the electronic device includes one or more processors 1310, one or more memories 1320, and the one or more memories 1320 store one or more computer programs including instructions. When the instructions are run by the one or more processors 1310, the first electronic device or the second electronic device performs the technical solutions in the above-mentioned embodiments.

[0265] The embodiments of the present application provide a system including a first electronic device and a second electronic device, and the system is used to perform the technical solutions in the above-mentioned embodiments. The implementation principles and technical effects are similar to those of the above-mentioned method-related embodiments, and thus are not described herein again.

[0266] The embodiments of the present application provide a computer program product, when the computer program product is run on a first electronic device (or, a notebook computer in the above-mentioned embodiments), the first electronic device performs the technical solutions in the above-mentioned embodiments. The implementation principles and technical effects are similar to those of the above-mentioned method-related embodiments, and thus are not described herein again.

[0267] The embodiments of the present application provide a computer program product, when the computer program product is run on a second electronic device (or, a mobile phone in the above-mentioned embodiments), the second electronic device performs the technical solutions in the above-mentioned embodiments. The implementation principles and technical effects are similar to those of the above-mentioned method-related embodiments, and thus are not described herein again.

[0268] The embodiments of the present application provide a readable storage medium, the readable storage medium contains instructions, when the instructions are run on a first electronic device (or, a notebook computer in the above-mentioned embodiments), the first electronic device performs the technical solutions in the above-mentioned embodiments. The implementation principles and technical effects are similar, and thus are not described herein again.

[0269] The embodiment of the present application provides a readable storage medium, the readable storage medium contains instructions, when the instructions are run in the second electronic device (or, the mobile phone in the above embodiment), the second electronic device executes the technical solution of the above embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0270] The embodiment of the present application provides a chip, the chip is used for executing instructions, when the chip is run, the technical solution in the above embodiment is executed. Its implementation principle and technical effect are similar, and will not be repeated here.

[0271] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solution. Professionals can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0272] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0273] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be realized by other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0274] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0275] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0276] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0277] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A system, characterized by The system comprises a first electronic device and a second electronic device, the first electronic device communicates with the second electronic device through a close-range wireless connection, wherein, the first electronic device is configured to, in response to receiving a first indication, display a first interface and send first information to the second electronic device, the first information comprising one or more interface elements on the first interface and label information of the one or more interface elements; the second electronic device is configured to, in response to receiving the first information, display a second interface according to the label information of the one or more interface elements, the second interface displaying at least part of the one or more interface elements; the second electronic device is specifically configured to display the second interface according to the label information of the one or more interface elements and a first mapping relationship, the first mapping relationship being a mapping relationship between a display area of the at least part of the interface elements and the label information of the at least part of the interface elements; the first mapping relationship is associated with a scenario corresponding to the first interface; the at least part of the interface elements comprises a thumbnail of a photo, and the second electronic device is further configured to, in response to detecting an input of the user dragging the thumbnail to a first folder, send request information to the first electronic device, the request information being used to request information of the photo, the first folder being a folder stored locally by the second electronic device; the first electronic device is further configured to, in response to receiving the request information, perform an operation of sending the photo to the second electronic device.

2. The system of claim 1, wherein, the first electronic device is specifically configured to: in response to detecting an operation of the user starting a first application program, display the first interface and send the first information to the second electronic device.

3. The system of claim 2, wherein, the first electronic device is specifically configured to: in response to detecting an operation of the user starting the first application program, display the first interface and prompt the user whether to send one or more interface elements on the first interface to the second electronic device for display; in response to detecting an operation of the user determining to send the one or more interface elements on the first interface to the second electronic device for display, send the first information to the second electronic device.

4. The system of claim 2, wherein, the first electronic device is further configured to, in response to detecting an operation of the user starting the first application program, display the first interface and send first indication information to the second electronic device, the first indication information being used to indicate that the first electronic device has started the first application program; the second electronic device is further configured to, in response to receiving the first indication information, prompt the user whether to display a display interface of the first application program through the second electronic device; the second electronic device is further configured to, in response to detecting an operation of the user determining to display the display interface of the first application program through the second electronic device, send first request information to the first electronic device, the first request information being used to request to display interface elements on the first interface; The first electronic device is specifically configured to, in response to receiving the first request information, send the first information to the second electronic device.

5. The system of claim 1, wherein, The second electronic device is further configured to, in response to detecting the operation of the user, send second request information to the first electronic device, the second request information being used to request display of interface elements on the first interface. The first electronic device is specifically configured to, in response to receiving the second request information sent by the second electronic device, display the first interface and send the first information to the second electronic device.

6. The system of any one of claims 1 to 5, wherein, The first electronic device is further configured to, before sending the first information to the second electronic device, determine that a distance between the first electronic device and the second electronic device is less than or equal to a preset distance.

7. The system of any one of claims 1 to 5, wherein, The at least part of the interface elements includes a first interface element, The first electronic device is further configured to, in response to the first interface element being updated, send second information to the second electronic device, the second information including the updated first interface element and label information of the first interface element. The second electronic device is further configured to, in response to receiving the second information, display the updated first interface element on the second interface.

8. The system of any one of claims 1 to 5, wherein, The first electronic device is further configured to, in response to performing the operation, update a third interface element, the at least part of the interface elements including the third interface element. The first electronic device is further configured to send third information to the second electronic device, the third information including the updated third interface element and label information of the third interface element. The second electronic device is further configured to, in response to receiving the third information, display the updated third interface element on the second interface.

9. The system of any one of claims 1 to 5, 8, wherein, The first information further includes information of a scene corresponding to the first interface.

10. A display method characterized by comprising: The method is applied to a first electronic device, the first electronic device communicates with a second electronic device through a close-range wireless connection, and the method includes: The first electronic device receives first indication; In response to receiving the first indication, the first electronic device displays a first interface and sends first information to the second electronic device, the first information including one or more interface elements on the first interface and label information of the one or more interface elements, wherein at least part of the one or more interface elements is displayed on a second interface of the second electronic device, and the at least part of the interface elements includes a thumbnail of a photo. The method further includes: The first electronic device sends information of a first mapping relationship to the second electronic device, the first mapping relationship being a mapping relationship between a display area of an interface element and label information of the interface element. The first mapping relationship is associated with a scene corresponding to the first interface; and the method further includes: In response to detecting the input of the user dragging the thumbnail to a first folder in the second electronic device, the first electronic device receives request information sent by the second electronic device, the request information being used to request information of the photo, and the first folder being a folder stored locally by the second electronic device; In response to receiving the request information, the first electronic device performs an operation of sending the photo to the second electronic device.

11. The method of claim 10, wherein, The first electronic device receives a first indication, including: The first electronic device detects an operation of the user starting a first application.

12. The method of claim 11, wherein, In response to receiving the first indication, the first electronic device displays a first interface and sends first information to the second electronic device, including: In response to detecting the operation of the user starting the first application, the first interface is displayed and the user is prompted whether to send one or more interface elements on the first interface to the second electronic device for display; In response to detecting the operation of the user determining to send the one or more interface elements on the first interface to the second electronic device for display, the first information is sent to the second electronic device.

13. The method of claim 11, wherein, In response to receiving the first indication, the first electronic device sends first information to the second electronic device, including: The first electronic device receives first request information sent by the second electronic device, the first request information being used to request display of interface elements on the first interface; The first electronic device sends the first information to the second electronic device in response to receiving the first request information.

14. The method according to any one of claims 10 to 13, characterized in that, The method further includes: Before sending the first information to the second electronic device, the first electronic device determines that a distance between the first electronic device and the second electronic device is less than or equal to a preset distance.

15. The method according to any one of claims 10 to 13, characterized in that, The one or more interface elements include a first interface element, and the method further includes: In response to the first interface element being updated, the first electronic device sends second information to the second electronic device, the second information including the updated first interface element and label information of the first interface element.

16. The method according to any one of claims 10 to 13, characterized in that, The method further includes: In response to performing the operation, the first electronic device updates a third interface element, and the one or more interface elements include the third interface element; The first electronic device sends third information to the second electronic device, the third information including the updated third interface element and label information of the third interface element.

17. A display method characterized by comprising: The method is applied to a second electronic device, the second electronic device being in communication with a first electronic device through a close-range wireless connection, and the method includes: The second electronic device receives first information sent by the first electronic device, the first information including one or more interface elements on a first interface displayed by the first electronic device and label information of the one or more interface elements; In response to receiving the tag information, the second electronic device displays a second interface according to the tag information of the one or more interface elements, the second interface displaying at least some of the one or more interface elements; The second electronic device displays a second interface according to the tag information of the one or more interface elements, including: The second electronic device displays the second interface according to the tag information of the one or more interface elements and a first mapping relationship, the first mapping relationship being a mapping relationship between a display area of the at least some interface elements and the tag information of the at least some interface elements; The first mapping relationship is associated with a scenario corresponding to the first interface; The at least some interface elements include a thumbnail of a photo, and the method further includes: In response to detecting that the user drags the thumbnail to a first folder on the second electronic device, the second electronic device sends request information to the first electronic device, the request information being used to request information of the photo, the first folder being a folder stored locally by the second electronic device; Receiving the photo sent by the first electronic device.

18. The method of claim 17, wherein, Before the second electronic device receives the first information sent by the first electronic device, the method further includes: The second electronic device receives first indication information sent by the first electronic device, the first indication information being used to indicate that the first electronic device has started a first application program; In response to receiving the first indication information, the second electronic device prompts the user whether to display a display interface of the first application program through the second electronic device; In response to detecting that the user determines to display the display interface of the first application program through the second electronic device, the second electronic device sends first request information to the first electronic device, the first request information being used to request to display an interface element on the first interface.

19. The method of claim 17, wherein, Before the second electronic device receives the first information sent by the first electronic device, the method further includes: In response to detecting the operation of the user, the second electronic device sends second request information to the first electronic device, the second request information being used to request to display an interface element on the first interface.

20. The method of any one of claims 17-19, wherein, The at least some interface elements include a first interface element, and the method further includes: The second electronic device receives second information sent by the first electronic device, the second information including an updated first interface element and tag information of the first interface element; In response to receiving the second information, the second electronic device displays the updated first interface element on the second interface.

21. The method of any one of claims 17-19, wherein, The at least some interface elements include a third interface element, and the method further includes: If the first electronic device updates the third interface element after receiving the request information, the second electronic device receives third information sent by the first electronic device, the third information including an updated third interface element and tag information of the third interface element; In response to receiving the third information, the third electronic device displays the updated third interface element on the second interface.

22. An electronic device, comprising: The electronic device includes: one or more processors; one or more memories; The one or more memories store one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 10-16.

23. An electronic device, comprising: The electronic device includes: one or more processors; one or more memories; The one or more memories store one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 17-21.

24. A computer-readable storage medium, characterized in that, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 10-16; or, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 17-21.

Images