Equipment collaboration method and electronic equipment

Abstract

The invention provides an equipment collaboration method and electronic equipment, relates to the technical field of terminals, and aims to solve the problems that a collaboration display interface is simple and the user experience feeling is poor. The method is applied to first equipment, the first equipment is connected with second equipment, the second equipment comprises an appearance assembly, and the method comprises the steps that information of a first interface from the second equipment is received, and the first interface is an interface currently displayed by the second equipment; displaying a first window in response to the information of the first interface; the first window comprises a screen projection interface, and the content of the screen projection interface is a mirror image of the first interface; the first window further comprises an appearance element, and the display effect presented by the appearance element is the same as the visual effect presented by the appearance component of the second equipment. The method and the device are applied to a device collaboration process.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a device collaboration method and electronic device. Background Technology

[0002] With the development of electronic technology and mobile internet, users can own more electronic devices such as mobile phones, tablets, and personal computers (PCs). Data sharing can be achieved between these various electronic devices.

[0003] To facilitate data sharing between different electronic devices, collaborative processing is required. However, current collaborative display interfaces are relatively simple, resulting in a poor user experience. Summary of the Invention

[0004] This application provides a device collaboration method and electronic device. When devices collaborate, the first device displays the appearance elements and projection interface of the second device, which highly restores the real second device, improves the realism, overall feel and refinement of device collaboration, and also enhances the user's operating experience.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a device collaboration method applied to a first device, the first device being connected to a second device, the second device including an appearance component, the method comprising: receiving information from a first interface of the second device, the first interface being the interface currently displayed on the second device; responding to the information from the first interface, displaying a first window; the first window including a projection interface, the content of the projection interface being a mirror image of the first interface; the first window also including appearance elements, the display effect of the appearance elements being the same as the visual effect presented by the appearance component of the second device.

[0007] In this application, when the first device and the second device are in a collaborative state, the first device displays the appearance elements and projection interface of the second device, highly replicating the display of the second device. This enhances the realism, overall feel, and sophistication of the collaborative device interaction, thereby improving the user's visual experience. Furthermore, the user can manipulate these appearance elements to trigger corresponding functions on the second device, increasing the user experience in manipulating these elements during collaborative device interaction, expanding the functionality and diversity of the collaborative interface, and enriching the user experience.

[0008] in,

[0009] According to the first aspect, or any implementation of the first aspect above, the appearance component includes a functional component, the appearance element includes a functional element, and the display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device. The method further includes: in response to a user's first operation on the functional element, sending a collaborative event to the second device; receiving information from a second interface of the second device; the second interface being the interface updated by the second device in response to the collaborative event; and updating the projection interface in the first window according to the information of the second interface.

[0010] According to the first aspect, or any implementation of the first aspect above, the method further includes: in response to the user inputting a second operation within a preset range of the functional element, displaying a target element corresponding to the functional element; wherein the second operation is an operation triggered by the user to display the target element.

[0011] In some examples, the first operation includes a second and a third operation. The second operation is the user triggering the display of the target element corresponding to the functional element, and the third operation is the user's action on the target element.

[0012] Specifically, the first device responds to the user's input of a second operation within the preset range of the functional element and displays the target element corresponding to the functional element; the first device responds to the user's third operation on the target element and generates a collaborative event.

[0013] In other examples, the first operation includes a fourth operation. This fourth operation is pre-defined and involves directly manipulating the functional element to trigger a collaborative event corresponding to the functional element generated by the first device.

[0014] Specifically, the first device responds to the user's fourth operation on the functional element by generating a collaborative event.

[0015] In this application, the first device responds to the user's operation on the functional elements in the appearance elements, generates a collaborative event, interacts with the second device based on the collaborative event, obtains the information of the second interface corresponding to the collaborative event, and updates the projection interface based on the information of the second interface. This can restore the user's operation experience of the functional components of the source device, increase the realism of the operation, and improve the user experience.

[0016] According to the first aspect, or any implementation of the first aspect above, the method further includes: receiving information from the appearance component of the second device from the second device.

[0017] According to the first aspect, or any implementation of the first aspect above, the method further includes: receiving device information of the second device from the second device; and obtaining information of the appearance components of the second device from the server based on the device information of the second device.

[0018] In this application, the first device can obtain information about the appearance components of the second device from the second device or a server, so as to subsequently generate appearance elements corresponding to the appearance components based on the information of the appearance components, thereby enriching the display content when the devices collaborate. Visually displaying a complete device enhances the realism and completeness of the collaborative display, improving the user experience.

[0019] According to the first aspect, or any implementation of the first aspect above, the method further includes: generating a projection interface based on the information of the first interface; and generating appearance elements based on the information of the appearance components of the second device.

[0020] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining information about the updated appearance components of the second device; and updating the display effect of the appearance elements in the first window according to the updated appearance component information.

[0021] In this application, the first device can display the status changes of the second device so that the user can quickly know the status changes of the second device. This can enhance the interactivity between collaborative devices, improve the display effect of device collaboration, and enhance the fun and user experience of collaborative use.

[0022] According to the first aspect, or any implementation of the first aspect above, the information of the appearance components of the second device includes one or more of the following: the specifications of the second device, the product drawing of the second device, or the appearance parameters of the second device.

[0023] In some examples, if the information of the appearance components of the second device includes the specifications of the second device, the product image of the second device, and the appearance parameters of the second device, then the display state of the appearance elements in the first window and the projection interface is the preset state.

[0024] In this application, the first device displays the appearance elements and projection interface in the first window according to the preset state, which can speed up the display efficiency and achieve efficient display.

[0025] According to the first aspect, or any implementation of the first aspect above, the information of the appearance components of the second device also includes the usage status of the second device.

[0026] In some examples, the second device determines its usage status based on data detected by the sensors.

[0027] In other examples, the second device determines its usage status based on a first preset rule and data detected by the sensor.

[0028] In some examples, if the information of the appearance components of the second device includes the specifications of the second device, the product image of the second device, the appearance parameters of the second device, and the usage status of the second device, then the display status of the appearance elements in the first window and the projection interface corresponds to the usage status of the second device.

[0029] For example, if the usage status of the second device is determined based on data detected by sensors (i.e., real-time usage status), then the display status of the appearance elements and the projection interface in the first window is the same as the usage status of the second device.

[0030] Alternatively, if the usage status of the second device is determined based on data detected by sensors (i.e., real-time usage status), then the display status of the appearance elements in the first window and the projection interface is determined based on the usage status of the second device and the second preset rule.

[0031] For example, if the usage status of the second device is determined based on the sensor detection data and the first preset rule, then the display status of the appearance elements in the first window and the projection interface is the same as the usage status of the second device.

[0032] In this application, the first device determines the display status of the appearance elements in the first window and the projection interface based on the usage status of the second device, so as to more intuitively understand and operate the displayed content in the first window and enhance the user experience.

[0033] According to the first aspect, or any implementation of the first aspect above, the appearance components include one or more of the following: frame, volume buttons, power button, camera, shell, interface, and sound outlet.

[0034] Secondly, this application provides a device collaboration method applied to a second device connected to a first device. The second device includes an appearance component. The method includes: displaying a first interface; the first interface being the interface currently displayed on the second device; sending information about the first interface to the first device; the information about the first interface being used by the first device to display a first window, the first window including a projection interface, the content of which is a mirror image of the first interface; the first window also including appearance elements, the display effect of which is the same as the visual effect of the appearance component of the second device.

[0035] According to the second aspect, or any implementation of the second aspect above, the appearance component includes a functional component, and the appearance element includes a functional element. The display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device. The method further includes: receiving a collaboration event from the first device; the collaboration event is generated by the second device in response to a first operation by the user on the functional element; sending information about a second interface to the first device, the second interface being the interface updated by the second device in response to the collaboration event, and the information about the second interface being used to update the projection interface in the first window.

[0036] According to the second aspect, or any implementation of the second aspect above, the method further includes: sending information about the appearance components of the second device to the first device;

[0037] According to the second aspect, or any implementation of the second aspect above, the method further includes: sending device information of the second device to the first device, wherein the device information of the second device is used by the first device to obtain information of the appearance components of the second device from the server based on the device information of the second device.

[0038] According to the second aspect, or any implementation of the second aspect above, the method further includes: sending information about the updated appearance components of the second device to the first device, wherein the updated appearance component information is used by the first device to update the display effect of the appearance elements in the first window.

[0039] According to the second aspect, or any implementation of the second aspect above, the information of the appearance components of the second device includes one or more of the following: the specifications of the second device, the product drawing of the second device, or the appearance parameters of the second device.

[0040] According to the second aspect, or any implementation of the second aspect above, the information of the appearance components of the second device also includes the usage status of the second device.

[0041] According to the second aspect, or any implementation of the second aspect above, the appearance components include one or more of the following: frame, volume buttons, power button, camera, shell, interface, and sound outlet.

[0042] Thirdly, this application provides an electronic device comprising: a processor and a memory, the memory being coupled to the processor, the memory being used to store computer-readable instructions, wherein when the processor reads the computer-readable instructions from the memory, the electronic device causes the electronic device to perform a method as described in the first aspect and any one of the embodiments of the first aspect, or a method as described in the second aspect and any one of the embodiments of the second aspect.

[0043] Fourthly, this application provides a device coordination apparatus, which includes: a processor and a memory, the memory being coupled to the processor, the memory being used to store computer-readable instructions, and when the processor reads the computer-readable instructions from the memory, causing the electronic device to perform a method as described in the first aspect and any one of the embodiments of the first aspect, or a method as described in the second aspect and any one of the embodiments of the second aspect.

[0044] Fifthly, this application provides a chip system including at least one processor and at least one interface circuit. The at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor. The at least one processor executes the instructions and performs the method as described in the first aspect and any one of the embodiments of the first aspect, or the method as described in the second aspect and any one of the embodiments of the second aspect.

[0045] Sixthly, this application provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform a method as described in the first aspect and any one of the embodiments of the first aspect, or a method as described in the second aspect and any one of the embodiments of the second aspect.

[0046] In a seventh aspect, this application provides a computer program product comprising: a computer program or instructions that, when executed on a computer, cause the computer to perform a method as described in the first aspect and any one of the embodiments of the first aspect, or a method as described in the second aspect and any one of the embodiments of the second aspect.

[0047] The technical effects corresponding to any implementation method of aspects two through seven, and all other aspects, can be found in the first aspect and the technical effects corresponding to any implementation method of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of device collaboration provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of the architecture of the communication system provided in the embodiments of this application;

[0050] Figure 3 A schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application;

[0051] Figure 4 A flowchart illustrating the device collaboration method provided in the embodiments of this application. Figure 1 ;

[0052] Figure 5Electronic device product sample provided in the embodiments of this application Figure 1 ;

[0053] Figure 6 Electronic device product sample provided in the embodiments of this application Figure 2 ;

[0054] Figure 7 Electronic device product sample provided in the embodiments of this application Figure 3 ;

[0055] Figure 8 This application provides a schematic diagram of a device collaboration scenario. Figure 1 ;

[0056] Figure 9 This application provides a schematic diagram of a device collaboration scenario. Figure 2 ;

[0057] Figure 10 This application provides a schematic diagram of a device collaboration scenario. Figure 3 ;

[0058] Figure 11 This application provides a schematic diagram of a device collaboration scenario. Figure 4 ;

[0059] Figure 12 This application provides a schematic diagram of a device collaboration scenario. Figure 5 ;

[0060] Figure 13 A flowchart illustrating the device collaboration method provided in the embodiments of this application. Figure 2 ;

[0061] Figure 14 This is a schematic diagram of the structure of the device collaboration device provided in the embodiments of this application;

[0062] Figure 15 This is a schematic diagram of the chip system provided in an embodiment of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two).

[0064] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0065] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0066] In some examples, data sharing can be achieved through multi-screen collaboration technology (which can also be described as multi-device collaboration technology). In a multi-screen collaboration scenario, after the source device (which can also be described as the projection device) and the sink device (which can also be described as the device being projected onto) establish a connection via wired or wireless means, the display interface of the source device can be projected onto the display screen of the sink device.

[0067] like Figure 1As shown, taking mobile phone 101 as the source device and PC 102 as the target device, this example illustrates multi-screen collaboration between mobile phone 101 and PC 102. When mobile phone 101 and PC 102 establish a connection wirelessly or via wired connection, and mobile phone 101 enables screen mirroring, a window 103 (which can also be described as a mobile phone window or a screen mirroring window) can be displayed on the screen of PC 102. This window 103 can be used to display the display interface of mobile phone 101. During multi-screen collaboration, the user can use the keyboard, mouse, or touchscreen of PC 102 to control the display interface of mobile phone 101 displayed in window 103 of PC 102. For example, the user can use the mouse of PC 102 to click on the icon of the application (APP) of mobile phone 101 displayed in window 103, thereby triggering mobile phone 101 to open the application.

[0068] However, in this example, window 103 displayed on PC102 only shows the interface displayed on the phone 101 screen (such as the phone's desktop and application icons), and does not display the phone 101's appearance elements (which can be understood as images corresponding to the phone 101's appearance components). For example, appearance components include the phone 101's frame and hardware buttons (such as volume buttons and power buttons). That is, after multi-screen collaboration, only one interface is displayed on the target device (i.e., the interface displayed on the source device is presented), failing to present the overall feel and realism of the source device. Users can only operate the content displayed in the interface, lacking the experience of controlling the appearance components of the source device. Moreover, after different source devices enable screen mirroring, the same interface is displayed on the target device, failing to reflect the specific differences between different source devices (such as differences in frame color, hardware button positions, etc.). The collaboratively displayed content lacks refinement and a high-end feel, the interface is relatively simple, and the user experience is poor.

[0069] To address the aforementioned issues, this application provides a device collaboration method in which the target device displays the appearance elements and projection interface of the source device during device collaboration, highly replicating the real source device, enhancing the realism, overall feel, and sophistication of device collaboration, and improving the user's operating experience.

[0070] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0071] The device coordination method provided in this application embodiment can be applied to… Figure 2 In the communication system 200 shown. For example... Figure 2 As shown, the communication system 200 may include a source device 201 and a target device 202.

[0072] The source device 201 and the target device 202 are electronic devices with displays. Electronic devices can also be called terminal equipment, terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc.

[0073] When the source device 201 and the target device 202 are performing device collaboration (taking the source device 201 as a screen projection display on the target device 202 as an example), the target device 202 (i.e. the target device) can not only display the projection interface corresponding to the current display interface of the source device 201, but also display the appearance elements corresponding to the appearance components of the source device 201.

[0074] As an example, source device 201 may include acquisition module 203 and coordination module 204. Target device 202 may include coordination module 205.

[0075] Source device 201 and target device 202 can communicate via a communication module ( Figure 2 (Not shown in the image) Establish a communication connection.

[0076] The acquisition module 203 of the source device 201 can acquire first information, which may include information about the appearance components of the source device and information about the first interface. Specifically, the information about the appearance components of the source device may include parameter information corresponding to the appearance components when the source device is triggered to start the device collaboration function. The information about the first interface may include control information and layout information of the first interface on the source device when the source device is triggered to start the device collaboration function. This first interface may be the interface currently displayed on the source device when the source device is triggered to start the device collaboration function.

[0077] The acquisition module 203 of the source device 201 can send the first information to the coordination module 204.

[0078] The source device 201 can send the first information to the collaboration module 205 of the target device 202 through the collaboration module 204.

[0079] The collaboration module 205 of the target device 202 generates a first window based on the first information and displays the first window on the display screen of the target device 202. The first window includes appearance elements and a projection interface. The display effect presented by the appearance elements is the same as the visual effect presented by the appearance components of the source device 201, and the content of the projection interface is a mirror image of the content of the first interface. The appearance elements can be understood as a virtualized representation of the appearance components of the source device 201.

[0080] In some embodiments, the source device 201 and the target device 202 may be devices with display functions such as mobile phones, PCs (including desktop computers or laptops), tablet computers, smart TVs, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, personal digital assistants (PDAs), wearable electronic devices, in-vehicle devices, and virtual reality devices.

[0081] The operating systems installed on the source device 201 and the target device 202 include, but are not limited to, those that are not included in the above. Alternatively, other operating systems may be used. This application does not impose any restrictions on the specific types of the source and target devices or the operating systems installed on them.

[0082] In this device, the source device 201 and the target device 202 can be the same type of electronic device or different types of electronic devices. For example, both the source device 201 and the target device 202 can be mobile phones. Alternatively, both the source device 201 and the target device 202 can be PCs. Alternatively, the source device 201 can be a mobile phone, and the target device 202 can be a PC or tablet computer. Alternatively, the source device 201 can be a tablet computer, and the target device 202 can be a smart TV.

[0083] Optionally, the number of source devices 201 can be one or more. One or more source devices 201 can send their own first information to a target device 202, so that the target device 202 can display a corresponding first window based on the received first information. This allows different devices to display and share data on the same device, thereby realizing multi-device multi-screen collaborative display functionality.

[0084] Specifically, the first information sent by the source device 201 includes information about the appearance components of the source device 201 and information about the first interface. After receiving the first information from the source device 201, the target device 202 generates appearance elements based on the information about the appearance components of the source device 201. The display effect presented by these appearance elements is the same as the visual effect presented by the appearance components of the source device 201. The target device 202 also generates a projection interface based on the information about the first interface in the first information. This projection interface is a mirror image of the first interface, and its content is a mirror image of the content of the first interface. That is, the display content in the projection interface is the same as the display content in the first interface. The target device 202 overlays the appearance elements and the projection interface to display the first window.

[0085] Optionally, the number of target devices 202 can be one or more. The source device 201 can send first information from its own information to one or more target devices 202, so that the one or more target devices 202 can display a first window on their screens based on the received first information. The first information includes information about the first interface and information about the appearance components of the source device 201. The first window includes appearance elements and a projection interface. This allows different users to use the relevant functions provided by the source device 201 on different target devices 202, thereby achieving multi-device, multi-screen collaborative display functionality.

[0086] In some embodiments, the source device 201 may establish a communication connection with the target device 202, and based on the established communication connection, realize multi-screen collaborative interaction functions across devices and systems.

[0087] Specifically, the communication connection established between the source device 201 and the target device 202 can be a wireless communication connection. Alternatively, the communication connection established between the source device 201 and the target device 202 can be a wired communication connection. This application does not limit the specific communication connection method between the source device 201 and the target device 202.

[0088] For example, the source device or target device in the embodiments of this application can be Figure 3 The electronic device shown. Figure 3 This is a schematic diagram of the hardware structure of the electronic device 300 provided in an embodiment of this application.

[0089] Electronic device 300 may include processor 310, internal memory 320, antenna 1, antenna 2, radio frequency module 330, communication module 340, display screen 350, sensor module 360, etc.

[0090] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 300. In other embodiments of this application, the electronic device 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0091] Processor 310 may include one or more processing units, such as: application processor (AP), system-on-chip (SoC), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The controller may be the central nervous system and command center of electronic device 300. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0092] In this embodiment, taking electronic device 300 as the source device as an example, processor 310 can obtain first information, which includes information about the appearance components of the source device and information about the first interface. The information about the appearance components of the source device may include parameter information corresponding to the appearance components when the source device is triggered to start the device collaboration function. The information about the first interface may include control information and layout information of the first interface in the source device when the source device is triggered to start the device collaboration function. This first interface may be the interface currently displayed on the source device when the source device is triggered to start the device collaboration function.

[0093] In this embodiment, taking electronic device 300 as the target device, processor 310 can generate a first window based on first information. The first window includes appearance elements and a projection interface. The display effect presented by the appearance elements is the same as the visual effect presented by the appearance components of the source device, and the content of the projection interface is a mirror image of the content of the first interface. Processor 310 can also send the first window to display screen 350 so that display screen 350 displays the first window.

[0094] The processor 310 may also include a memory for storing instructions and data. In this embodiment, the memory may be used to store information such as the appearance components of the aforementioned source device.

[0095] In some embodiments, the memory in the processor 310 is a cache memory. This memory can store instructions or data that the processor 310 has just used or that are used repeatedly. If the processor 310 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0096] In some embodiments, the processor 310 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0097] The wireless communication function of electronic device 300 can be realized through antenna 1, antenna 2, radio frequency module 330, communication module 340, modem processor and baseband processor, etc.

[0098] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 300 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0099] The radio frequency (RF) module 330 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use in electronic devices 300. The RF module 330 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The RF module 330 can also amplify the signal modulated by the modem processor and radiate it as electromagnetic waves via antenna 1. In some embodiments, at least some functional modules of the RF module 330 can be housed within the processor 310. In some embodiments, at least some functional modules of the RF module 330 and at least some modules of the processor 310 can be housed in the same device.

[0100] The modem processor may include a modulator and a demodulator. The modulator modulates a low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor displays images or videos via the display screen 350. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 310 and may be housed within the same device as the radio frequency module 330 or other functional modules.

[0101] The communication module 340 can provide solutions for wireless communication applications on the electronic device 300, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The communication module 340 can be one or more devices integrating at least one communication processing module. The communication module 340 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 310. The communication module 340 can also receive signals to be transmitted from processor 310, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0102] In some embodiments, antenna 1 of electronic device 300 is coupled to radio frequency module 330, and antenna 2 is coupled to communication module 340, enabling electronic device 300 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0103] For example, the electronic device 300 provided in this application embodiment can send messages or instructions to other electronic devices through antenna 1, antenna 2, radio frequency module 330, communication module 340, etc. Alternatively, the electronic device provided in this embodiment can receive information or instructions sent by other electronic devices through antenna 1, antenna 2, radio frequency module 330, communication module 340, etc.

[0104] In this embodiment, taking electronic device 300 as the source device, the source device establishes a communication connection with other electronic devices (such as the target device) through radio frequency module 330 or communication module 340. The source device can also send first information to other electronic devices (such as the target device) through radio frequency module 330 or communication module 340, so that the other electronic devices (such as the target device) can display a first window based on the first information. The first information includes information about the appearance components of the source device and information about the first interface. The first window includes appearance elements and a projection interface.

[0105] In this embodiment, taking electronic device 300 as the target device as an example, the target device establishes a communication connection with other electronic devices (such as source devices) through radio frequency module 330 or communication module 340. The target device can also receive first information sent by other electronic devices (such as source devices) through radio frequency module 330 or communication module 340.

[0106] Electronic device 300 implements display functions through a GPU, display screen 350, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 350 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 310 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0107] The display screen 350 is used to display images, videos, etc. The display screen 350 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include one or N displays 350, where N is a positive integer greater than 1.

[0108] In this embodiment of the application, taking electronic device 300 as the target device as an example, the display screen 350 of the target device displays the first window.

[0109] The sensor module 360 ​​of the electronic device 300 may specifically include: gyroscope sensor, accelerometer sensor, angle sensor, magnetic sensor, pressure sensor, barometric pressure sensor, distance sensor, proximity sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor, bone conduction sensor, etc.

[0110] In this embodiment, the electronic device 300 can determine its usage state through the sensor module 360. The usage state refers to the current physical form of the electronic device 300. For example, the electronic device 300 is currently in landscape or portrait mode. Alternatively, the electronic device 300 is currently in a folded state (also referred to as folded state) or an unfolded state (also referred to as unfolded state), etc.

[0111] For example, the electronic device 300 provided in this application embodiment can determine its usage state based on the sensor module 360. For instance, the gyroscope sensor in the sensor module 360 ​​can detect the rotation and orientation changes of the electronic device 300 (which can also be described as the motion posture of the electronic device 300), and the electronic device 300 determines its usage state based on the data detected by the gyroscope sensor. Alternatively, the accelerometer sensor in the sensor module 360 ​​can detect the acceleration and motion state of the electronic device 300, and the electronic device 300 combines an algorithm with the data detected by the accelerometer sensor to determine its usage state. For example, whether the electronic device 300 is currently in landscape mode or portrait mode.

[0112] Optionally, the display screen 350 of the electronic device 300 is a foldable screen. In some embodiments, the foldable screen may be a flexible foldable screen, which can be folded along the folding edge to form multiple screens. In other embodiments, the foldable screen may be a multi-screen foldable screen. The multi-screen foldable screen may include multiple screens. These multiple screens may be connected sequentially by a folding axis (which may also be described as a pivot). Each screen can rotate about the folding axis connected to it to realize the folding of the multi-screen foldable screen. For example, taking the foldable screen as a flexible foldable screen, if the electronic device 300 has only one folding edge, after folding along the folding edge, the display screen (i.e., the foldable screen) can be folded into two independently displayable screens, such as a first screen and a second screen. Accordingly, the usage state of the electronic device 300 with the foldable screen may include a folded state and an unfolded state.

[0113] The sensor module 360 ​​of the electronic device 300 may further include a folding angle sensor. For example, the electronic device 300 provided in this embodiment can determine its usage state based on the sensor module 360. For instance, taking a flexible folding screen as an example, where the electronic device 300 has only one folding edge, and the folding screen is divided into a first screen and a second screen after folding, the folding angle sensor in the sensor module 360 ​​can detect the angle between the first screen and the second screen. The electronic device 300 determines its usage state based on the data detected by the folding angle sensor. For example, whether the electronic device 300 is currently in a folded state or an unfolded state.

[0114] Alternatively, in the sensor module 360, the gyroscope sensor can detect the motion posture of the electronic device 300, the accelerometer sensor can detect the acceleration and motion state of the electronic device 300, and the magnetic sensor can detect the magnetic field (such as magnetic flux / magnetic intensity) of the electronic device 300. The electronic device 300 can determine its usage status based on the data detected by the gyroscope sensor, the accelerometer sensor, and / or the magnetic sensor, according to an algorithm.

[0115] It is understood that in some embodiments, when electronic device 300 interacts collaboratively with other electronic devices, it can send the usage status of the electronic device determined by sensor module 360 ​​to the other electronic devices. In other embodiments, electronic device 300 can also send the data detected by sensor module 360 ​​to the other electronic devices. This allows other electronic devices to simulate and reconstruct the usage status of electronic device 300 based on the received data, thereby improving the realism and real-time performance of the collaborative interaction display.

[0116] Internal memory 320 can be used to store computer executable program code, which includes instructions. Processor 310 executes various functional applications and data processing of electronic device 300 by running the instructions stored in internal memory 320. Internal memory 320 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 300 (such as audio data, phonebook, etc.). Furthermore, internal memory 320 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0117] The device coordination method of this application will be described in detail below with reference to specific embodiments and accompanying drawings. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0118] The device collaboration method in this application embodiment can be applied to any device collaboration scenario (which can also be described as a multi-device collaboration scenario or a multi-screen collaboration scenario). For example, office collaboration scenarios, entertainment scenarios, media scenarios, education scenarios, etc.

[0119] The scenarios for collaborative office work can include multiple devices displaying information collaboratively on the same screen during meetings, such as collaborative document editing on multiple devices. Entertainment scenarios can include multiple devices playing online games collaboratively. For example, a mobile phone can act as a game controller, a tablet can display game content, and the mobile phone's display can be projected onto the tablet. Media scenarios can include multiple devices playing videos collaboratively. For example, a PC can display a video, a mobile phone can be projected onto the PC, and supplementary information about the video content can be displayed. Educational scenarios can include multiple devices collaboratively conducting teaching demonstrations. This application does not limit the scope of device collaboration scenarios.

[0120] It is understood that the system architecture and business scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute the only limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0121] The following example uses a mobile phone as the source device and a PC as the target device. Figure 4 This is a flowchart illustrating a device collaboration method provided in an embodiment of this application. It should be noted that this method does not rely on... Figure 4 The specific order described below is a limitation. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0122] S400: The source device and the target device establish a communication connection.

[0123] In this embodiment, the source device can be an electronic device that initiates device collaboration. The target device is an electronic device that collaborates with the source device to display content related to the source device.

[0124] It is understandable that the source device activates the device collaboration function or screen mirroring function before establishing a communication connection with the target device.

[0125] In some embodiments, the source device may establish a communication connection with the target device in response to a user operation.

[0126] As an optional implementation, the source device discovers surrounding devices in response to a user's triggering of the device collaboration function. The source device identifies the electronic device selected by the user from the surrounding devices as the target device for communication connection with the source device, and the source device establishes a communication connection with the target device.

[0127] For example, when the phone (i.e., the source device) is unlocked and the screen is on, the phone displays the desktop. The phone can receive a user's touch gesture, such as swiping down from the top of the screen. In response, the phone can display a control center interface. The user clicks a device collaboration control (or screen mirroring control) in the control center interface. In response to the user's click on the device collaboration control, the phone broadcasts a device collaboration discovery request on the local area network. The phone identifies at least one electronic device (such as a PC, smart TV, etc.) that returns a device collaboration response on the same local area network as a peripheral device and displays information about each peripheral device (such as displaying the device name) on the phone for the user to select. The phone identifies the electronic device (such as the PC) selected by the user from the peripheral devices as the target device. The phone and PC establish a communication connection via WiFi.

[0128] In some embodiments, the source device establishes a communication connection with the target device via a wireless connection communication method.

[0129] The wireless connection communication methods include, but are not limited to, at least one of the following: wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT) (e.g., classic Bluetooth or Bluetooth Low Energy (BLE) Bluetooth), near field communication (NFC), Zigbee, frequency modulation (FM), and infrared (IR), etc.

[0130] For example, if the source and target devices activate NFC, they can establish a communication connection by tapping each other. Alternatively, if the source and target devices are connected to the same local area network, they can establish a communication connection via Wi-Fi. Or, they can establish a communication connection via Bluetooth.

[0131] In other embodiments, the source device establishes a communication connection with the target device via a wired connection.

[0132] For example, a wired communication connection is established between the source device and the target device via a video graphics array (VGA), digital visual interface (DVI), high-definition multimedia interface (HDMI), or data transmission cable. Information transmission is achieved between the source device and the target device through this established wired communication connection.

[0133] This application does not limit the specific implementation method of the communication connection between the source device and the target device.

[0134] S401. The source device obtains first information, which includes information about the appearance components of the source device and information about the first interface.

[0135] In this embodiment, the information of the appearance components of the source device may include parameter information corresponding to the appearance components of the source device when the source device is triggered to start the device collaboration function.

[0136] In some embodiments, information about the appearance components of the source device may include the specifications of the source device, product drawings of the source device, and appearance parameters of the source device.

[0137] It is understandable that the specifications, product images, and appearance parameters of the source device are inherent information of the appearance components when the source device leaves the factory.

[0138] The specifications of the source device may include the brand name of the source device, the device information (such as the device model), the basic information of the source device (such as the device dimensions (such as the length, width and aspect ratio of the device), the body information, the display screen information, etc.).

[0139] Among them, the product sample image of the source device is one or more images used to show the product appearance and design features of the source device. For example, the product sample image of the source device includes multiple images showing the appearance structure and design style of the source device from different angles (such as front, side, back, top view, etc.) and different physical forms.

[0140] Specifically, product images of source devices can showcase the overall design of the device, such as bezel style (non-foldable or foldable bezel), display style, casing (i.e., back cover) style, and camera style. Style can include color, material, size, and design (such as textures or patterns). Product images of source devices can also showcase the device's external components, such as their location, shape, color, and size. These external components can include one or more of the following: casing, bezel, hardware buttons (such as volume buttons and power buttons), interfaces, cameras (such as front and rear cameras), audio jacks, ports, and notch displays.

[0141] Optionally, the product sample image of the source device can also be a 3D image of the source device. For example, a 3D image of the source device obtained by simulation based on a 360° image and / or product data can be used to display the source device in 360°.

[0142] It is understandable that product sample images from the source device are used to help users understand the product's appearance and display effects. In this application, the source device sends its product sample images to the target device, so that the target device can display the appearance elements corresponding to the source device's appearance components based on the source device's actual appearance image, highly replicating the source device's true appearance and improving the user experience.

[0143] For example, if the phone's screen cannot be folded, such as Figure 5 The image shown is a product sample of a mobile phone. This sample includes a front view 501 of the phone in portrait mode, a rear view 502 of the phone in portrait mode, and a front view 503 of the phone in landscape mode.

[0144] It is understandable that the above Figure 5 This is just one example. Product images of mobile phones can include product appearance images from different angles (such as mobile phone product images tilted at a certain angle) and different physical forms, and are not limited here.

[0145] For example, if a phone's screen can be folded, then the phone is a foldable phone. Figure 6 The image shown is a product sample of the foldable screen phone. The product sample includes a front view 601 and a rear view 602 of the foldable screen phone in its unfolded state. The product sample also includes a front view 603 and a rear view 604 of the foldable screen phone in its folded state. The product sample also includes sample images 605 and 606 of the foldable screen phone in its folded state.

[0146] In the unfolded front view 601, the phone's display screen can be folded along the folding edge 6011 (this folding edge 6011 is not shown in the actual product) to divide into two independently displayable screens, such as screen A located to the left of the folding edge 6011 and screen B located to the right of the folding edge 6011. The folding method of this foldable screen phone is inward folding, that is, folding inward along the folding edge 6011 (the folding method can be understood as closing an open book).

[0147] In the unfolded rear view 602, the middle component is the folding edge 6021, connecting the left and right sides of the device. The left side of the device, including the rear camera, has the B screen behind it, as shown in the unfolded front view 601. The right side of the folding edge 6021 is a non-foldable display screen (i.e., the C screen), with the A screen behind it, as shown in the unfolded front view 601.

[0148] If the foldable phone starts from the unfolded state shown in 601, and screen A is fully folded along the folding edge 6011, the foldable phone becomes the folded state shown in 603. During the folding process, screen A gets closer and closer to screen B until screen A touches screen B, at which point screen A and screen B are opposite each other and invisible to the user. 603 is a front view of the foldable phone in the folded state. At this time, screen C, which is opposite to screen A, faces the user and is visible to the user.

[0149] If the foldable phone is in the unfolded state shown in 602, and the C screen is completely folded along the folding edge 6021, the foldable phone becomes the folded state shown in 604.

[0150] The image shown in sample 605 is a sample image of a folding phone at a certain folding angle during the folding process from the unfolded state shown in 601 to the folded state shown in 603. The image shown in sample 606 is a sample image of a folding phone at a certain folding angle during the folding process from the unfolded state shown in 602 to the folded state shown in 604.

[0151] It is understandable that the foldable phone is also in a folded state in the images shown in Figures 605 and 606. It should be understood that the folded state of a foldable phone includes all usage states from the unfolded state shown in Figure 601 to the folded state shown in Figure 603; or, all usage states from the unfolded state shown in Figure 602 to the folded state shown in Figure 604. Different folding angles correspond to different usage states of the foldable phone.

[0152] It is understandable that the above Figure 6 This is just one example. Product images of mobile phones can include product appearance images from different angles and with different physical forms; there are no limitations here.

[0153] like Figure 7The image shown is a sample product image of a mobile phone that includes a notch screen. Figure 7 As shown, the product sample includes a front view 701 of the phone in portrait mode, a rear view 702 of the phone in portrait mode, and a front view 703 of the phone in landscape mode. In the front view 701 of the phone in portrait mode and the front view 703 of the phone in landscape mode, the phone display also includes a notch area 704.

[0154] It is understandable that the product images of the source devices are only for visual purposes, so that users can intuitively understand the appearance of the source devices.

[0155] Among them, the appearance parameters of the source device include the specific data of the appearance components of the source device.

[0156] For example, the appearance parameters of the source device include one or more of the following: the shell parameters of the source device, the bezel parameters of the source device, the hardware button parameters of the source device, the interface parameters of the source device, the camera parameters of the source device, the audio output parameters of the source device, the jack parameters of the source device, and the notch screen parameters of the source device.

[0157] The parameters for the source device's casing can include data such as the casing's material, color, size, and weight. The parameters for the source device's bezel can include data such as the bezel's material, color, size, and weight. The parameters for the source device's hardware buttons can include data such as the location, shape, and size of the hardware buttons (e.g., volume buttons for volume control, power buttons for power on / off). The parameters for the source device's interfaces can include data such as the location, shape, and size of the interfaces (e.g., charging port, headphone jack). The parameters for the source device's cameras can include data such as the location, shape, size, color, and material of the cameras (e.g., front and rear cameras). The parameters for the source device's audio output can include data such as the location, shape, and size of the audio output. The parameters for the source device's ports can include data such as the location, shape, and size of the ports (e.g., SD card slot). The parameters for the source device's notch screen can include data such as the location, shape, and size of the notch screen.

[0158] It is understood that the appearance parameters of the aforementioned source device are those of a non-foldable screen device. If the source device is a foldable screen device, then the appearance parameters of the source device also include folding axis parameters, which may include data information such as the material, color, and size of the folding axis.

[0159] It is understandable that the specifications of the source device focus on describing the overall attributes of the source device, the appearance parameters of the source device focus on describing the specific data information of the appearance components of the source device, and the product sample image of the source device is a product image of the source device.

[0160] It is understood that the specifications, product images, and appearance parameters of the aforementioned source device are inherent information of the source device, belonging to system data, and will not change with the use of the source device. The specifications, product images, and appearance parameters of the source device can be pre-configured in the source device, and the source device retrieves this information from its own memory when performing device collaboration. Alternatively, the specifications, product images, and appearance parameters of the source device may not be stored in the source device itself; this information can be obtained in real-time through interaction between the source device and the server or cloud during device collaboration.

[0161] Understandably, if a user configures a new appearance component for a source device, the information of that appearance component can also be modified accordingly. This modification can be automatic or manual. For example, when configuring a phone case, the user can modify the phone's body information (e.g., adding a phone case model), the shell parameters in the appearance parameters, and the product image (e.g., adding a new phone case image) from the pre-stored appearance component information on the phone. Alternatively, the user can upload the modified appearance component information on the device side, and the cloud or server can update it based on the modified information. This ensures that the appearance elements corresponding to the changed appearance component can be displayed during collaborative display, guaranteeing the authenticity of the device. As another example, if the phone case is an active device (e.g., a ring flash phone case, a liquid-cooled phone case, etc.), the phone can automatically recognize the model of the configured phone case after configuring such a case, and thus the information of the phone's appearance components can be automatically modified.

[0162] It should be understood that the information about the appearance components in the first piece of information in the above example is obtained from the source device. However, the specifications, product drawings, and appearance parameters of the source device are inherent attributes of the source device, and this information can also be obtained from a server or the cloud.

[0163] In order to reduce the amount of information contained in the first information, in some other embodiments, the information of the appearance components of the source device in the first information may include the device information of the source device.

[0164] It is understandable that the source device transmits its device information as information about the appearance components of the source device to the target device. The target device can then obtain the specifications, product images, and hardware parameters of the source device from the server or cloud based on the source device's device information, thereby obtaining the complete parameter information of the appearance components of the source device.

[0165] In other embodiments, the information regarding the appearance components of the source device may also include the usage status of the source device. The usage status of the source device may include its physical form in space. For example, the usage status may include portrait mode, landscape mode, etc.

[0166] In some examples, the source device can determine its usage status based on data detected by sensors.

[0167] For example, based on the above Figure 5 For example, a mobile phone includes a gyroscope sensor that can detect changes in the phone's rotation and orientation (which can also be described as the phone's motion posture). The phone determines its usage status based on the data detected by the gyroscope sensor. For example, whether the phone is currently in landscape or portrait mode. Alternatively, the phone also includes an accelerometer sensor that can detect the phone's acceleration and motion status. The phone combines the data from the accelerometer sensor with algorithms to determine its usage status.

[0168] It is understandable that the above description of the usage states of the source device is based on the example of a non-foldable screen device. If the source device is a foldable screen device, its usage states can also include unfolded state, folded state, etc.

[0169] Optionally, the usage status of the source device may also include the device's posture and folding angle.

[0170] For example, such as Figure 6 As shown, the device postures of a foldable screen device in its unfolded state include, for example: Figure 6 As shown in Figure 601, in the unfolded state 1, the folding angle (i.e., the angle between screens A and B folded along the folding edge) is 180°. The large screen composed of screens A and B faces upwards or towards the user, while screen C and the rear camera face downwards or away from the user. Other device postures for the foldable screen device in the unfolded state include... Figure 6 As shown in Figure 602, in the unfolded state 2, the folding angle is 180°. The large screen, consisting of screens A and B, faces downwards or away from the user, while screen C and the rear camera face upwards or towards the user. Other device postures of the foldable screen device in the folded state include... Figure 6 As shown in Figure 603, in folded state 1, the folding angle is 0°. The foldable screen device is folded with the C-screen facing upwards or towards the user, and the rear camera facing downwards or away from the user. Other device postures for the foldable screen device in folded state include... Figure 6 As shown in Figure 604, in folded state 2, the folding angle is 0°. The foldable screen device is folded with the C-screen facing down or away from the user, and the rear camera facing up or towards the user. Other device postures for the foldable screen device in folded state include... Figure 6The folded state 3 shown in Figure 605, under this device posture, has a folding angle of 3. A foldable device is folded with the C-screen facing upwards or towards the user, and the rear camera facing downwards or away from the user. Other postures of a foldable device in its folded state include... Figure 6 The folded state 4 shown in Figure 606, under this device posture, has a folding angle of 4. Foldable screen devices are folded with the C-screen facing down or away from the user, and the rear camera facing up or towards the user.

[0171] In the above examples, "facing the user" includes: the screen (the entire flexible screen (screens A and B), or screen C) facing the user at a substantially parallel angle to the user's face; and the screen (the entire flexible screen (screens A and B), or screen C) facing the user at a certain tilt angle. "Facing upwards" in the above examples includes: the screen (the entire flexible screen (screens A and B), or screen C) facing upwards at a horizontal angle; and the screen (the entire flexible screen (screens A and B), or screen C) facing upwards at a certain angle while remaining on a horizontal plane.

[0172] For example, based on the above Figure 6 For example, a foldable phone includes a folding angle sensor. Based on data detected by this sensor, the foldable phone determines its folding angle and thus its usage state. For instance, the usage state of the foldable phone is... Figure 6 As shown in Figure 601. Alternatively, the foldable phone includes a gyroscope sensor, an accelerometer sensor, and a magnetometer sensor, and the foldable phone determines its usage status based on the data detected by the above sensors and algorithms.

[0173] Understandably, in this example, the usage status of the source device is the real-time usage status detected by the sensors within the source device. The source device can send its own usage status to the target device so that the target device can recreate the usage status of the source device in real time. For example, after the source and target devices collaborate, if the source device changes from state 601 to state 603, the target device's display interface will synchronously show the change process of the source device.

[0174] In other examples, the source device can determine its usage status based on a first preset rule and sensor detection data.

[0175] Specifically, the source device acquires the detection data from the sensor and determines the corresponding usage status based on a first preset rule. The first preset rule is a pre-defined rule for classifying the usage status of the source device.

[0176] For example, based on the above Figure 6For example, the first preset rule can be that if the folding angle is within the range of [x1°, 180°] and the screen of the source device faces upward, it is determined that the usage state of the source device is the unfolded upward state shown in 601, and the angle of this usage state is 180°. Here, x1 is a positive number, and 0 < x1 < 180. If the folding angle is within the range of [0°, x1°) and the screen of the source device faces upward, it is determined that the usage state of the source device is the C-screen upward state shown in 603, and the angle of this usage state is 0°.

[0177] It can be understood that in this example, the usage states of the source device are classified into several different categories, and the target device displays a certain category of usage state. For example, if the source device changes from the state shown in 601 to the state shown in �03, the display interface of the target device first displays the unfolded upward state shown in 601, and when it is detected that the folding angle of the source device is less than x1°, the display interface of the target device displays the C-screen upward state shown in 603. That is, in this example, the folding process of the source device is not displayed, only the folding change result is displayed.

[0178] It can be understood that the embodiments of the present application do not limit the specific content and specific implementation manners of the information of the appearance components of the source device.

[0179] In the embodiments of the present application, the information of the first interface may include the control information and layout information of the first interface in the source device when the source device is triggered to start the device collaboration function. The control information may include text controls, buttons, picture controls, progress bars, etc. The layout information may include interface layout information, control layout information, etc.

[0180] Here, the first interface is the interface currently displayed on the source device (i.e., the display interface of the application program running in the foreground) when the source device is triggered to start the device collaboration function. The application program may be a desktop application or any application on the source device, such as a video application.

[0181] Exemplarily, based on the example of S400 above, the first interface is the display interface of the mobile phone desktop application. The control information of the first interface includes multiple application program icons in the mobile phone desktop, etc., and the layout information of the first interface includes the positions and arrangements of multiple application program icons, etc.

[0182] S402. The source device sends the first information to the target device.

[0183] In some examples, the source device sends the first information to the target device based on the established communication connection.

[0184] S403. The target device displays a first window based on the first information. The first window includes appearance elements and a projection interface. The display effect presented by the appearance elements is the same as the visual effect presented by the appearance components of the source device. The content of the projection interface is a mirror image of the content of the first interface.

[0185] The projection interface is a mirror image of the first interface, and the content displayed in the projection interface is the same as that displayed in the first interface. Appearance elements can be virtual representations of appearance components on the source device. For example, an appearance element is an image corresponding to an appearance component on the source device. Users can trigger the corresponding appearance component on the source device to perform the corresponding function through this appearance element.

[0186] It is understandable that the appearance components are the actual physical hardware of the source device. During collaborative display, the target device can only simulate and mimic the display of the appearance components of the source device. The target device generates corresponding images or virtual buttons for peripheral components, i.e., appearance elements, based on the information of the appearance components of the source device.

[0187] An appearance component can be understood as a physical concept, corresponding to a specific hardware entity. An appearance component can be a visible part of a physical product or device. Taking a mobile phone as an example, appearance components can include the casing, frame, hardware buttons (such as volume buttons and power buttons), interfaces, cameras (such as front-facing cameras and rear-facing cameras), speaker jacks, ports, and notch displays, etc.

[0188] Visual elements can be understood as a kind of virtualized display content that relies on the display of electronic devices to be presented to the user. Visual elements can be a collection of visual elements such as graphics and icons corresponding to visual components. These visual elements are rendered and presented on the display screen and displayed to the user in a visible form.

[0189] Display effect can be understood as the result of presenting visual elements to the user through the display screen of an electronic device. The display effect depends on factors such as the performance, resolution, and color reproduction capabilities of the display screen. Display effect can be visual characteristics such as the display form of visual elements, the clarity of visual elements, color vividness, and contrast.

[0190] Visual effects can be understood as the visual effect or impression produced by the appearance of a physical device or product. Visual effects depend on factors such as the product's design, materials, and contours. Visual effects can include the reflective effects of exterior components under different lighting conditions, and the visual perception of the materials used in these components.

[0191] For example, taking the volume buttons as an example, these are physical buttons, black in color, oval in shape, 10mm in length, and protrude 1mm from the phone's edge. These physical, visually perceptible aspects can be understood as the visual effect of the volume buttons. Similarly, when the phone and PC are in a collaborative state, with the phone screen projected onto the PC, the volume button element displayed on the PC screen is black in color, oval in shape, 10mm in height, and protrudes 1mm from the phone's edge. The color and shape of the volume button element displayed on the PC screen can be understood as the display effect.

[0192] The same visual effect presented by the appearance elements as that presented by the appearance components of the source device can be understood as the appearance elements displayed on the target device and the actual appearance components in the source device having similar or identical appearance features. Although the appearance component is a physical hardware and the appearance element is a virtual display content, the visual perception or impression given to the user is the same, maintaining consistency in the user's senses.

[0193] In some examples, if the information of the appearance components of the source device in the first information includes the specifications of the source device, the product image of the source device, and the appearance parameters of the source device, then the display state of the appearance elements and the projection interface in the first window is the preset state.

[0194] Specifically, the target device displays a first window based on the first information and a preset state. The target device determines its corresponding preset state based on the information of the first interface, and draws and generates the appearance elements corresponding to the appearance components in the preset state based on the preset state and the appearance component information of the source device. The target device also draws and generates the projection interface corresponding to the first interface based on the information of the first interface. The target device displays the first window based on the appearance elements and the projection interface using an overlay algorithm.

[0195] In one approach, a preset state corresponding to each device type is pre-configured based on the device type. For example, for non-foldable screen phones, the corresponding preset state is as follows: Figure 5 As shown in Figure 501, this is the portrait mode. For foldable phones, the corresponding preset modes are as follows: Figure 6 As shown in Figure 601, the large screen is in its unfolded state.

[0196] In another approach, preset states are set for each usage scenario. For example, taking a non-foldable screen device as an example, if the device is playing a video and the video is maximized during device collaboration, and the first interface in this usage scenario is also in landscape orientation, then the preset state for this usage scenario is set to landscape mode. If the device is playing music during device collaboration, then the preset state for this usage scenario is portrait mode.

[0197] Understandably, displaying the appearance elements and projection interface in the first window according to the preset state can speed up the display process and improve efficiency.

[0198] In other examples, if the information of the appearance components of the source device in the first information includes the specifications of the source device, the product drawing of the source device, the appearance parameters of the source device, and the usage status of the source device, then the display status of the appearance elements in the first window and the projection interface corresponds to the usage status of the source device.

[0199] Specifically, the target device draws and generates appearance elements corresponding to the usage status of the source device based on the appearance component information of the source device, and the target device draws and generates the projection interface corresponding to the first interface based on the information of the first interface; the target device displays the first window based on the overlay algorithm, according to the appearance elements and projection interface corresponding to the usage status of the source device.

[0200] Understandably, in this example, the display status of the appearance elements and the projection interface in the first window of the target device corresponds to the usage status of the source device, and the display status of the appearance elements and the projection interface in the first window can change according to the usage status of the source device.

[0201] For example, if the source device is a foldable screen device that connects to a PC, and the user folds the foldable screen device from an unfolded state to a folded state, the visual elements in the first window of the target device will also change from an unfolded state to a folded state, and the projection interface will be updated accordingly.

[0202] It should be understood that the first window of the target device displays not only the appearance elements corresponding to the appearance components of the source device, but also the projection interface corresponding to the first interface in the source device.

[0203] In some examples, the target device generates appearance elements corresponding to the appearance components of the source device based on the size of the first window and the appearance component information in the first information. The target device generates the projection interface corresponding to the first interface based on the size of the first window and the information of the first interface. The target device displays the first window based on the appearance elements and the projection interface using an overlay algorithm.

[0204] The size of the first window can be determined based on the first information (for example, the size of the first window can be determined based on the body size of the source device in the first information. In this case, the aspect ratio of the first window is the same as that of the source device). Alternatively, the size of the first window can be a preset size (the preset size can be the display size of the target device for collaborative display. In this case, the aspect ratio of the first window may be the same as or different from that of the source device).

[0205] It should be understood that the source device's body size and screen size remain unchanged after it leaves the factory; replacing the phone case simply involves replacing it with a case of the same size. In other words, the device's external hardware can be altered, but its dimensions remain the same. Therefore, during collaborative display, the target device can adaptively adjust the size of the first window based on the information of the source device's external components. Alternatively, the target device can adaptively modify the sizes of its external elements and the projection interface based on the preset size of the collaborative display window (i.e., the first window). That is, the sizes of the external elements and the projection interface are adjusted according to the size of the first window.

[0206] Understandably, to maintain a better collaborative display effect, the aspect ratio of the appearance elements and the projection interface is usually the same as that of the source device. For example, if the size of the first window is a preset size, and the aspect ratio of the first window is different from that of the source device, then the size of the first window is larger than the size of the appearance elements and the projection interface. The aspect ratio of the appearance elements and the projection interface is the same as that of the source device, and the length and width of the appearance elements and the projection interface are determined according to the preset size of the first window and the aspect ratio of the source device. The display area in the first window other than the appearance elements and the projection interface is filled with a black image.

[0207] In actual use, the size of the first window can also dynamically change according to user interaction (such as zooming in or out). Correspondingly, the sizes of the appearance elements and the projection interface will change along with the changes in the first window. That is, the target device can adjust the size of the first window and the sizes of the appearance elements and the projection interface within the first window in response to user operations.

[0208] For example, when a mobile phone screen is projected onto a PC, the PC displays the visual elements and the projected interface in a first window. The user can move the mouse to the border or corner of the first window, and the mouse cursor will change to a double-headed arrow. The user can click the left mouse button and drag the border or corner to adjust the size of the first window. Alternatively, the user can adjust the window size using keyboard shortcuts. The PC responds to the user's actions by adjusting the size of the first window, adaptively modifying the sizes of the visual elements and the projected interface based on the adjusted window size, and updating the first window accordingly.

[0209] In some examples, if the user enlarges the size of the first window to match the screen size of the target device (i.e., the first window maximizes to fill the entire screen of the target device), the first window may only include the projection interface. In this first window, the projection interface is displayed in maximum size according to the aspect ratio of the source device, without displaying any external elements of the source device.

[0210] Optionally, if the content displayed in the first window of the projection interface is video, and the user enlarges the size of the first window to match the screen size of the target device, then the target device can fill the first window with the content displayed in the projection interface, and no visual elements will be displayed in the first window. This is to improve the user's viewing experience.

[0211] In some examples, when the target device displays the first window, the window elements of the first window may not be displayed. Visually, only the appearance elements and the projection interface are displayed on the target device's screen.

[0212] Window elements include one or more of the following: window border, title bar, navigation bar, etc.

[0213] The window border refers to the boundary surrounding the content displayed in the first window (i.e., appearance elements and projection interface), used to separate the first window from other content displayed on the target device's screen. The window border can also indicate the position and size of the first window on the target device's screen; users can adjust the position and size of the first window by dragging the window border.

[0214] The title bar is used for window management and is typically located at the top of the window. It provides various functional controls for users to manage the window. The title bar may also include descriptions of the window's functions. For example, it may include one or more controls such as close, maximize, minimize, and keep on top. The close control closes the window. The maximize control enlarges the window to its maximum size on the screen. The minimize control minimizes the window to a location such as the taskbar for easy reopening. The keep-on-top control keeps the window always on top of other windows, ensuring it remains in the foreground.

[0215] The navigation bar provides users with quick interface navigation and operation control, and is typically located at the bottom of the window. It offers various functional controls for rapid user navigation. For example, the navigation bar may include one or more of the following: a back control, a home screen control, and a multitasking control. The back control is usually located on the far left of the navigation bar and is used to return to the previous screen or operation step. Users can use this back control to navigate within the application or system, navigating back level by level. The home screen control is usually located in the middle of the navigation bar and is used to return to the home screen. Regardless of the current screen, clicking the home screen control will return the user to the home screen. The multitasking control is usually located on the far right of the navigation bar and is used to display recently used applications. Users can use this multitasking control to quickly switch between or close applications.

[0216] It's understandable that while touchscreen-equipped electronic devices (like smartphones) allow for precise operations via touch or gestures, when projected onto a non-touchscreen device (like a PC), touch or gesture controls of visual elements and the projected interface are impossible. Therefore, non-touchscreen devices can utilize title bars, navigation bars, etc., to replace touchscreen operations. Furthermore, navigation bars and title bars help users understand the interface context and operation sequence, facilitating smooth use of collaborative features, ensuring a better user experience, improving the efficiency and comfort of multi-device collaboration, and enhancing the overall user experience.

[0217] It is understandable that some electronic devices will display a navigation bar. If the first interface displayed by the source device includes a navigation bar, then the corresponding navigation bar will be displayed in the screen mirroring interface.

[0218] For example, based on the examples of S400-S401 above and Figure 6 In the example of Section 601, the target device displays the corresponding appearance elements and projection interface of the source device in the first window at the same scale (i.e., the size of the appearance elements and projection interface is the same as the size of the source device). Figure 8 As shown in (a), this is the display interface of the foldable phone during communication. Specifically, the foldable phone is in its unfolded state with its front facing the user. The first interface currently displayed by the foldable phone is the desktop 801. The external components of the foldable phone include the frame 802, volume buttons 803, power button 804, and front-facing camera 805. When the foldable phone communicates with the PC, it sends information about its external components (such as the phone's specifications, product images, appearance parameters, and usage status) and the information from the desktop 801 to the PC.

[0219] like Figure 8 As shown in (b), the PC generates the appearance element 806 and the projection interface 811. Specifically, the PC generates the appearance element 806 corresponding to the appearance components of the foldable phone based on the information of the foldable phone's appearance components. This appearance element 806 includes a border element 807, volume button element 808, power button element 809, and front-facing camera element 810. The PC generates the projection interface 811 based on the information of the desktop 801 on the foldable phone.

[0220] like Figure 8As shown in (c), this is the display interface of the first window on the PC. Specifically, the PC overlays the appearance element 806 and the projection interface 811 in the first window 812. The window border 813 of the first window 812 is represented by a dashed line, and this window border 813 is not displayed on the PC. Overlay display can be understood as nesting the projection interface 811 within the appearance element 806. Thus, the first window 812 displayed on the PC contains both the appearance element 806 corresponding to the source device and the projection interface 811 corresponding to the first interface. Visually, the foldable screen phone is displayed realistically and completely on the PC's screen.

[0221] In some examples, when the target device displays a first window but not its window elements, the target device displays the first window element in response to a user's operation on the hotspot corresponding to the first window element. The first window element can be any one of the window elements in the first window. The hotspot corresponding to the first window element is the display area that triggers the target device to display the first window element; the hotspot corresponding to the first window element can also be described as being within a preset range of the first window element.

[0222] It is understandable that each window element in the first window can have a corresponding hotspot, so that users can manipulate the window elements to perform corresponding functions through the hotspot.

[0223] For example, such as Figure 9 As shown, the PC displays the first window 900 (its border is represented by a dashed line), but does not display any of its window elements. The first window 900 displays the projection interface corresponding to the foldable phone's desktop and the appearance elements corresponding to the foldable phone's appearance components. The PC sets a title bar hotspot 901 at the top of the appearance elements and projection interface. This title bar hotspot 901 covers both the appearance elements on the top border of the foldable phone and a portion of the top display area of ​​the projection interface, as well as a portion of the display area above the top border of the foldable phone that is not part of the projection interface. When the user moves the mouse cursor 902 from its current position upwards to the title bar hotspot 901, the PC displays the title bar 903 for user interaction. The title bar 903 is elliptical in shape and includes multiple controls, from left to right: a pin to top control, a minimize control, a maximize control, and a close control. The pin to top control keeps the first window 900 on top, preventing it from being obscured by other windows on the PC. The minimize control minimizes the first window 900 to a location such as the PC's taskbar, so the first window 900 is not displayed on the PC. The maximize control enlarges the first window 900 to the maximum size of the PC screen, i.e., maximizes the display of the first window 900. The close control closes the first window 900. Correspondingly, if the user continues to move the cursor 902 upwards until the cursor 902 is no longer located in the title bar hotspot 901, the PC will no longer display the title bar 903.

[0224] like Figure 9 As shown, the PC has a navigation bar hotspot 904 at the bottom of the appearance elements and the projection interface. This hotspot 904 covers both the appearance elements of the foldable phone's bottom bezel and a portion of the projection interface's bottom display area, as well as a portion of the non-projection interface display area below the bottom bezel. When the user moves the cursor 902 down from its current position to the navigation bar hotspot 904, the PC displays the navigation bar 905 for user operation. The navigation bar 905 is oval-shaped and includes multiple controls, from left to right: a back control, a home screen control, and a multitasking control. The back control allows the projection interface to return to the previous screen. The home screen control allows the projection interface to jump from the current screen to the home screen. The multitasking control allows the user to control the projection interface to display the multitasking interface. Conversely, if the user continues to move the cursor 902 down, causing it to move out of the navigation bar hotspot 904, the PC will no longer display the navigation bar 905.

[0225] It is understood that in the above example, the hot zones corresponding to the first window elements are all located within the first window. These hot zones encompass both a portion of the display area of ​​the projection interface and visual elements, as well as a portion of the display area of ​​the non-projection interface and visual elements within the first window. In practical applications, the hot zones corresponding to the first window elements may only cover a portion of the display area of ​​the projection interface and visual elements. This application does not limit the method of dividing the hot zones corresponding to the first window elements in the target device.

[0226] It is understood that in the above example, the first window element is displayed in an elliptical shape near the hot zone. In practical applications, the first window element can also be displayed in a rectangular or other style, and its display position can also be set in other locations. This application does not limit the display style and position of the first window.

[0227] For example, such as Figure 9 As shown, if cursor 902 is outside the first window 900, and the user moves the cursor from outside the first window 900 into the hotspot corresponding to the first window element, the first window element may or may not be displayed in the first window 900. For example, cursor 902 is located above the first window 900, as shown in position 910. When the user moves the mouse, moving cursor 902 down from 910 to the title bar hotspot 910, the title bar 903 is displayed on the PC. Alternatively, when the user moves the mouse, moving cursor 902 down from 910 to the title bar hotspot 910, the title bar 903 is not displayed on the PC.

[0228] It is understandable that the above example uses a mobile phone screen mirroring onto a PC, with the user operating the mouse and the cursor moving within the PC's display interface. In practical applications, the target device can be an electronic device including a touchscreen. Users can operate the target device's touchscreen display interface using their fingers, styluses, capacitive styluses, etc. For example, if a user touches the title bar of the first window in the display interface, the target device will respond by displaying the title bar.

[0229] In this embodiment, the target device may be pre-configured with operations that trigger the display of a first window element. When the target device displays the first window, it receives user operations on the first window. If it determines that the operation is a pre-configured operation that triggers the display of the first window element, it displays the first window element in response to the operation. This application does not limit the specific implementation of the target device displaying the first window element.

[0230] In this application, when the target device displays the first window, it does not display the window elements of the first window. The target device can fully utilize the size of the first window, maximizing the display of visual elements and the projection interface within the window, providing a richer and clearer visual experience and making the window more concise. Furthermore, not displaying window elements reduces visual interference, allowing users to focus more on the projection interface and visual elements. The target device can dynamically display or hide the window elements of the first window based on user operations. When the user needs to view or operate the window elements, the window elements are displayed in response to the user's operation; when the user does not need to operate the window elements, they are not displayed, providing a more flexible user experience.

[0231] In this embodiment, the target device may also set corresponding hot zones for functional elements in the first window. In response to user operation on the hot zone corresponding to a functional element, the target device displays the functional area of ​​that element. The user can manipulate this functional area, and the target device executes the corresponding function in response to the user's operation.

[0232] In some examples, the target device responds to a user's trigger operation within a preset range of the functional element by displaying the target element corresponding to the functional element.

[0233] The appearance components include functional components, and the appearance elements include functional elements. The display effect of the functional elements is the same as the visual effect presented by the functional components of the source device. That is, the functional elements are the elements in the appearance elements that can trigger the source device to perform corresponding functions. For example, volume buttons, power buttons, cameras, etc. The functional area of ​​a functional element includes the target element corresponding to the functional element.

[0234] The hot zone corresponding to the functional element is the display area of ​​the target element that triggers the target device to display the corresponding functional element. The hot zone corresponding to the functional element can also be described as being within the preset range of the functional element.

[0235] For example, based on the above Figure 9 For example, the appearance elements in the first window 900 include border elements, volume key elements, power button elements, and front-facing camera elements. For instance, the PC sets a preset range of display area around the volume key elements as a volume key hotspot 906. This hotspot 906 covers both a portion of the display area near the volume key elements in the projection interface and a portion of the display area to the right of the volume key elements in the non-projection interface. When the user moves the cursor 902 from its current position to the volume key hotspot 906, the PC displays the function area 907 of the volume key elements (i.e., the target element corresponding to the volume key elements). Subsequently, the user can manipulate this function area 907 to change the volume of the foldable phone. Similarly, the PC sets a preset range of display area around the power button elements as a power button hotspot 908. This hotspot 908 covers both a portion of the display area near the power button elements in the projection interface and a portion of the display area to the right of the power button elements in the non-projection interface. When the user moves the cursor 902 from its current position to the power button hotspot 908, the PC displays the function area 909 of the power button element (i.e., the target element corresponding to the power button). The user can then manipulate this function area 909 to turn off the power. Similarly, the PC can set a preset display area around the front-facing camera element as the front-facing camera hotspot. Figure 9 (Not shown in the image), when the user moves the cursor 902 from its current position to the front-facing camera hotspot, the PC displays the function area for the front-facing camera element (…). Figure 9 (Not shown in the image), users can then manipulate this function area to perform functions such as taking photos.

[0236] In some scenarios, a mobile phone and a PC are placed around the same object (e.g., the phone is in front of the object, and the PC is behind it). The phone establishes a communication connection with the PC, sending information about its appearance components and the first interface to the PC. The PC then displays the first window on its screen based on this information. The user can manipulate the camera element in the first window on the PC to activate the phone's camera function and take a picture of the object from the front. The PC can also activate its camera function to take a picture of the object from behind. In this way, the user can control both the phone and the PC to take pictures of the same object from different angles, enabling multi-angle photography.

[0237] In this embodiment, the target device can be pre-configured with operations that can trigger the display of functional elements. When the target device displays the first window, it receives user operations on the first window. If it determines that the operation is a pre-configured operation that can trigger the display of functional elements, it displays the functional elements in response to the operation. This application does not limit the specific implementation method or display style of the functional elements.

[0238] It should be understood that, as described in the first window element above, this application does not limit the display style and display position of the functional elements, nor does it limit the division method of the hot zones corresponding to the functional elements.

[0239] In this application, the target device sets hot zones and functional areas for the corresponding appearance elements (i.e., functional elements) of the functional components within the appearance components of the source device. Users can manipulate the corresponding functional components in the source device through the hot zones and functional areas of the functional elements in the target device to achieve the desired functions. This direct control method enhances the interactivity and real-time responsiveness between the user and the device. It also improves the user's control experience over the appearance components of the source device, expands the functionality and applicability of device collaboration, and enriches the user experience.

[0240] Furthermore, the target device can dynamically display or hide the function areas corresponding to functional elements based on user operations. When the user needs to view or operate a functional element, the corresponding function area is displayed in response to the user's operation; when the user does not need to operate the functional element, the corresponding function area is not displayed, providing a more flexible user experience. Users can clearly see the functional controls of the element they are operating through the function area, improving the visibility, controllability, and convenience of the operation. This enhances the intelligence and completeness of device collaboration.

[0241] It is understandable that the display area of ​​the aforementioned window elements and the functional area corresponding to the appearance elements can be described as a functional window. When the hotspot corresponding to the functional window is not triggered (e.g., the cursor is not in the hotspot), the functional window is not displayed. When the hotspot corresponding to the functional window is triggered (e.g., the cursor is in the hotspot), the corresponding functional window is displayed.

[0242] Optionally, embodiments of this application can also dynamically change the display effect of the function window based on the distance from the operation point to the hot zone. For example, when the user moves the cursor 902 from its current position closer to the title bar hot zone 901, the closer the cursor 902 is to the title bar hot zone 901, the higher the clarity of the title bar 903. When the cursor 902 is located in the title bar hot zone 901, the title bar 903 is always displayed with the highest clarity. When the user continues to move the cursor 902 away from the title bar hot zone 901, the farther the distance between the cursor 902 and the title bar hot zone 901, the lower the clarity of the title bar 903, until it is not displayed. Alternatively, the closer the cursor 902 is to the title bar hot zone 901, the higher the opacity of the title bar 903; the farther the distance between the cursor 902 and the title bar hot zone 901, the lower the opacity of the title bar 903.

[0243] In other examples, when the target device displays the first window, it also displays the window elements of the first window. That is, the first window displays both the window elements and the appearance elements and the projection interface.

[0244] For example, such as Figure 10 As shown, the window elements of the first window 1000 displayed on the PC include a window border 1001, a title bar 1002, and a navigation bar 1003. The first window 1000 also includes the appearance elements corresponding to the foldable phone and a projection interface 1004. The title bar 1002 includes a pin to top control 1005, a minimize control 1006, a maximize control 1007, and a close control 1008. The title bar 1002 also includes a textual description of "Device Collaboration," used to briefly introduce the window's functions; alternatively, the description could be a name indicating the foldable phone model, such as "My Mate X5." The navigation bar 1003 includes a back control 1009, a home screen control 1010, and a multitasking control 1011. In response to an operation on the return control 1009, the projection interface 1004 displays the previous level interface of the current interface; in response to an operation on the home screen control 1010, the projection interface 1004 displays the home screen; in response to an operation on the multitasking control 1011, the projection interface 1004 displays the multitasking interface. The appearance elements corresponding to the foldable phone and the projection interface 1004 include the appearance elements corresponding to the appearance components of the foldable phone and the projection interface corresponding to the desktop of the foldable phone.

[0245] Understandably, in this example, when the PC displays the first window, it shows the window elements of that first window. These window elements provide visual guidance, helping users quickly identify the function and purpose of the current first window, thus improving the efficiency of user operations. Furthermore, standardized window elements make the content displayed in the first window look more consistent and organized, enhancing the consistency of the user experience.

[0246] For example, such as Figure 10 As shown, the appearance elements displayed in the first window 1000 of the PC include border elements, volume key elements, power button elements, and front-facing camera elements. Taking the volume key elements and power button elements as examples, a volume key hotspot 1012 is set near the volume key element, and a power button hotspot 1014 is set near the power button element. Figure 10 The dashed box surrounding the volume buttons defines the display area as volume button hotspot 1012, and the dashed box surrounding the power button defines the display area as power button hotspot 1014. If the user moves the mouse cursor to volume button hotspot 1012, the PC displays the function area 1013 of the volume buttons in the projection interface 1004. If the user moves the mouse cursor to volume button hotspot 1014, the PC displays the function area 1015 of the power button in the projection interface 1004.

[0247] It is understandable that the above Figure 9 and Figure 10 The functional area corresponding to a functional element includes a target element that indicates the function of the functional element and guides the user. Therefore, the PC can respond to the user's operation on the hotspot corresponding to the functional element, draw and generate the target element corresponding to the functional element, and display the target element.

[0248] It is understandable that the above example uses the display of the target element corresponding to the functional element on the target device. In other methods, the target device may not display the target element corresponding to the functional element. The target device interacts with the source device based on the user's operation on the functional element, updating the projection interface in the first window.

[0249] In this embodiment, after device collaboration, the target device can further detect user operations on functional elements, generate a collaboration event in response to the user's operation (such as operation information, operation object, operation location, etc. The operation information may include information about clicks, long presses, etc.), and send the collaboration event to the source device. Then, the source device can determine the user's operation based on the collaboration event and send information from the second interface to the target device. The second interface is the interface updated by the source device in response to the collaboration event. The target device updates the projection interface in the first window based on the information from the second interface.

[0250] In some examples, taking the display of a target element on the target device as an example, the target device detects the user's operation on the hotspot of the functional element and displays the target element corresponding to the functional element in response to the operation; the target device detects the user's operation on the target element and generates a collaboration event in response to the operation. The target device sends the collaboration event to the source device, and the source device determines the information of the second interface based on the collaboration event and sends the information of the second interface to the target device.

[0251] Understandably, in this example, the user can see the target element on the target device, and the collaborative event is activated based on the user's operation on the target element, which can improve operation efficiency, improve operation accuracy, and avoid problems such as accidental touches.

[0252] In other examples, assuming the target device does not display the target element, the target device detects user interaction with the functional element and generates a collaboration event in response. The target device sends the collaboration event to the source device, which then determines the information of the second interface based on the collaboration event and sends the second interface information back to the target device.

[0253] Optionally, the target device can be pre-configured to directly manipulate functional elements to trigger the generation of corresponding collaborative events. This could be done by the user clicking the functional element or hovering the cursor over it. For example, if the target device detects a user click on a volume button element, determines the location of the operation point, and if the point is located at the volume up button's volume up element, it generates a volume up collaborative event in response. The target device sends this event to the source device, which then determines the volume up interface information based on the event and sends that information back to the target device.

[0254] Understandably, in this example, the target device does not display the target element, which saves space in the first window. The first window maximizes the display of the projection interface and visual elements, making the content more concise and clear, and providing a better visual experience. Furthermore, responding to user actions on functional elements to determine the user's intent allows for a more realistic portrayal of the user's experience with the source device's functional components, increasing the realism of the operation and enhancing the user experience.

[0255] In this embodiment, the device collaboration process includes mirroring collaboration and push collaboration. During mirroring collaboration, the content displayed on the projection interface of the target device is the same as the content displayed on the interface of the source device. During push collaboration, the content displayed on the projection interface of the target device is different from the content displayed on the interface of the source device.

[0256] Understandably, during mirror collaboration, after the source device determines the user's action based on the collaboration event, it displays a second interface in response to that user action (i.e., the source device executes the corresponding data processing and logic for the user action and displays the processing result on its local interface), and sends the information of the second interface to the target device. However, during push collaboration, after the source device determines the user's action based on the collaboration time, it executes the corresponding data processing and logic for that user action and sends the information of the second interface to the target device (i.e., the source device does not update its local interface).

[0257] For example, based on the above Figure 9 For example, if the user moves the cursor 902 to the volume key hotspot 906, the PC displays the volume key element's function area 907. The user then moves the cursor 902 to the volume up control in the function area 907 (i.e.,...). Figure 9 The user clicks the "+" control in function area 907 and then clicks the volume up control. The PC responds to this action by generating a Co-Event 1 (which may include the click action, the volume up control, etc.) and sends Co-Event 1 to the foldable phone. The foldable phone receives Co-Event 1 and determines that the user's action is to increase the volume.

[0258] If the device collaboration process is mirror collaboration, the foldable phone, based on collaboration event 1, updates the currently displayed first interface (i.e., the desktop) to the second interface (i.e., the interface displayed when the foldable phone increases volume, which includes a volume bar showing the volume indicator after the foldable device increases volume in response to collaboration event 1). The foldable phone sends the information of the second interface to the PC. The PC displays the corresponding mirrored interface based on the information of the second interface, such as... Figure 11 As shown, Figure 11 The screen mirroring interface displays a volume bar. If the user continues to click the volume increase control, the volume indicator in the volume bar will... Figure 11 The shaded area in the volume bar will increase. For example, if a video is playing in the mirrored interface, and the user adjusts the video volume by clicking the volume control on the PC, such as from 30% to 60%, the phone's volume will also adjust to 60%. Even if the foldable phone and PC are no longer in device collaboration mode, the foldable phone's volume will remain at 60%.

[0259] Understandably, during mirroring collaboration, the display interface of the source device and the projection interface of the target device remain consistent. Regardless of whether the user inputs an action on the source device or the target device, the source device responds to the action, updating its display interface, and the projection interface of the target device updates synchronously. In other words, the display and operation effects of the source device's display interface and the projection interface of the target device are synchronized, improving the smoothness of user operation and the consistency of the experience. Furthermore, it simplifies the operational interaction between the source and target devices, enhancing the user experience and allowing users to use multiple devices more efficiently for work or daily life.

[0260] If the device collaboration process is push collaboration, the foldable phone, based on collaboration event 1, executes the volume increase processing flow and logic indicated by collaboration event 1, and determines the information of the second interface (that is, during push collaboration, the foldable phone will respond to the collaboration event sent by the PC, but will not affect the current display of the foldable phone (it will not update the currently displayed first interface) and operation. In other words, when the foldable phone is processing the foldable phone and the PC, the foldable phone treats the foldable phone and the PC as independent systems. For example, the foldable phone can be seen as a server, capable of processing information from two devices, namely the foldable phone and the PC. The foldable phone has independent processing flow and logic for each device, without interference). The foldable phone sends the information of the second interface to the PC. The PC displays the corresponding projection interface based on the information of the second interface, such as... Figure 11 As shown, Figure 11 The screen mirroring interface displays a volume bar. If the user continues to click the volume increase control, the volume indicator in the volume bar will... Figure 11 The shaded area in the volume bar will increase. For example, if a video is playing in the mirroring interface, and the user adjusts the video volume by clicking the volume control on the PC, such as from 30% to 60%, the phone's volume will not adjust and will remain at 30% as before device collaboration. If the user does not adjust the foldable phone's volume during device collaboration, the foldable phone's volume will remain at 30% even after the foldable phone and PC are no longer in device collaboration mode.

[0261] Understandably, during push notification collaboration, the source device asynchronously processes tasks from both the source and target devices without interference. The source device can respond to collaboration events sent by the target device and handle its operations, such as a volume increase event, without interfering with its own current display and operation. This decoupled control enhances user management of devices and content, increases flexibility and personalization, and significantly improves device collaboration efficiency and user experience. Furthermore, the source device maintains its own display stability and remains unaffected by collaboration events from the target device. This stability ensures the continuity and predictability of user experience with the source device.

[0262] In this embodiment of the application, when the source device and the target device are in a device collaboration state, if a video or music is playing on the screen projection interface of the target device, the audio of the video or music is managed by the target device and the audio is output by the speaker of the target device.

[0263] As can be understood, the essence of device collaboration is that the source device acts as the image output device, the target device displays the image sent by the source device, but the audio is managed and output by the target device. This design helps to unify the audio output source, simplifies management, and improves the convenience and efficiency of operation. It also provides a consistent audio and control experience, avoiding the confusion or duplication that can result from multiple devices playing audio simultaneously.

[0264] For example, based on the above Figure 9 For example, if a user clicks the maximize control in title bar 903, the PC responds to the action by displaying, as shown below. Figure 12 The interface shown. (As shown) Figure 12 As shown in (a), the projection interface corresponding to the foldable screen device is displayed proportionally. The PC can set a title bar hotspot; when the user's operation point is detected within this hotspot, the title bar is displayed. For example, the title bar hotspot is set in the upper right corner of the screen; when the user's cursor moves to the upper right corner of the PC screen, the title bar is displayed. The title bar includes multiple controls, from left to right: minimize control, restore control, and close control. The minimize control minimizes the window to the PC's taskbar, etc. The restore control restores the window to its previous size before maximizing. The close control closes the window.

[0265] It is understandable that, such as Figure 12 As shown in (a), in order to ensure the authenticity of the source device display, even if the first window is enlarged to the maximum size of the display screen, the projection interface of the source device is displayed in the first window at the same scale, and the projection interface displayed in the first window does not increase in size as the window increases.

[0266] Or, such as Figure 12 As shown in (b), the projection interface is maximized based on the PC's maximum size and the aspect ratio of the foldable device, allowing users to clearly view the content. A title bar can also be displayed when the projection interface is maximized. For example, the title bar is displayed in the upper right corner of the screen. The title bar includes several controls, from left to right: minimize, restore, and close. The minimize control minimizes the window to the PC's taskbar, etc. The restore control restores the window to its original size before maximizing. The close control closes the window.

[0267] It is understandable that in the above example, when the first window is maximized, the title bar can be dynamically displayed based on user actions, or it can be directly displayed in the first window. In practical applications, the maximized display can also be entered or exited based on preset gestures, shortcut keys, etc. This application does not limit the specific implementation method of the maximized display.

[0268] It is understandable that the target device can also receive user input on the projection interface in the first window, and interact with the source device in response to the user's operation to update the projection interface in the first window. The specific implementation of the target device updating the projection interface in response to user operations is described in existing technology and will not be elaborated here.

[0269] Understandably, when a user interacts with the first window on the target device, they can only trigger the corresponding functions of the projection interface or visual elements within the first window. The target device responds to user actions and interacts with the source device, changing the displayed content of the projection interface in the first window based on information transmitted from the source device; it cannot change the visual elements within the first window. The visual elements displayed on the target device will only change when the usage state of the source device changes during the mirroring collaboration process.

[0270] Understandably, the above examples all assume the usage state of a foldable screen phone. Figure 6 The usage state shown in Figure 601 will be used as an example for explanation. In practical applications, during device collaboration, the hardware structure, usage state, and display content of the source device will change according to user usage. Correspondingly, the content displayed on the target device can change according to the information sent by the source device.

[0271] In some examples, if the hardware configuration of the source device changes during collaboration, such as when a user configures new appearance components for the source device, the source device updates the appearance component information. The target device can obtain the updated appearance component information from the source device or the server, and update the display effect of the appearance elements in the first window according to the updated appearance component information.

[0272] For example, during device collaboration, if a user changes their phone case from black to red, the phone automatically modifies the pre-stored appearance component information, changing the phone case color to red. The phone then sends the updated appearance component information to the PC. Based on this updated information, the PC changes the color of the phone case appearance element in the first window from black to red.

[0273] In other examples, if the usage state of the source device changes during collaboration, such as a phone changing from an unfolded state to a folded state, the source device updates the information of the appearance components. The target device can obtain the updated appearance component information from the source device and update the display effect of the appearance elements in the first window accordingly.

[0274] In one possible implementation, the source device determines its real-time usage status based on sensor detection data and sends this status to the target device. The target device then displays a first window based on the source device's real-time usage status. That is, the appearance elements of the first window and the display status of the projection interface on the target device remain consistent with the real-time usage status of the source device.

[0275] For example, the usage state when a foldable phone activates the collaboration function is... Figure 6 As shown in usage state 601, after the foldable phone establishes a communication connection with the PC, the user completely folds screen A along the folding edge 6011, and the foldable phone becomes the folded state shown in 603. During the folding process, the foldable phone synchronizes its own appearance component information and the current display interface information to the PC in real time. Based on the information transmitted by the foldable phone, the PC updates the first window synchronously, displaying the folding process of the foldable phone. That is, the first window displays animations / effects based on the change of the foldable phone from an unfolded state to a folded state.

[0276] Understandably, the target device displays the status changes of the source device in real time, so that users can quickly know the status changes of the source device. It can also enhance the interactivity between collaborative devices, improve the display effect of device collaboration, and enhance the fun and user experience of collaborative use.

[0277] Understandably, in this implementation, the hardware structure, usage status, and display content of the source device projected onto the target device can be identical to the actual usage of the source device. The target device simulates and recreates the display of the source device based on the information sent by the source device.

[0278] In another possible implementation, the source device determines the real-time usage status of the source device based on the sensor detection data, and sends the real-time usage status of the source device to the target device. When displaying, the target device displays the first window according to the real-time usage status of the source device and the second preset rule.

[0279] Among them, the second preset rule includes the correspondence between the usage status and the display status of the source device. For example, based on the above Figure 6 example, the second preset range can be that if the folding angle in the usage status of the source device is within the range of [x2°, 180°] and the screen of the source device faces upward, it is determined that the display status of the appearance elements and the screen mirroring interface in the first window is the unfolded upward state shown in 601. Where x2 is a positive number, 0 < x2 < 180. If the folding angle in the usage status of the source device is within the range of [0°, x2°) and the screen of the source device faces upward, it is determined that the display status of the appearance elements and the screen mirroring interface in the first window is the C-screen upward state shown in 603.

[0280] Specifically, when the target device receives the real-time usage status of the source device, it determines the display status corresponding to the real-time usage status according to the second preset rule. The target device displays the first window according to this display status.

[0281] For example, taking the usage status when a folding screen mobile phone starts the collaboration function as Figure 6 the unfolded state shown in 601. After the folding screen mobile phone and the PC establish collaboration, the user folds the folding screen mobile phone from the unfolded state shown in 601 to the folded state shown in 603, and the display interface of the folding screen mobile phone shows the desktop when the user folds it. If the PC determines according to the second rule and the usage status sent by the folding mobile phone that the folding angle in the usage status is within the range of x2 to 180 and the screen of the folding screen device faces upward, the PC determines that the display status of the display content in the first window is the unfolded state shown in 601, and the PC displays the appearance elements according to this display status. When the PC determines according to the second preset rule and the usage status sent by the folding mobile phone that the folding angle in the usage status is less than x2 in the second preset rule and the screen of the source device faces upward, the PC determines that the display status of the display content in the first window is the C-screen upward state shown in 603, and the PC updates the display effect of the appearance elements in the first window according to the C-screen upward state. That is, during the folding process of the folding screen mobile phone,

[0282] It can be understood that in this example, the correspondence between the display status and the usage status is preset in advance, and the target device displays the appearance elements according to the display status corresponding to the real-time usage status of the source device, without showing the complete change process. This simplifies the operation process and helps the user focus on the screen mirroring interface rather than the change process.

[0283] In another possible implementation, the source device determines its usage status based on sensor detection data and a first preset rule, and sends this status to the target device. The target device then displays a first window based on the source device's usage status.

[0284] The first preset rule is described in S401 above.

[0285] Understandably, in this implementation, before the source and target devices interact, the real-time usage status is determined according to a first preset rule. After receiving data from the source device, the target device does not need to process it further and can directly display the data, thus speeding up the display process and improving collaborative efficiency.

[0286] It is understood that the appearance component information in the first information of this application refers to the parameter information corresponding to the appearance component of the source device, so that when the source device and the target device are displayed collaboratively, the target device can generate the appearance element corresponding to the appearance component in the source device based on the appearance component information of the source device. The first interface information in this application refers to the information of the currently displayed interface of the source device, so that when the source device and the target device are displayed collaboratively, the target device can generate a projection interface corresponding to the first interface based on the information of the first interface, and the display content in the projection interface is the same as that of the first interface.

[0287] In this way, the target device displays not only the physical elements corresponding to the hardware of the source device, but also the projection interface corresponding to the source device's display interface during collaborative display. Visually presenting a complete device enhances the realism and completeness of the collaborative display, improving the user experience.

[0288] In this application, through the aforementioned method, during device collaboration, the appearance and content of the source device projected onto the target device are completely consistent with the actual appearance and content of the source device, highly replicating the real device and enhancing the realism, overall coherence, and sophistication of device collaboration, thereby improving the user's sensory experience. Furthermore, by displaying visual elements (i.e., virtual buttons) of the appearance components on the target device, users can directly trigger the hardware functions of the source device by manipulating these visual elements. Enhancing the user experience of operating the visual components on the source device through these visual elements improves the collaborative operation experience. In addition, different forms of the source device (landscape or portrait, folded or unfolded, etc.) can be displayed, and the form of the source device projected onto the target device is consistent with its actual form, making it even more realistic.

[0289] like Figure 13 As shown, another device collaboration method provided in this application embodiment is applied to a first device and a second device. The method includes the following steps.

[0290] The first device is connected to the second device, the first device is the target end device mentioned above, and the second device is the source end device mentioned above. The second device includes an appearance component.

[0291] S1301, The second device displays the first interface, which is the interface currently displayed by the second device.

[0292] The specific implementation of the second device displaying the first interface in this embodiment can be found in S401 above, and will not be repeated here.

[0293] S1302, The second device sends information from the first interface to the first device.

[0294] Correspondingly, the first device receives information from the first interface of the second device.

[0295] The specific implementation method of the second device sending information of the first interface to the first device in this embodiment can be found in S402 above, and will not be repeated here.

[0296] S1303, the first device responds to the information of the first interface and displays a first window; the first window includes a projection interface, the content of which is a mirror image of the first interface; the first window also includes appearance elements, the display effect of which is the same as the visual effect of the appearance components of the second device.

[0297] In this embodiment of the application, the appearance component includes a functional component, and the appearance element includes a functional element. The display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device.

[0298] In this embodiment, the first device may also send a collaborative event to the second device in response to the user's first operation on the functional element; the first device receives information from the second interface of the second device; the second interface is the interface updated by the second device in response to the collaborative event; the first device updates the projection interface in the first window according to the information of the second interface.

[0299] Correspondingly, the second device can also receive collaborative events from the first device; these collaborative events are generated by the second device in response to the user's first operation on a functional element; the second device can also send information about a second interface to the first device, which is the interface updated by the second device in response to the collaborative event, and the information about the second interface is used to update the projection interface in the first window.

[0300] The first operation can be an operation in which the user triggers the collaborative event corresponding to the functional component of the first device.

[0301] In some examples, the first operation includes a second and a third operation. The second operation is the user triggering the display of the target element corresponding to the functional element, and the third operation is the user's action on the target element.

[0302] In this example, the first device responds to the user's input of a second operation within a preset range of the functional element and displays the target element corresponding to the functional element; the first device responds to the user's third operation on the target element and generates a collaborative event.

[0303] Understandably, in this example, the first device displays the target element corresponding to the functional element, and the first device detects the user's operation on the target element to generate a collaborative event, which can improve the accuracy of operation and avoid accidental touches.

[0304] In other examples, the first operation includes a fourth operation. This fourth operation is pre-defined and involves directly manipulating the functional element to trigger a collaborative event corresponding to the functional element generated by the first device.

[0305] In this example, the first device generates a collaborative event in response to the user's fourth action on a functional element.

[0306] Understandably, in this example, the first device does not need to display the target element corresponding to the functional element. The first device detects the user's operation on the functional element and directly generates a collaborative event, which can speed up the collaboration process.

[0307] In this embodiment of the application, the first device may also receive information about the appearance components of the second device from the second device.

[0308] Correspondingly, the second device can also send information about the appearance components of the second device to the first device.

[0309] In this embodiment of the application, the first device may also receive device information of the second device from the second device, and the first device may also obtain information about the appearance components of the second device from the server based on the device information of the second device.

[0310] Correspondingly, the second device can also send its device information to the first device. The device information is used by the first device to obtain information about the appearance components of the second device from the server.

[0311] In this embodiment of the application, the first device may also generate a projection interface based on the information of the first interface; the first device may also generate appearance elements based on the information of the appearance components of the second device.

[0312] In this embodiment of the application, the first device can also obtain information about the updated appearance components of the second device; the first device updates the display effect of the appearance elements in the first window according to the updated appearance component information.

[0313] Correspondingly, the second device can also send information about the updated appearance components of the second device to the first device. The updated appearance component information is used by the first device to update the display effect of the appearance elements in the first window.

[0314] In this application embodiment, the information of the appearance components of the second device includes one or more of the following: the specifications of the second device, the product sample drawing of the second device, or the appearance parameters of the second device.

[0315] In this embodiment of the application, the information of the appearance components of the second device also includes the usage status of the second device.

[0316] In this application embodiment, the appearance components include one or more of the following: frame, volume buttons, power button, camera, housing, interface, and sound outlet.

[0317] In this embodiment of the application, the specific implementation of the first device responding to the information of the first interface and displaying the first window can be found in S403 above, and will not be repeated here.

[0318] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0319] This application embodiment can divide the above-described electronic device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0320] Based on the same inventive concept, embodiments of this application provide a device coordination apparatus. The device coordination apparatus provided in embodiments of this application is applied to… Figure 3The electronic device 300 shown is an example. Figure 14 The diagram shown is a structural schematic of a device collaboration device provided in an embodiment of this application. The device collaboration device can be used to implement the methods described in the above method embodiments. For example, the device collaboration device may specifically include: a processing module 1401, a transceiver module 1402, and a display module 1403.

[0321] The processing module 1401 is used to support the execution of the device coordination device. Figures 4-13 The processing function of any one of them. In this embodiment of the application, taking the device collaboration device applied to the first device as an example, the processing module 1401 is used to process the information of the first interface to obtain a first window. The first window includes a projection interface, the content of which is a mirror of the first interface. The first window also includes appearance elements, and the display effect of the appearance elements is the same as the visual effect of the appearance components of the second device.

[0322] The transceiver module 1402 is used to support the execution of the device coordination device. Figures 4-13 The transceiver module 1402 is configured to receive information from the first interface of the second device, taking the device collaboration device applied to the first device as an example. Correspondingly, taking the device collaboration device applied to the second device as an example, the transceiver module 1402 is configured to send information from the first interface to the first device.

[0323] The display module 1403 is used to support the execution of the device collaboration device. Figures 4-13 The display module 1403 is used to display any of the following functions: In this embodiment, taking the device collaboration device applied to a first device as an example, the display module 1403 is used to display a first window. In this embodiment, taking the device collaboration device applied to a second device as an example, the display module 1403 is used to display a first interface, which is the interface currently displayed on the second device.

[0324] Figure 14 The technical effects of the device coordination device shown can be referred to the technical effects of the method described in the above method embodiments, and will not be repeated here. Figure 14 The processing module 1401 involved in the device coordination device shown can be implemented by a processor or processor-related circuit components, and can be a processor or processing module, etc. The display module 1403 can be implemented by a display screen or related components.

[0325] This application also provides a chip system, such as... Figure 15 As shown, the chip system 1500 includes at least one processor 1501 and at least one interface circuit 1502. As an example, when the chip system 1500 includes a processor and an interface circuit, the processor can be... Figure 15The processor 1501 shown in the solid box (or the processor 1501 shown in the dashed box) may have an interface circuit that can be... Figure 15 The interface circuit 1502 is shown in the solid box (or the dashed box). When the chip system 1500 includes two processors and two interface circuits, the two processors include... Figure 15 The processor 1501 shown in the solid box and the processor 1501 shown in the dashed box, the two interface circuits include Figure 15 Interface circuit 1502 is shown in both solid and dashed boxes. No limitations are imposed on this.

[0326] Processor 1501 and interface circuit 1502 can be interconnected via a line. For example, interface circuit 1502 can be used to receive signals. As another example, interface circuit 1502 can be used to send signals to other devices (e.g., processor 1501). Exemplarily, interface circuit 1502 can read instructions stored in memory and send the instructions to processor 1501. When the instructions are executed by processor 1501, the steps in the above embodiments can be performed. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0327] Optionally, there can be one or more processors in the chip system. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory.

[0328] Optionally, the chip system may also include a memory ( Figure 15 (As shown in the image), there can be one or more memories. Memories can be integrated with the processor or separated from it; this application does not limit this. For example, a memory can be a non-transient processor, such as read-only memory (ROM), which can be integrated with the processor on the same chip or disposed on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0329] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0330] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0331] This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the method described in the above-described method embodiments.

[0332] Computer-readable storage media include, but are not limited to, any of the following: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media capable of storing program code.

[0333] This application provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0334] In addition, this application also provides an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the apparatus to perform the methods in the above-described method embodiments.

[0335] In addition, this application also provides a system, which may specifically be a chip, component or module. The system may include a connected processor and a memory. The memory is used to store computer execution instructions. When the system is running, the processor can execute the computer execution instructions stored in the memory to enable the system to perform the methods in the above-described method embodiments.

[0336] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0337] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, compact disc read-only memory (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an application-specific integrated circuit (ASIC).

[0338] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the division of the above functional modules is only used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0339] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The embodiments can be combined with or referenced to each other without conflict. The apparatus embodiments described above are merely illustrative; for example, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0340] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0341] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0342] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0343] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A device coordination method, characterized in that, Applied to a first device, the first device being connected to a second device, the second device including an appearance component, the method includes: Receive information from a first interface of the second device, where the first interface is the interface currently displayed by the second device; In response to information from the first interface, a first window is displayed; the first window includes a projection interface, the content of which is a mirror image of the first interface; the first window also includes appearance elements, the display effect of which is the same as the visual effect of the appearance components of the second device.

2. The method according to claim 1, characterized in that, The appearance component includes a functional component, and the appearance element includes a functional element. The display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device. The method further includes: In response to the user's first operation on the functional element, a collaborative event is sent to the second device; Receive information from a second interface of the second device; the second interface is the interface of the second device in response to the collaborative event update; Update the screen mirroring interface in the first window based on the information from the second interface.

3. The method according to claim 2, characterized in that, The method further includes: In response to a user inputting a second operation within a preset range of the functional element, a target element corresponding to the functional element is displayed; wherein, the second operation is an operation triggered by the user to display the target element.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Receive information about the appearance components of the second device from the second device; or, Receive device information from the second device; Information about the appearance components of the second device is obtained from the server based on the device information of the second device.

5. The method according to claim 4, characterized in that, The method further includes: The screen projection interface is generated based on the information from the first interface; The appearance element is generated based on the information of the appearance components of the second device.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Obtain information about the updated appearance components of the second device; Based on the updated information of the appearance components, update the display effect of the appearance elements in the first window.

7. The method according to any one of claims 1-6, characterized in that, The information on the appearance components of the second device includes one or more of the following: the specifications of the second device, a product drawing of the second device, or the appearance parameters of the second device.

8. The method according to claim 7, characterized in that, The information on the appearance components of the second device also includes the usage status of the second device.

9. The method according to any one of claims 1-8, characterized in that, The exterior components include one or more of the following: frame, volume buttons, power button, camera, housing, interface, and speaker.

10. A device coordination method, characterized in that, Applied to a second device, the second device being connected to the first device, the second device including an appearance component, the method includes: The first interface is displayed; the first interface is the interface currently displayed by the second device. The information of the first interface is sent to the first device; the information of the first interface is used by the first device to display a first window, the first window including a projection interface, the content of the projection interface being a mirror image of the first interface; the first window also includes appearance elements, the display effect of the appearance elements being the same as the visual effect of the appearance components of the second device.

11. The method according to claim 10, characterized in that, The appearance component includes a functional component, and the appearance element includes a functional element. The display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device. The method further includes: Receive a collaboration event from the first device; the collaboration event is generated by the second device in response to a first operation by the user on the functional element; Send information about the second interface to the first device. The second interface is the interface of the second device in response to the collaborative event update. The information of the second interface is used to update the projection interface in the first window.

12. The method according to claim 10 or 11, characterized in that, The method further includes: Send information about the appearance components of the second device to the first device; or, The device information of the second device is sent to the first device. The device information of the second device is used by the first device to obtain information about the appearance components of the second device from the server based on the device information of the second device.

13. The method according to any one of claims 10-12, characterized in that, The method further includes: The updated appearance component information of the second device is sent to the first device, and the updated appearance component information is used by the first device to update the display effect of the appearance element in the first window.

14. The method according to any one of claims 10-13, characterized in that, The information on the appearance components of the second device includes one or more of the following: the specifications of the second device, a product drawing of the second device, or the appearance parameters of the second device.

15. The method according to claim 14, characterized in that, The information on the appearance components of the second device also includes the usage status of the second device.

16. The method according to any one of claims 10-15, characterized in that, The exterior components include one or more of the following: frame, volume buttons, power button, camera, housing, interface, and speaker.

17. An electronic device, characterized in that, The electronic device includes a processor and a memory coupled to the processor, the memory being used to store computer-readable instructions, which, when read from the memory by the processor, cause the electronic device to perform the method as described in any one of claims 1-9 or 10-16.

18. A chip system, characterized in that, It includes at least one processor and at least one interface circuit, the at least one interface circuit being used to perform transceiver functions and send instructions to the at least one processor, the at least one processor executing the instructions, the at least one processor performing the method as described in any one of claims 1-9 or 10-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1-9 or 10-16.

20. A computer program product, characterized in that, The computer program product includes: a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9 or 10-16.

Images