Device collaboration method and electronic device

Abstract

The present application relates to the technical field of terminals, and provides a device collaboration method and an electronic device, for use in solving the problem of poor user experience caused by an overly simplistic collaborative display interface. The method is applied to a first device, wherein the first device is connected to a second device, and the second device comprises an appearance component. The method comprises: receiving information from a first interface of the second device, wherein the first interface is an interface currently displayed by the second device; and displaying a first window in response to the information of the first interface, wherein the first window comprises a screen mirroring interface, the content of the screen mirroring interface is a mirror image of the first interface, the first window further comprises an appearance element, and the display effect presented by the appearance element is identical to the visual effect presented by the appearance component of the second device. The present application is applied in a device collaboration process.

Description

Device cooperation method and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202411115614.4, filed on August 14, 2024, and entitled "Device cooperation method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal, and in particular, to a device cooperation method and an electronic device. BACKGROUND

[0003] With the development of electronic technology and mobile Internet, users can have more electronic devices such as mobile phones, tablet computers, personal computers (PCs), etc. Data sharing can be achieved between multiple electronic devices.

[0004] In order to more conveniently achieve data sharing between different electronic devices, different electronic devices need to be cooperated. However, the interface of the current cooperation display is relatively simple, and the user experience is poor. SUMMARY

[0005] The present application provides a device cooperation method and an electronic device. In the device cooperation, the first device displays the appearance element and the screen projection interface of the second device, highly restores the real second device, improves the authenticity, overall feeling and delicacy of the device cooperation, and improves the operation experience of the user.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a device cooperation method applied to a first device, the first device being connected with a second device, the second device comprising an appearance component, the method comprising: receiving information of a first interface from the second device, the first interface being a currently displayed interface of the second device; in response to the information of the first interface, displaying a first window; the first window comprising a screen projection interface, the content of the screen projection interface being a mirror image of the first interface; the first window further comprising an appearance element, the display effect presented by the appearance element being the same as the visual effect presented by the appearance component of the second device.

[0008] In the present application, when the first device and the second device are in a device cooperation state, the first device displays the appearance element and the screen projection interface of the second device, highly restores the display of the second device, improves the authenticity, overall feeling and delicacy of the device cooperation, and improves the visual sensory experience of the user. Moreover, the user can manipulate the appearance element to trigger the second device to realize corresponding functions, increases the operation experience of the appearance element in the device cooperation, expands the functionality and the diversity of the cooperation interface of the device cooperation, and enriches the use experience of the user.

[0009] wherein,

[0010] According to the first aspect, or any one of the implementations of the first aspect, the appearance component comprises a function component, and the appearance element comprises a function element, and the display effect presented by the function element is the same as the visual effect presented by the function component of the second device. The method further comprises: in response to the first operation of the user on the function element, sending a collaborative event to the second device; receiving information of a second interface of the second device; the second interface being an interface updated by the second device in response to the collaborative event; and updating the projection interface in the first window according to the information of the second interface.

[0011] According to the first aspect, or any one of the implementations of the first aspect, the method further comprises: in response to the user inputting a second operation within a preset range of the function element, displaying a target element corresponding to the function element; wherein the second operation is an operation of the user triggering the display of the target element.

[0012] In some examples, the first operation comprises a second operation and a third operation. The second operation is an operation of the user triggering the display of the target element corresponding to the function element, and the third operation is an operation of the user on the target element.

[0013] Specifically, the first device displays the target element corresponding to the function element in response to the user inputting the second operation within the preset range of the function element, and generates the collaborative event in response to the third operation of the user on the target element.

[0014] In other examples, the first operation comprises a fourth operation. The fourth operation is an operation of directly manipulating the function element to trigger the first device to generate the collaborative event corresponding to the function element.

[0015] Specifically, the first device generates the collaborative event in response to the fourth operation of the user on the function element.

[0016] In this application, the first device generates a collaborative event in response to the operation of the user on the function element in the appearance element, interacts with the second device according to the collaborative event, obtains information of a second interface corresponding to the collaborative event, and updates the projection interface according to the information of the second interface, which can restore the operation experience of the user on the function component of the source device, increase the real feeling of operation, and improve the user experience.

[0017] According to the first aspect, or any one of the implementations of the first aspect, the method further comprises: receiving information of the appearance component of the second device from the second device.

[0018] According to a first aspect, or any one of the implementations of the first aspect, the method further includes receiving device information of the second device from the second device; and obtaining information of the appearance component of the second device from the server according to the device information of the second device.

[0019] In the present application, the first device can obtain the information of the appearance component of the second device from the second device or the server, so as to subsequently generate the appearance element corresponding to the appearance component according to the information of the appearance component, and enrich the display content during device cooperation. The display of a complete device from the sense organ can enhance the sense of reality and completeness of the cooperative display, and improve the user experience.

[0020] According to the first aspect, or any one of the implementations of the first aspect, the method further includes generating a screen projection interface according to the information of the first interface; and generating an appearance element according to the information of the appearance component of the second device.

[0021] According to the first aspect, or any one of the implementations of the first aspect, the method further includes obtaining information of an updated appearance component of the second device; and updating the display effect of the appearance element in the first window according to the information of the updated appearance component.

[0022] In the present application, the first device can display the state change of the second device, so that the user can quickly know the state change of the second device, the interactivity between the cooperative devices can be enhanced, the display effect of device cooperation can be improved, and the fun and user experience of cooperative use are also improved.

[0023] According to the first aspect, or any one of the implementations of the first aspect, the information of the appearance component of the second device includes one or more of the following: a specification parameter of the second device, a product drawing of the second device, or an appearance parameter of the second device.

[0024] In some examples, if the information of the appearance component of the second device includes the specification parameter of the second device, the product drawing of the second device, and the appearance parameter of the second device, the display state of the appearance element and the screen projection interface in the first window is a preset state.

[0025] In the present application, the first device displays the appearance element and the screen projection interface in the first window according to the preset state, which can accelerate the display efficiency and efficiently display.

[0026] According to the first aspect, or any one of the implementations of the first aspect, the information of the appearance component of the second device further includes a use state of the second device.

[0027] In some examples, the second device determines the use state of the second device according to the data detected by the sensor.

[0028] In some examples, the second device determines the usage state of the second device according to the first preset rule and the data detected by the sensor.

[0029] In some examples, if the information of the appearance component of the second device includes a specification parameter of the second device, a product sketch of the second device, an appearance parameter of the second device, and a usage state of the second device, the display state of the appearance element and the screen projection interface in the first window corresponds to the usage state of the second device.

[0030] For example, if the usage state of the second device is determined according to the data detected by the sensor (i.e., a real-time usage state), the display state of the appearance element and the screen projection interface in the first window is the same as the usage state of the second device.

[0031] Alternatively, if the usage state of the second device is determined according to the data detected by the sensor (i.e., a real-time usage state), the display state of the appearance element and the screen projection interface in the first window is determined according to the usage state of the second device and a second preset rule.

[0032] For another example, if the usage state of the second device is determined according to the data detected by the sensor and a first preset rule, the display state of the appearance element and the screen projection interface in the first window is the same as the usage state of the second device.

[0033] In this application, the first device determines the display state of the appearance element and the screen projection interface in the first window according to the usage state of the second device, so as to more intuitively understand and operate the display content in the first window, and enhance the user experience.

[0034] According to the first aspect, or any one of the implementations of the first aspect, the appearance component includes one or more of the following: a frame, a volume key, a power key, a camera, a housing, an interface, and a sound outlet.

[0035] In a second aspect, the application provides a device cooperation method applied to a second device, the second device being connected with a first device, and the second device including an appearance component. The method includes: displaying a first interface; the first interface being a currently displayed interface of the second device; sending information of the first interface to the first device; the information of the first interface being used by the first device to display a first window, the first window including a screen projection interface, and the content of the screen projection interface being a mirror image of the first interface; the first window further including an appearance element, and the display effect presented by the appearance element being the same as the visual effect presented by the appearance component of the second device.

[0036] According to a second aspect, or any possible implementation mode of the second aspect, the appearance component comprises a function component, the appearance element comprises a function element, and the display effect presented by the function element is the same as the visual effect presented by the function component of the second device. The method further comprises: receiving a collaborative event from the first device; the collaborative event is generated by the second device in response to the first operation of the user on the function element; and sending information of a second interface to the first device, the second interface being an interface updated by the second device in response to the collaborative event, and the information of the second interface being used to update the projection interface in the first window.

[0037] According to the second aspect, or any possible implementation mode of the second aspect, the method further comprises: sending information of an appearance component of the second device to the first device.

[0038] According to the second aspect, or any possible implementation mode of the second aspect, the method further comprises: sending device information of the second device to the first device, the device information of the second device being used by the first device to acquire information of the appearance component of the second device from a server according to the device information of the second device.

[0039] According to the second aspect, or any possible implementation mode of the second aspect, the method further comprises: sending information of an updated appearance component of the second device to the first device, the information of the updated appearance component of the second device being used by the first device to update the display effect of the appearance element in the first window.

[0040] According to the second aspect, or any possible implementation mode of the second aspect, the information of the appearance component of the second device comprises one or more of the following: a specification parameter of the second device, a product drawing of the second device, or an appearance parameter of the second device.

[0041] According to the second aspect, or any possible implementation mode of the second aspect, the information of the appearance component of the second device further comprises a use state of the second device.

[0042] According to the second aspect, or any possible implementation mode of the second aspect, the appearance component comprises one or more of the following: a frame, a volume key, a power key, a camera, a shell, an interface, and a sound outlet.

[0043] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory, the memory being coupled to the processor, and the memory being used to store computer readable instructions, when the processor reads the computer readable instructions from the memory, the electronic device executes the method of the first aspect and any possible implementation mode of the first aspect, or the method of the second aspect and any possible implementation mode of the second aspect.

[0044] In a fourth aspect, the present application provides a device coordination apparatus, comprising: a processor and a memory coupled to the processor, the memory being configured to store computer readable instructions, and the processor being configured to read the computer readable instructions from the memory and cause an electronic device to perform the method according to the first aspect and any one of the implementations of the first aspect, or the method according to the second aspect and any one of the implementations of the second aspect.

[0045] In a fifth aspect, the present application provides a chip system, comprising at least one processor and at least one interface circuit, the at least one interface circuit being configured to perform a transceiving function and send instructions to the at least one processor, the at least one processor being configured to execute the instructions, and the at least one processor being configured to execute the method according to the first aspect and any one of the implementations of the first aspect, or the method according to the second aspect and any one of the implementations of the second aspect.

[0046] In a sixth aspect, the present application provides a computer readable storage medium, comprising a computer program, and the computer program is configured to cause an electronic device to execute the method according to the first aspect and any one of the implementations of the first aspect, or the method according to the second aspect and any one of the implementations of the second aspect when the computer program is executed on the electronic device.

[0047] In a seventh aspect, the present application provides a computer program product, comprising: a computer program or instructions, and the computer program or instructions are configured to cause a computer to execute the method according to the first aspect and any one of the implementations of the first aspect, or the method according to the second aspect and any one of the implementations of the second aspect when the computer program or instructions are executed on the computer.

[0048] The technical effects of the second aspect to the seventh aspect and any one of the implementations of the aspects can refer to the technical effects of the first aspect and any one of the implementations of the first aspect, which will not be described herein. BRIEF DESCRIPTION OF DRAWINGS

[0049] FIG. 1 is a schematic diagram of device coordination provided by an embodiment of the present application;

[0050] FIG. 2 is a schematic diagram of an architecture of a communication system provided by an embodiment of the present application;

[0051] FIG. 3 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application;

[0052] FIG. 4 is a schematic diagram of a device coordination method provided by an embodiment of the present application;

[0053] FIG. 5 is a product drawing of an electronic device provided by an embodiment of the present application;

[0054] FIG. 6 is a second product sample of an electronic device according to an embodiment of the present application;

[0055] FIG. 7 is a third product sample of an electronic device according to an embodiment of the present application;

[0056] FIG. 8 is a first scene diagram of device cooperation according to an embodiment of the present application;

[0057] FIG. 9 is a second scene diagram of device cooperation according to an embodiment of the present application;

[0058] FIG. 10 is a third scene diagram of device cooperation according to an embodiment of the present application;

[0059] FIG. 11 is a fourth scene diagram of device cooperation according to an embodiment of the present application;

[0060] FIG. 12 is a fifth scene diagram of device cooperation according to an embodiment of the present application;

[0061] FIG. 13 is a second flow diagram of a device cooperation method according to an embodiment of the present application;

[0062] FIG. 14 is a structural diagram of a device cooperation apparatus according to an embodiment of the present application;

[0063] FIG. 15 is a structural diagram of a chip system according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing the specific embodiments of the present application, and are not intended to be limiting on the present application. As used in the specification and the appended claims of the present application, the singular forms "a," "an," and "the" are intended to include the plural forms, such as "at least one," unless the context clearly indicates otherwise. It should also be understood that "at least one" and "one or more" refer to one or two or more (including two) in the following embodiments of the present application.

[0065] Reference within the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified

[0066] In the present application, the word "exemplary" or "for example" is used to mean "an example of" or "an example, only. Any implementation described herein as "exemplary" or "for example" is not necessarily to be construed as preferred or advantageous over other implementations. Rather, the use of terms such as "exemplary" or "for example" is merely intended to present concepts in a concrete manner.

[0067] In some examples, data sharing can be achieved by multi-screen cooperation technology (also can be described as multi-device cooperation technology). In a multi-screen cooperation scenario, a source device (also can be described as a casting device) and a sink device (also can be described as a casted device) establish a connection through a wired or wireless manner, and a display interface of the source device can be displayed on a display screen of the sink device.

[0068] As shown in FIG. 1, taking a mobile phone 101 as a source device and a PC 102 as a sink device as an example, when the mobile phone 101 and the PC 102 establish a connection through a wireless or wired manner, and the mobile phone 101 opens a casting function, a display screen of the PC 102 can display a window 103 (also can be described as a mobile phone window or a casting window), which can be used to display a display interface of the mobile phone 101. In the operation process of the multi-screen cooperation, a user can control the display interface of the mobile phone 101 displayed in the window 103 of the PC 102 through a keyboard, a mouse or a touch screen of the PC 102, so as to control the mobile phone 101. For example, the user can use the mouse of the PC 102 to perform a clicking operation on an icon of an application (APP) of the mobile phone 101 displayed in the window 103, so as to trigger the mobile phone 101 to open the application.

[0069] But in the example, the window 103 displayed on the PC 102 only displays the display interface (such as the desktop of the mobile phone and the icons of the applications on the desktop) displayed on the display screen of the mobile phone 101, and does not display the appearance elements (which can be understood as the images corresponding to the appearance components of the mobile phone 101) of the mobile phone 101. For example, the appearance components include the frame and the hardware keys (such as the volume key and the power key) of the mobile phone 101. That is, after the multi-screen cooperation, only one interface is displayed on the target end device (that is, the interface displayed by the source end device is presented), and the overall feeling and the reality of the source end device cannot be presented. The user can only operate the content displayed in the interface, and lacks the manipulation experience of the appearance components of the source end device. Moreover, after different source end devices open the screen projection, the interface is displayed on the target end device, and the specific differences (such as the differences in the frame color and the hardware key position of different source end devices) of different source end devices cannot be embodied. The content cooperatively displayed lacks the delicate and high-end feeling, the interface is relatively simple, and the user experience is poor.

[0070] To solve the above problems, the embodiment of the present application provides a device cooperation method. When the device cooperates, the target end device displays the appearance elements and the projection interface of the source end device, highly restores the real source end device, improves the reality, the overall feeling and the delicacy of the device cooperation, and improves the operation experience of the user.

[0071] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0072] The device cooperation method provided by the embodiment of the present application can be applied to the communication system 200 shown in FIG. 2. As shown in FIG. 2, the communication system 200 can include a source end device 201 and a target end device 202.

[0073] The source end device 201 and the target end device 202 are electronic devices with display screens. The electronic device can also be referred to as a terminal device, a terminal, a user equipment, a mobile station, a mobile terminal, etc.

[0074] When the source end device 201 and the target end device 202 cooperate (taking the example that the source end device 201 projects and displays on the target end device 202), the target end device 202 (that is, the target end device) can not only display the projection interface corresponding to the current display interface of the source end device 201, but also display the appearance elements corresponding to the appearance components of the source end device 201.

[0075] As an example, the source device 201 can include an acquisition module 203 and a collaboration module 204. The target device 202 can include a collaboration module 205.

[0076] The source device 201 and the target device 202 can establish a communication connection through a communication module (not shown in FIG. 2).

[0077] The acquisition module 203 of the source device 201 can acquire first information, which can include information of an appearance component of the source device and information of a first interface. The information of the appearance component of the source device can include parameter information corresponding to the appearance component of the source device when the source device triggers to start a device collaboration function. The information of the first interface can include control information and layout information of the first interface in the source device when the source device triggers to start the device collaboration function. The first interface can be the interface currently displayed by the source device when the source device triggers to start the device collaboration function.

[0078] The acquisition module 203 of the source device 201 can send the first information to the collaboration module 204.

[0079] The source device 201 can send the first information to the collaboration module 205 of the target device 202 through the collaboration module 204.

[0080] The collaboration module 205 of the target device 202 generates a first window according to the first information and displays the first window in the display screen of the target device 202. The first window includes an appearance element and a screen mirroring interface. The display effect of the appearance element is the same as the visual effect of the appearance component of the source device 201, and the content of the screen mirroring interface is a mirror image of the content of the first interface. The appearance element can be understood as a virtual representation of the appearance component of the source device 201.

[0081] In some embodiments, the source device 201 and the target device 202 can be a mobile phone, a PC (including a desktop computer or a notebook computer), a tablet computer, a smart television, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable electronic device, a vehicle-mounted device, a virtual reality device, and the like, which are devices with display functions.

[0082] The operating systems installed in the source device 201 and the target device 202 include, but are not limited to or other operating systems. The embodiments of the present application do not make any limitation on the specific types of the source device and the target device and the operating systems installed therein.

[0083] The source device 201 and the target device 202 can be the same type of electronic device or different types of electronic devices. For example, the source device 201 and the target device 202 can both be mobile phones. Or, the source device 201 and the target device 202 can both be PCs. Or, the source device 201 can be a mobile phone, and the target device 202 can be a PC or a tablet computer, etc. Or, the source device 201 can be a tablet computer, and the target device 202 can be a smart television, etc.

[0084] Optionally, the number of source devices 201 can be one or more. One or more source devices 201 can send first information of the source device 201 to one target device 202, so that the target device 202 displays a corresponding first window according to the received one or more first information. Different devices can be displayed on the same device, and data can be shared, thereby realizing the multi-screen collaborative display function of multiple terminals.

[0085] Specifically, the first information sent by the source device 201 includes information of the appearance component of the source device 201 and information of the first interface. After receiving the first information from the source device 201, the target device 202 generates an appearance element according to the information of the appearance component of the source device 201. The display effect of the appearance element is the same as the visual effect of the appearance component of the source device 201. The target device 202 also generates a screen projection interface according to the information of the first interface in the first information. The screen projection interface is a mirror interface of the first interface, and the content of the screen projection interface is a mirror of the content of the first interface. That is, the display content in the screen projection interface is the same as the display content in the first interface. The target device 202 superimposes the appearance element and the screen projection interface and displays the first window.

[0086] Optionally, the number of target devices 202 can be one or more. The source device 201 can send the first information of the source device 201 to one or more target devices 202, so that the one or more target devices 202 display a first window in the display screen according to the received first information. The first information includes information of the first interface and information of the appearance component of the source device 201. The first window includes an appearance element and a screen projection interface. Different users can use the related functions provided by the source device 201 in different target devices 202, thereby realizing the multi-screen collaborative display function of multiple terminals.

[0087] In some embodiments, the source device 201 can establish a communication connection with the target device 202, and based on the established communication connection, realize the multi-screen collaborative interaction function across devices and systems.

[0088] Specifically, the communication connection established between the source device 201 and the target device 202 can be a wireless communication connection. Alternatively, the communication connection established between the source device 201 and the target device 202 can be a wired communication connection. The specific communication connection manner between the source device 201 and the target device 202 is not limited in the present application.

[0089] For example, the source device or the target device in the embodiments of the present application can be an electronic device shown in FIG. 3. FIG. 3 is a schematic diagram of a hardware structure of an electronic device 300 provided in the embodiments of the present application.

[0090] The electronic device 300 can include a processor 310, an internal memory 320, an antenna 1, an antenna 2, a radio frequency module 330, a communication module 340, a display screen 350, a sensor module 360, and the like.

[0091] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0092] The processor 310 can include one or more processing units. For example, the processor 310 can include an application processor (AP), a system on chip (SoC), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors. The controller can be the nervous system and command center of the electronic device 300. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0093] In the embodiments of the present application, taking the electronic device 300 as an example, the processor 310 can obtain first information, and the first information includes information of an appearance component of the source device and information of a first interface. The information of the appearance component of the source device can include parameter information corresponding to the appearance component of the source device when the source device triggers to start the device collaboration function. The information of the first interface can include control information and layout information of the first interface in the source device when the source device triggers to start the device collaboration function. The first interface can be the interface currently displayed by the source device when the source device triggers to start the device collaboration function.

[0094] In the embodiments of the present application, taking the electronic device 300 as an example, the processor 310 can generate a first window according to the first information. The first window includes an appearance element and a screen projection interface. The display effect of the appearance element is the same as the visual effect of the appearance component of the source device, and the content of the screen projection interface is a mirror image of the first interface content. The processor 310 can also send the first window to the display screen 350, so that the display screen 350 displays the first window.

[0095] The processor 310 can also be provided with a memory for storing instructions and data. In the embodiments of the present application, the memory can be used to store the information of the appearance component of the source device and the like.

[0096] In some embodiments, the memory in the processor 310 is a cache memory. The memory can save instructions or data that have just been used or are recycled by the processor 310. If the processor 310 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 310, thereby improving the efficiency of the system.

[0097] In some embodiments, the processor 310 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0098] The wireless communication function of the electronic device 300 can be implemented by the antenna 1, the antenna 2, the radio frequency module 330, the communication module 340, the modem processor, and the baseband processor, etc.

[0099] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0100] The radio frequency module 330 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the electronic device 300. The radio frequency module 330 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed signals to the modem processor for demodulation. The radio frequency module 330 can also amplify the signals modulated by the modem processor, and convert the signals into electromagnetic waves radiated by the antenna 1. In some embodiments, at least part of the functional modules of the radio frequency module 330 can be arranged in the processor 310. In some embodiments, at least part of the functional modules of the radio frequency module 330 and at least part of the modules of the processor 310 can be arranged in the same device.

[0101] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor displays images or videos through the display screen 350. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 310 and be arranged in the same device as the radio frequency module 330 or other functional modules.

[0102] The communication module 340 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like. The communication module 340 can be one or more devices that integrate at least one communication processing module. The communication module 340 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 310. The communication module 340 can also receive signals to be transmitted from the processor 310, perform frequency modulation and amplification, and convert the signals into electromagnetic wave radiation via the antenna 2.

[0103] In some embodiments, the antenna 1 and the radio frequency module 330 of the electronic device 300 are coupled, and the antenna 2 and the communication module 340 are coupled, so that the electronic device 300 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0104] For example, the electronic device 300 provided by the embodiments of the present application can send messages or instructions to other electronic devices through the antenna 1, the antenna 2, the radio frequency module 330, the communication module 340, etc. Alternatively, the electronic device provided by the embodiments of the present application can receive information or instructions sent by other electronic devices through the antenna 1, the antenna 2, the radio frequency module 330, the communication module 340, etc.

[0105] In the embodiments of the present application, the electronic device 300 is taken as an example of a source device, and the source device establishes a communication connection with other electronic devices (such as a target device) through the radio frequency module 330 or the communication module 340. The source device can also send first information to other electronic devices (such as a target device) through the radio frequency module 330 or the communication module 340, so that other electronic devices (such as a target device) can display a first window according to the first information. The first information includes the information of the appearance component of the source device and the information of the first interface. The first window includes an appearance element and a screen projection interface.

[0106] In the embodiments of the present application, the electronic device 300 is taken as an example of a target device, and the target device establishes a communication connection with other electronic devices (such as a source device) through the radio frequency module 330 or the communication module 340. The target device can also receive the first information sent by other electronic devices (such as a source device) through the radio frequency module 330 or the communication module 340.

[0107] The electronic device 300 realizes the display function through the GPU, the display screen 350, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 350 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 can include one or more GPUs that execute program instructions to generate or change display information.

[0108] The display screen 350 is used to display images, videos, etc. The display screen 350 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 300 can include 1 or N display screens 350, and N is a positive integer greater than 1.

[0109] In the embodiments of the present application, the electronic device 300 is taken as an example of a target device, and the display screen 350 of the target device displays a first window.

[0110] The sensor module 360 of the electronic device 300 can specifically include a gyroscope sensor, an acceleration sensor, an angle sensor, a magnetic sensor, a pressure sensor, an air pressure sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0111] In the embodiments of the present application, the electronic device 300 can determine the use state of the electronic device 300 through the sensor module 360. The use state is the current physical form of the electronic device 300. For example, the electronic device 300 is currently in a landscape state or a portrait state. Alternatively, the electronic device 300 is currently in a folded state (which can also be referred to as a folded state) or an unfolded state (which can also be referred to as an unfolded state).

[0112] For example, the electronic device 300 provided by the embodiments of the present application can determine the use state of the electronic device 300 according to the sensor module 360. For example, the gyroscope sensor in the sensor module 360 can detect the rotation and direction change (which can also be described as the motion posture of the electronic device 300) of the electronic device 300, and the electronic device 300 determines the use state of the electronic device 300 according to the data detected by the gyroscope sensor. Alternatively, the acceleration sensor in the sensor module 360 can detect the acceleration and motion state of the electronic device 300, and the electronic device 300 determines the use state of the electronic device 300 in combination with the algorithm and the data detected by the acceleration sensor. For example, the electronic device 300 is currently in a landscape state or a portrait state.

[0113] Optionally, the display screen 350 of the electronic device 300 is a folding screen. In some embodiments, the folding screen can be a flexible folding screen, which can be folded along a folding edge to form multiple screens. In other embodiments, the folding screen can be a multi-screen folding screen. The multi-screen folding screen can include multiple screens. The multiple screens can be connected in sequence through folding shafts (which can also be described as rotating shafts). Each screen can rotate around the folding shaft connected thereto to realize the folding of the multi-screen folding screen. For example, taking the flexible folding screen as an example, if the electronic device 300 has only one folding edge, the display screen (i.e., the folding screen) can be folded into two independent screens, such as a first screen and a second screen, after being folded along the folding edge. Correspondingly, the use state of the electronic device 300 with the folding screen can include a folded state and an unfolded state.

[0114] The sensor module 360 of the electronic device 300 can further include a folding angle sensor. For example, the electronic device 300 can determine a use state of the electronic device 300 according to the sensor module 360. For example, in a case where the folding screen is a flexible folding screen and the electronic device 300 has only one folding edge, the folding screen is divided into a first screen and a second screen after folding. The folding angle sensor in the sensor module 360 can detect an angle between the first screen and the second screen, and the electronic device 300 can determine a use state of the electronic device 300 according to data detected by the folding angle sensor. For example, the electronic device 300 can determine whether the electronic device 300 is in a folded state or an unfolded state.

[0115] Alternatively, in the sensor module 360, the gyroscope sensor can detect a motion posture of the electronic device 300, the acceleration sensor can detect acceleration and a motion state of the electronic device 300, and the magnetic sensor can detect a magnetic field (e.g., magnetic flux, magnetic force intensity, etc.) of the electronic device 300. The electronic device 300 can determine a use state of the electronic device 300 according to data detected by the gyroscope sensor, the acceleration sensor, and / or the magnetic sensor based on an algorithm.

[0116] It can be understood that, in some embodiments, when the electronic device 300 cooperatively interacts with other electronic devices, the electronic device 300 can send a use state of the electronic device determined according to the sensor module 360 to the other electronic devices cooperatively interacting with the electronic device. In some other embodiments, the electronic device 300 can further send data detected by the sensor module 360 to the other electronic devices cooperatively interacting with the electronic device. In this way, the other electronic devices can simulate and restore the use state of the electronic device 300 according to the received data, and improve the authenticity and real-time performance of the cooperative interaction display.

[0117] The internal memory 320 can be configured to store computer-executable program codes including instructions. The processor 310 can execute various function applications and data processing of the electronic device 300 by running the instructions stored in the internal memory 320. The internal memory 320 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (e.g., a sound playing function, an image playing function, etc.), and the like. The data storage area can store data (e.g., audio data, a phonebook, etc.) created during use of the electronic device 300, and the like. In addition, the internal memory 320 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0118] The device cooperation method of the embodiments of the present application is described in detail below through specific examples and drawings. The following examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples.

[0119] The device cooperation method in the embodiments of the present application can be applied to any device cooperation scenario (which can also be described as a multi-device cooperation scenario or a multi-screen cooperation scenario). For example, an office cooperation scenario, an entertainment scenario, a media scenario, an education scenario, and the like.

[0120] The office cooperation scenario can include, for example, multiple devices cooperating to display on the same display screen in a meeting, multiple devices cooperating to edit a document, and the like. The entertainment scenario can include, for example, multiple devices cooperating to play online games, and the like. For example, a mobile phone is used as a game controller, a tablet computer displays game content, and the mobile phone displays content and casts the content to the tablet computer. The media scenario can include, for example, multiple devices cooperating to play a video, and the like. For example, a PC displays a video, a mobile phone casts content on the PC, and displays video content supplement information, and the like. The education scenario can include, for example, multiple devices cooperating to complete a teaching demonstration, and the like. The device cooperation scenarios are not limited in the embodiments of the present application.

[0121] It can be understood that the system architecture and business scenarios described in the present application are for more clearly illustrating the technical solutions of the present application, and do not constitute the only limitation on the technical solutions provided by the present application. Those skilled in the art can know that the technical solutions provided by the present application are also applicable to similar technical problems as the system architecture evolves and new business scenarios appear.

[0122] The following describes an example in which the source device is a mobile phone and the target device is a PC. FIG. 4 is a flowchart of a device cooperation method provided by an embodiment of the present application. It should be noted that the method is not limited to the specific order described in FIG. 4 and the following description. It should be understood that in other embodiments, the order of some steps in the method can be exchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0123] S400, the source device and the target device establish a communication connection.

[0124] In the embodiments of the present application, the source device can be an electronic device that initiates device cooperation. The target device is an electronic device that cooperates with the source device to display content related to the source device.

[0125] It can be understood that before the source device and the target device establish a communication connection, the source device starts a device cooperation function or a screen projection function.

[0126] In some embodiments, the source device can establish a communication connection with the target device in response to a user operation.

[0127] As an optional implementation, the source device discovers the peripheral devices in response to a user triggering an operation of the source device to start the device collaboration function. The source device determines the electronic device selected by the user from the peripheral devices as a target device to which the source device is to be communicatively connected, and the source device establishes a communication connection with the target device.

[0128] For example, when the phone (i.e., the source device) is unlocked with the screen on, the phone displays a desktop. The phone can receive an operation of the user touching the screen of the phone and swiping down from the top of the screen. In response, the phone can display a control center interface. The user clicks a device collaboration control (which can also be described as a screen projection control) in the control center interface. The phone broadcasts a device collaboration discovery request in the local area network in response to the operation of the user clicking the device collaboration control. The phone determines at least one electronic device (such as a PC, a smart TV, etc.) that returns a device collaboration response in the same local area network as a peripheral device, and displays information of the peripheral devices (such as the device names of the peripheral devices) in the phone to enable the user to select. The phone determines the electronic device (such as the PC) selected by the user from the peripheral devices as a target device. The phone establishes a communication connection with the PC through WiFi.

[0129] In some embodiments, the source device establishes the communication connection with the target device through a wireless connection communication mode.

[0130] The wireless connection communication mode includes, but is not limited to, at least one of the following: a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT) (for example, traditional Bluetooth or Bluetooth low energy (BLE)), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), and the like.

[0131] For example, the source device and the target device start an NFC function, and the source device and the target device establish a communication connection through “tapping”. Alternatively, the source device and the target device are connected to the same local area network, and the source device and the target device establish a communication connection through WiFi. Alternatively, the source device and the target device establish a communication connection through Bluetooth.

[0132] In other embodiments, the source device establishes the communication connection with the target device through a wired connection communication mode.

[0133] For example, the source device and the target device are connected through a video graphics array (VGA), a digital visual interface (DVI), a high definition multimedia interface (HDMI) or a data transmission line to establish a wired communication connection. The source device and the target device implement information transmission through the established wired communication connection.

[0134] The application does not limit the specific implementation of the communication connection between the source device and the target device.

[0135] In S401, the source device acquires first information, which includes information of an appearance component of the source device and information of a first interface.

[0136] In the embodiments of the application, the information of the appearance component of the source device can include parameter information corresponding to the appearance component of the source device when the source device triggers the device collaboration function.

[0137] In some embodiments, the information of the appearance component of the source device can include a specification parameter of the source device, a product drawing of the source device and an appearance parameter of the source device.

[0138] It can be understood that the specification parameter of the source device, the product drawing of the source device and the appearance parameter of the source device are inherent information of the appearance component of the source device when the source device is shipped.

[0139] The specification parameter of the source device can include a brand name of the source device, device information (such as device model) of the source device, basic information (such as device size (such as device length, width and aspect ratio), body information, display screen information, etc.) of the source device, etc.

[0140] The product drawing of the source device is one or more images for displaying the product appearance and design features of the source device. For example, the product drawing of the source device includes multiple images for displaying the appearance structure and design style of the source device from different angles (such as front, side, back, top view angle, etc.), different physical forms.

[0141] Specifically, the product sketch of the source device can show the overall modeling of the source device, such as the frame style (non-foldable frame or foldable frame), the display screen style, the shell (i.e. the back shell) style, the camera style, etc. Among them, the style can include color, material, size, design (such as texture or pattern, etc.), etc. The product sketch of the source device can also show the appearance components of the source device, such as the position, shape, color, size, etc. of the appearance components. Among them, the appearance components of the source device can include one or more of the shell, the frame, the hardware key (such as the volume key, the power key), the interface, the camera (such as the front camera, the rear camera), the sound outlet, the jack, the notch screen, etc.

[0142] Optionally, the product sketch of the source device can also be a three-dimensional image of the source device. For example, a 3D image of the source device obtained according to the 360° image and / or product data simulation of the source device, for 360° showing the source device.

[0143] It can be understood that through the product sketch of the source device, the user can understand the product appearance and the product display effect, etc. In this application, the source device sends the product sketch of the source device to the target device, so that the target device can display the appearance elements corresponding to the appearance components of the source device according to the real appearance image of the source device, highly restore the real appearance of the source device, and improve the user experience.

[0144] For example, if the display screen of the mobile phone is not foldable, as shown in FIG. 5, it is the product sketch of the mobile phone. The product sketch includes the front view 501 of the mobile phone in portrait state, the back view 502 of the mobile phone in portrait state and the front view 503 of the mobile phone in landscape state.

[0145] It can be understood that the above FIG. 5 is only an example. The product sketch of the mobile phone can include product appearance sketches of different angles (such as mobile phone product sketches at an angle), different physical forms, which are not limited here.

[0146] For another example, if the display screen of the mobile phone is foldable, the mobile phone is a foldable screen mobile phone, as shown in FIG. 6, which is the product sketch of the foldable screen mobile phone. The product sketch includes the front view 601 and the back view 602 of the foldable screen mobile phone in the unfolded state. The product sketch also includes the front view 603 and the back view 604 of the foldable screen mobile phone in the folded state. The product sketch also includes the sketch 605 and the sketch 606 of the foldable screen mobile phone in the folded state.

[0147] In the front view 601 in the unfolded state, the display screen of the folding screen mobile phone can be folded along the folding edge 6011 (the folding edge 6011 is not actually displayed in the product) into two independently displayable screens, such as the A screen on the left of the folding edge 6011 and the B screen on the right of the folding edge 6011. The folding manner of the folding screen mobile phone is inward folding, that is, folding inward along the folding edge 6011 (the folding manner can be understood as closing the unfolded book).

[0148] In the rear view 602 in the unfolded state, the middle part is the folding edge 6021 connecting the left and right bodies. The body on the left of the folding edge 6021 includes a rear camera, and the back of the rear camera is the B screen in the front view 601 in the unfolded state. The right of the folding edge 6021 is a non-foldable display screen (that is, the C screen), and the back of the C screen is the A screen in the front view 601 in the unfolded state.

[0149] If the folding screen mobile phone is unfolded as shown in 601, and the A screen is completely folded along the folding edge 6011, the folding screen mobile phone becomes the folded state shown in 603. In the folding process, the A screen is closer and closer to the B screen until the A screen contacts the B screen, and the A screen and the B screen are opposite to each other and invisible to the user. 603 is the front view of the folding screen mobile phone in the folded state. At this time, the C screen opposite to the A screen faces the user, and the user can see the C screen.

[0150] If the folding screen mobile phone is unfolded as shown in 602, and the C screen is completely folded along the folding edge 6021, the folding screen mobile phone becomes the folded state shown in 604.

[0151] The image shown in the diagram 605 is a diagram of the folding screen mobile phone at a certain folding angle during the folding process of folding the folding screen mobile phone from the unfolded state shown in 601 to the folded state shown in 603. The image shown in the diagram 606 is a diagram of the folding screen mobile phone at a certain folding angle during the folding process of folding the folding screen mobile phone from the unfolded state shown in 602 to the folded state shown in 604.

[0152] It can be understood that the folding screen mobile phone is also in the folded state in the images shown in the diagram 605 and the diagram 606. It should be understood that the folded state of the folding screen mobile phone includes all use states between the unfolded state shown in 601 and the folded state shown in 603 of the folding screen mobile phone, or all use states between the unfolded state shown in 602 and the folded state shown in 604 of the folding screen mobile phone. The folding angle is different, and the corresponding use state of the folding screen mobile phone is also different.

[0153] It can be understood that the above-mentioned FIG. 6 is only an example. The product diagram of the mobile phone can include product appearance diagrams of different angles and different physical forms, which are not limited here.

[0154] As shown in FIG. 7, it is a product diagram of a mobile phone including a notch screen. As shown in FIG. 7, the product diagram includes a front view of the mobile phone in a portrait screen state 701, a rear view of the mobile phone in the portrait screen state 702, and a front view of the mobile phone in a landscape screen state 703. In the front view of the mobile phone in the portrait screen state 701 and the front view of the mobile phone in the landscape screen state 703, the mobile phone display screen further includes a notch area 704.

[0155] It can be understood that the product diagram of the source device is an image so that the user can intuitively understand the appearance of the source device.

[0156] The appearance parameters of the source device include specific data of appearance components of the source device.

[0157] For example, the appearance parameters of the source device include one or more of a shell parameter of the source device, a frame parameter of the source device, a hardware key parameter of the source device, an interface parameter of the source device, a camera parameter of the source device, a sound outlet parameter of the source device, a jack parameter of the source device, and a notch screen parameter of the source device.

[0158] The shell parameter of the source device can include data information such as the material, color, size, and weight of the shell. The frame parameter of the source device can include data information such as the material, color, size, and weight of the frame. The hardware key parameter of the source device can include data information such as the position, shape, and size of the hardware key (such as a volume key and a power key), and the hardware key parameter can further include specific functions that can be implemented by the hardware key on the source device (such as volume control by the volume key and power on / off by the power key). The interface parameter of the source device can include data information such as the position, shape, and size of the interface (such as a charging interface and a headphone jack). The camera parameter of the source device can include data information such as the position, shape, size, color, and material of the camera (such as a front camera and a rear camera). The sound outlet parameter of the source device can include data information such as the position, shape, and size of the sound outlet. The jack parameter of the source device can include data information such as the position, shape, and size of the jack (such as an SD card slot). The notch screen parameter of the source device can include data information such as the position, shape, and size of the notch screen.

[0159] It can be understood that the above-mentioned appearance parameters of the source device are the appearance parameters of a non-foldable screen device. If the source device is a foldable screen device, the appearance parameters of the source device further include a folding shaft parameter, and the folding shaft parameter can include data information such as the material, color, and size of the folding shaft.

[0160] It can be understood that the above-mentioned specification parameters of the source device focus on describing the overall attributes of the source device, the appearance parameters of the source device focus on describing the specific data information of the appearance components of the source device, and the product diagram of the source device is a product image of the source device.

[0161] It can be understood that the specification parameters, product drawings and appearance parameters of the source device are inherent information of the source device, belong to system data, and will not be changed with the use of the source device. The specification parameters, product drawings and appearance parameters of the source device can be preconfigured in the source device, and the source device obtains the above information from its own memory when performing device cooperation. Alternatively, the specification parameters, product drawings and appearance parameters of the source device can also not be stored in the source device, and the above information can be obtained by the source device in real time by interacting with the server or the cloud when the source device performs device cooperation.

[0162] It can be understood that if the user configures a new appearance component for the source device, the information of the appearance component of the source device can also be modified, for example, it can be automatically modified, or it can be manually modified by the user. For example, the user configures a phone case for the phone, and the user can modify the phone body information (such as adding the phone case model information to the phone) in the pre-stored appearance component information of the phone, modify the shell parameter in the appearance parameter, and modify the product drawing of the phone (such as adding the product drawing of the phone case). Alternatively, the user uploads the modified appearance component information on the terminal side, and the cloud or server updates according to the modified appearance component information. So that the appearance element corresponding to the changed appearance component can be displayed during collaborative display, ensuring the authenticity of the device. For another example, if the phone case is an active device (such as a ring flash phone case, a liquid cooling phone case, etc.), the phone can automatically identify the model of the configured phone case after the phone is configured with such a phone case, and then the information of the appearance component of the phone can be automatically modified.

[0163] It should be understood that the information of the appearance component in the first information in the above examples is obtained by the source device. However, the specification parameters, product drawings and appearance parameters of the source device are inherent attribute information of the source device, and these information can also be obtained from the server or the cloud.

[0164] In order to reduce the amount of information contained in the first information, in some other embodiments, the information of the appearance component of the source device in the first information can include the device information of the source device.

[0165] It can be understood that the source device transmits the device information of the source device as the information of the appearance component of the source device to the target device, and the target device can obtain the specification parameters, product drawings and hardware parameters of the source device from the server or the cloud according to the device information of the source device, thereby obtaining the complete parameter information of the appearance component of the source device.

[0166] In some other embodiments, the information of the appearance component of the source device can also include the use state of the source device. The use state of the source device can include the physical form of the source device in space. For example, the use state can include a vertical screen state, a horizontal screen state, etc.

[0167] In some examples, the source device can determine the usage state of the source device according to the data detected by the sensor.

[0168] For example, based on the example of FIG. 5, the mobile phone includes a gyroscope sensor that can detect the rotation and direction change (which can also be described as the motion posture of the mobile phone) of the mobile phone. The mobile phone determines the usage state of the mobile phone according to the data detected by the gyroscope sensor. For example, the mobile phone is currently in a landscape state or a portrait state. Alternatively, the mobile phone also includes an acceleration sensor that can detect the acceleration and motion state of the mobile phone. The mobile phone determines the usage state of the mobile phone in combination with the algorithm and the data detected by the acceleration sensor.

[0169] It can be understood that the above introduction of the usage state of the source device is an example of a non-foldable screen device. If the source device is a foldable screen device, the usage state of the source device can also include an unfolded state, a folded state, etc.

[0170] Optionally, the usage state of the source device can also include a device posture and a folding angle, etc.

[0171] For example, as shown in FIG. 6, the device postures of the foldable screen device in the unfolded state include an unfolded state 1 as shown in 601 of FIG. 6, in which the folding angle (i.e., the included angle between the A screen and the B screen folded along the folding edge) is 180°. The large screen composed of the A screen and the B screen faces upwards or towards the user, and the C screen and the rear camera face downwards or away from the user. The device postures of the foldable screen device in the unfolded state also include an unfolded state 2 as shown in 602 of FIG. 6, in which the folding angle is 180°. The large screen composed of the A screen and the B screen faces downwards or away from the user, and the C screen and the rear camera face upwards or towards the user. The device postures of the foldable screen device in the folded state include a folded state 1 as shown in 603 of FIG. 6, in which the folding angle is 0°. The foldable screen device is folded and the C screen faces upwards or towards the user, and the rear camera faces downwards or away from the user. The device postures of the foldable screen device in the folded state also include a folded state 2 as shown in 604 of FIG. 6, in which the folding angle is 0°. The foldable screen device is folded and the C screen faces downwards or away from the user, and the rear camera faces upwards or towards the user. The device postures of the foldable screen device in the folded state also include a folded state 3 as shown in 605 of FIG. 6, in which the folding angle is The foldable screen device is folded and the C screen faces upwards or towards the user, and the rear camera faces downwards or away from the user. The device postures of the foldable screen device in the folded state also include a folded state 4 as shown in 606 of FIG. 6, in which the folding angle is The foldable screen device is folded and the C screen faces downwards or away from the user, and the rear camera faces upwards or towards the user.

[0172] In the above examples, the screen (the entire flexible screen (A screen and B screen), or the C screen) is oriented to the user at a substantially parallel angle, and the screen (the entire flexible screen (A screen and B screen), or the C screen) is oriented to the user at an inclined angle. In the above examples, the screen (the entire flexible screen (A screen and B screen), or the C screen) is oriented upward at a horizontal angle, and the screen (the entire flexible screen (A screen and B screen), or the C screen) is oriented upward at an angle with respect to the horizontal plane.

[0173] For example, based on the example of FIG. 6, the folding screen mobile phone includes a folding angle sensor, and the folding screen mobile phone determines the folding angle of the folding screen mobile phone according to the data detected by the folding angle sensor, and further determines the use state of the folding screen mobile phone. For example, the use state of the folding screen mobile phone is shown in 601 of FIG. 6. Alternatively, the folding screen mobile phone includes a gyroscope sensor, an acceleration sensor, and a magnetic sensor, and the folding screen mobile phone determines the use state of the folding screen mobile phone according to the data detected by the above sensors and an algorithm.

[0174] It can be understood that in this example, the use state of the source device is the real-time use state of the source device detected by the sensor in the source device. The source device can send its use state to the target device, so that the target device can restore the use state of the source device in real time. For example, after the source device and the target device cooperate, the source device changes from the state shown in 601 to the state shown in 603, and the target device synchronously displays the change process of the source device in the display interface.

[0175] In other examples, the source device can determine the use state of the source device according to a first preset rule and sensor detection data.

[0176] Specifically, the source device obtains the detection data of the sensor, and determines the use state corresponding to the detection data according to a first preset rule. The first preset rule is a pre-set rule for dividing the use state of the source device.

[0177] For example, based on the example of FIG. 6, the first preset rule can be that if the folding angle is in the range of [x1°, 180°] and the screen of the source device is oriented upward, the use state of the source device is determined to be the unfolded upward state shown in 601, and the angle of this use state is 180°. Wherein x1 is a positive number, 0 < x1 < 180. If the folding angle is in the range of [0°, x1°), and the screen of the source device is oriented upward, the use state of the source device is determined to be the C screen upward state shown in 603, and the angle of this use state is 0°.

[0178] It can be understood that in the example, the usage state of the source device is classified into several different categories, and the target device displays the usage state of a certain category. For example, if the source device changes from the state shown in 601 to the state shown in 603, the display interface of the target device first displays the unfolded upward state shown in 601, and when it is detected that the folding angle of the source device is less than x1°, the display interface of the target device displays the C-screen upward state shown in 603. That is, in this example, the folding process of the source device is not displayed, and only the folding change result is displayed.

[0179] It can be understood that the embodiments of the present application do not limit the specific content and specific implementation mode of the information of the appearance components of the source device.

[0180] In the embodiments of the present application, the information of the first interface can include the control information and the layout information of the first interface in the source device when the source device is triggered to start the device collaboration function. The control information can include text controls, buttons, picture controls, progress bars, etc. The layout information can include interface layout information, control layout information, etc.

[0181] The first interface is the interface currently displayed by the source device when the source device is triggered to start the device collaboration function (i.e., the display interface of the application program currently running in the foreground). The application program can be a desktop application or any application on the source device, such as a video application.

[0182] For example, based on the example of S400 described above, the first interface is the display interface of the desktop application of the mobile phone. The control information of the first interface includes a plurality of application program icons in the desktop of the mobile phone, and the layout information of the first interface includes the positions and arrangements of the plurality of application program icons, etc.

[0183] S402, the source device sends the first information to the target device.

[0184] In some examples, the source device sends the first information to the target device based on the established communication connection.

[0185] S403, the target device displays a first window according to the first information, the first window includes an appearance element and a screen projection interface, the display effect presented by the appearance element is the same as the visual effect presented by the appearance component of the source device, and the content of the screen projection interface is a mirror image of the first interface content.

[0186] The screen projection interface is a mirror image of the first interface, and the content displayed in the screen projection interface is the same as the content displayed in the first interface. The appearance element can be a virtual representation of the appearance component of the source device. For example, the appearance element is an image corresponding to the appearance component of the source device. The user can trigger the appearance component corresponding to the appearance element in the source device through the appearance element to realize the corresponding function.

[0187] It can be understood that the appearance component is the real appearance hardware of the source device. In the collaborative display, the target device can only simulate and emulate the display of the appearance component of the source device, and the target device generates the corresponding image or virtual key of the peripheral component according to the information of the appearance component of the source device, that is, the appearance element.

[0188] The appearance component can be understood as an entity concept, which corresponds to a specific hardware entity. The appearance component can be a hardware entity that is visible on a physical product or device. Taking a mobile phone as an example, the appearance component can include a shell, a frame, a hardware key (such as a volume key and a power key), an interface, a camera (such as a front camera and a rear camera), a sound outlet, a jack, a notch screen, etc.

[0189] The appearance element can be understood as a virtual display content that can be materialized and presented to the user by relying on the display of the electronic device. The appearance element can be a collection of visual elements such as graphics and icons corresponding to the appearance component, which are rendered and presented on the display and displayed to the user in a visible form through the display screen.

[0190] The display effect can be understood as the result of presenting the appearance element to the user through the display screen of the electronic device, and the display effect depends on the performance, resolution, color rendering capability, etc. of the display screen. The display effect can be the display form of the appearance element, the definition, color vividness, contrast, etc. of the appearance element.

[0191] The visual effect can be understood as the effect or impression produced by the appearance of the physical device or product itself in the visual aspect, and the visual effect depends on factors such as the design, material, and outline of the product. The visual effect can be the reflection effect produced by the appearance component under different light, the material visual perception of the appearance component, etc.

[0192] For example, taking the volume key as the appearance component, the volume key is a physical key, the color of the volume key is black, the shape of the volume key is oval, the length is 10 mm, and the height protruding from the frame of the mobile phone is 1 mm. These visual perceptions that can be observed by the naked eye of the user in the physical aspect can be understood as the visual effect of the volume key. Correspondingly, when the mobile phone and the PC are in a device collaboration state, the mobile phone is displayed on the PC, and the color of the volume key element displayed on the display screen of the PC is black, the shape is oval, the height is 10 mm, and the height protruding from the frame of the mobile phone is 1 mm. The color and shape of the volume key element displayed on the display screen of the PC can be understood as the display effect.

[0193] The display effect of the appearance element is the same as the visual effect of the appearance component of the source device, which means that the appearance element displayed by the target device and the real appearance component of the source device have similar or the same appearance characteristics in vision. Although the appearance component is a real hardware and the appearance element is a virtual display content, the perception or impression of a user in vision is the same, which maintains the consistency of the user's senses.

[0194] In some examples, if the information of the appearance component of the source device in the first information includes the specification parameter of the source device, the product drawing of the source device and the appearance parameter of the source device, the display state of the appearance element and the screen projection interface in the first window is a preset state.

[0195] Specifically, the target device displays the first window according to the first information and the preset state. The target device determines the preset state corresponding to the first interface according to the information of the first interface, and generates the appearance element corresponding to the appearance component in the preset state according to the preset state and the information of the appearance component of the source device. The target device generates the screen projection interface corresponding to the first interface according to the information of the first interface; and the target device displays the first window according to the appearance element and the screen projection interface based on the superposition algorithm.

[0196] In one way, the preset state corresponding to each device type is pre-configured according to the device type. For example, for a non-foldable screen mobile phone, the preset state corresponding thereto is as shown in 501 of FIG. 5, i.e., a vertical screen state. For a foldable screen mobile phone, the preset state corresponding thereto is as shown in 601 of FIG. 6, i.e., a large screen unfolded state.

[0197] In another way, the preset state corresponding to each use scenario is set according to the use scenario. For example, taking a non-foldable screen device as an example, if the device is playing a video and the video is maximized displayed at the same time, the display direction of the first interface in this use scenario is also a horizontal screen direction, and then the preset state corresponding to this use scenario is set as a horizontal screen state. If the device is playing music at the same time, the preset state corresponding to this use scenario is a vertical screen state.

[0198] It can be understood that displaying the appearance element and the screen projection interface in the first window according to the preset state can accelerate the display efficiency and efficiently display.

[0199] In some other examples, if the information of the appearance component of the source device in the first information includes the specification parameter of the source device, the product drawing of the source device, the appearance parameter of the source device and the use state of the source device, the display state of the appearance element and the screen projection interface in the first window corresponds to the use state of the source device.

[0200] Specifically, the target terminal device generates the appearance element corresponding to the use state of the source terminal device according to the information of the appearance component of the source terminal device, and generates the screen projection interface corresponding to the first interface according to the information of the first interface; and the target terminal device displays the first window based on the superposition algorithm according to the appearance element corresponding to the use state of the source terminal device and the screen projection interface.

[0201] It can be understood that in this example, the display state of the appearance element and the screen projection interface in the first window of the target terminal device corresponds to the use state of the source terminal device, and the display state of the appearance element and the screen projection interface in the first window can change along with the use state of the source terminal device.

[0202] For example, the source terminal device is a folding screen device, the folding screen device establishes a connection with the PC, and the user folds the folding screen device from the unfolded state to the folded state. Correspondingly, the appearance element in the first window of the target terminal device also changes from the unfolded state to the folded state, and the screen projection interface is also updated accordingly.

[0203] It should be understood that the first window of the target terminal device displays not only the appearance element corresponding to the appearance component of the source terminal device, but also the screen projection interface corresponding to the first interface of the source terminal device.

[0204] In some examples, the target terminal device generates the appearance element corresponding to the appearance component of the source terminal device according to the size of the first window and the information of the appearance component in the first information, and generates the screen projection interface corresponding to the first interface according to the size of the first window and the information of the first interface; and the target terminal device displays the first window based on the superposition algorithm according to the appearance element and the screen projection interface.

[0205] The size of the first window can be determined according to the first information (for example, the size of the first window is determined according to the body size of the source terminal device in the first information. At this time, the aspect ratio of the first window is the same as that of the source terminal device). Or, the size of the first window can be a preset size (the preset size can be a display size preset by the target terminal device during cooperative display. At this time, the aspect ratio of the first window is the same as or different from that of the source terminal device).

[0206] It should be understood that after the source terminal device is manufactured, the body size and the display size of the device will not be changed, and when the phone case is replaced, only the phone case of the same size is replaced. That is, the appearance hardware of the device can be changed, but the size of the appearance hardware of the device cannot be changed. Therefore, during cooperative display, the target terminal device can adaptively adjust the size of the first window according to the information of the appearance component of the source terminal device. Or, the target terminal device adaptively modifies the size of the appearance element and the screen projection interface according to the size of the preset cooperative display window (i.e., the first window). That is, the size of the appearance element and the screen projection interface is adjusted according to the size of the first window.

[0207] It can be understood that, in order to maintain a better collaborative display effect, the aspect ratio of the appearance element and the projection interface is usually the same as the aspect ratio of the source device. For example, if the size of the first window is a preset size, and the aspect ratio of the first window is different from the aspect ratio of the source device. In this case, the size of the first window is larger than the size of the appearance element and the projection interface, the aspect ratio of the appearance element and the projection interface is the same as the aspect ratio of the source device, and the length and width of the appearance element and the projection interface are determined according to the preset size of the first window and the aspect ratio of the source device. The display area of the first window other than the appearance element and the projection interface is filled with a black image.

[0208] In actual use, the size of the first window can also dynamically change (such as zooming in or out) following the user's use. Correspondingly, the size of the appearance element and the projection interface will change following the change of the first window. That is, the target device can adjust the size of the first window and the size of the appearance element and the projection interface in the first window in response to user operations.

[0209] For example, a mobile phone is projected and displayed on a PC, the PC displays the appearance element and the projection interface in the first window, and the user can move the mouse to the frame or corner of the first window, and the mouse cursor will change to a double-headed arrow. The user clicks the left mouse button and drags the frame or corner to adjust the size of the first window. Alternatively, the user adjusts the window size through keyboard shortcuts. The PC adjusts the size of the first window in response to user operations, the PC adaptively modifies the size of the appearance element and the projection interface according to the adjusted size of the first window, and the PC updates the first window according to the modified appearance element and the projection interface.

[0210] In some examples, if the user zooms in the size of the first window to the size of the display screen of the target device (that is, the first window maximizes to fill the entire display screen of the target device). In this case, the first window can only include the projection interface. In this first window, the projection interface is maximally displayed according to the aspect ratio of the source device, and the appearance element of the source device is not displayed.

[0211] Optionally, if the content displayed in the projection interface in the first window is a video, and the user zooms in the size of the first window to the size of the display screen of the target device, the content displayed in the projection interface can be filled in the first window in the target device, and the appearance element is not displayed in the first window. In order to improve the user's viewing experience.

[0212] In some examples, when the target device displays the first window, the window element of the first window can not be displayed. In vision, only the appearance element and the projection interface are displayed on the display screen of the target device.

[0213] The window element includes one or more of the window frame, the title bar, the navigation bar, and the like.

[0214] The window frame refers to a border around the first window display content (i.e. the appearance element and the screen mirroring interface), which is used to separate the first window from other content displayed on the target device display screen. The window frame can also indicate the position and size of the first window in the target device display screen, and the user can adjust the position and size of the first window by dragging the window frame.

[0215] The title bar is used to implement window management functions, and the title bar is usually located at the top of the window. Various functional controls can be provided in the title bar to allow the user to manage the window through the functional controls. The title bar can also include a window function description. For example, the title bar can include one or more of a close control, a maximize control, a minimize control, and a top control. The close control is used to close the window. The maximize control is used to enlarge the window to the maximum size of the display screen. The minimize control is used to minimize the window to a location such as the taskbar, so that it can be opened again when needed. The top control is used to always display the window on top of other windows, i.e. the window that is on top always remains in the front of the display screen.

[0216] The navigation bar is used to provide quick interface navigation and operation control for the user, and the navigation bar is usually located at the bottom of the window. Various functional controls can be provided in the navigation bar to allow the user to quickly navigate. For example, the navigation bar can include one or more of a back control, a home screen control, and a multi-task control. The back control is usually located at the leftmost side of the navigation bar, and the back control is used to return to the previous display interface or the previous operation step. The user can navigate within an application or system through the back control, returning step by step. The home screen control is usually located in the middle of the navigation bar, and the home screen control is used to return to the home screen. No matter which interface the user is currently in, clicking the home screen control will return to the home screen. The multi-task control is usually located at the rightmost side of the navigation bar, and the multi-task control is used to display recently used applications. The user can quickly switch or close applications through the multi-task control.

[0217] It can be understood that since electronic devices with touch screens (such as mobile phones) can implement specific operation steps through touch or gestures, when the electronic devices are screen mirrored on electronic devices without touch screens (such as PCs), the appearance element and the screen mirroring interface cannot be controlled through touch or gestures. Therefore, electronic devices without touch screens can replace the touch operations of users on electronic devices with touch screens through the title bar, the navigation bar, etc. Moreover, the navigation bar and the title bar can help users understand the context and operation sequence of the interface, so that the user can use the collaborative function smoothly, guarantee the user's use experience, improve the efficiency and comfort of multi-device collaboration, and improve the user experience.

[0218] It can be understood that the navigation bar is displayed in some electronic devices. If the first interface displayed by the source device includes the navigation bar, the navigation bar is correspondingly displayed in the screen mirroring interface when the screen mirroring is performed.

[0219] For example, based on the examples of S400-S401 and the example of 601 in FIG. 6, the target device displays the appearance elements corresponding to the source device and the screen mirroring interface in the first window in the same proportion (that is, the size of the appearance elements and the screen mirroring interface is the same as the size of the source device). As shown in (a) of FIG. 8, it is a display interface when the folding screen mobile phone is in communication connection. Specifically, the folding screen mobile phone is in an unfolded state and the front surface faces the user. The first interface currently displayed by the folding screen mobile phone is a desktop 801. The appearance components of the folding screen mobile phone include a frame 802, a volume key 803, a power key 804, and a front camera 805. The folding screen mobile phone is in communication connection with the PC, and the folding screen mobile phone sends the information of the appearance components of the folding screen mobile phone (such as the specification parameters, product drawings, appearance parameters, and use state of the folding screen mobile phone) and the information of the desktop 801 to the PC.

[0220] As shown in (b) of FIG. 8, the PC displays the appearance elements 806 and the screen mirroring interface 811. Specifically, the PC generates the appearance elements 806 corresponding to the appearance components of the folding screen mobile phone according to the information of the appearance components of the folding screen mobile phone. The appearance elements 806 include a frame element 807, a volume key element 808, a power key element 809, and a front camera element 810. The PC generates the screen mirroring interface 811 according to the information of the desktop 801 in the folding screen mobile phone.

[0221] As shown in (c) of FIG. 8, the PC displays the display interface of the first window. Specifically, the PC superimposes and displays the appearance elements 806 and the screen mirroring interface 811 in the first window 812. The window frame 813 of the first window 812 is represented by a dashed line, and the PC does not display the window frame 813. The superimposed display can be understood as embedding the screen mirroring interface 811 in the appearance elements 806. In this way, the first window 812 displayed on the PC includes both the appearance elements 806 corresponding to the source device and the screen mirroring interface 811 corresponding to the first interface. In the visual aspect, the folding screen mobile phone is displayed on the display screen of the PC in a real and complete manner.

[0222] In some examples, in the case where the target device displays the first window and does not display the window elements of the first window, the target device displays the first window element in response to the operation of the user on the hot area corresponding to the first window element. The first window element can be any one of the window elements of the first window. The hot area corresponding to the first window element is a display area that triggers the target device to display the first window element. The hot area corresponding to the first window element can also be described as being within a preset range of the first window element.

[0223] It can be understood that each window element of the first window can have a corresponding hot area, so that the user manipulates the window element through the hot area to perform a corresponding function.

[0224] As shown in FIG. 9, the PC displays a first window 900 (the window frame of the first window is represented by a dashed line) and does not display window elements of the first window. The first window 900 displays a projection interface corresponding to a desktop of the folding-screen mobile phone and appearance elements corresponding to an appearance of the folding-screen mobile phone. The PC sets a title bar hot area 901 at the top of the appearance elements and the projection interface. The title bar hot area 901 covers not only the appearance elements of the top frame of the folding-screen mobile phone and a part of the display area at the top of the projection interface, but also a part of the display area above the top frame of the folding-screen mobile phone and not in the projection interface. When the user operates the mouse to move the cursor 902 from the current position to the title bar hot area 901, the PC displays a title bar 903 for the user to operate. The title bar 903 has an elliptical display style and includes multiple controls, which are, from left to right, a pin control, a minimize control, a maximize control, and a close control. The pin control is used to pin the first window 900 to be displayed on top and not be blocked by other windows in the PC. The minimize control is used to minimize the first window 900 to the taskbar or other position of the PC, and the PC does not display the first window 900. The maximize control is used to enlarge the first window 900 to the maximum size of the display screen of the PC, that is, to maximize the display of the first window 900. The close control is used to close the first window 900. Correspondingly, if the user continues to move the cursor 902 to make the cursor 902 not in the title bar hot area 901, the PC no longer displays the title bar 903.

[0225] As shown in FIG. 9, the PC sets a navigation bar hot area 904 at the bottom of the appearance elements and the projection interface. The navigation bar hot area 904 covers not only the appearance elements of the bottom frame of the folding-screen mobile phone and a part of the display area at the bottom of the projection interface, but also a part of the display area below the bottom frame of the folding-screen mobile phone and not in the projection interface. When the user operates the cursor 902 to move from the current position to the navigation bar hot area 904, the PC displays a navigation bar 905 for the user to operate. The navigation bar 905 has an elliptical display style and includes multiple controls, which are, from left to right, a back control, a home control, and a multi-task control. The back control is used to return the projection interface to the upper-level interface of the currently displayed interface. The home control is used to control the projection interface to jump from the currently displayed interface to a home screen interface. The multi-task control is used to control the projection interface to display a multi-task interface. Correspondingly, if the user continues to move the cursor 902 to make the cursor 902 not in the navigation bar hot area 904, the PC no longer displays the navigation bar 905.

[0226] It can be understood that in the above example, the hot area corresponding to the first window element is located in the first window, and the hot area corresponding to the first window element covers a part of the display area of the screen mirroring interface and the appearance element and a part of the display area of the non-screen mirroring interface and the appearance element in the first window. In actual application, the hot area corresponding to the first window element can only cover a part of the display area of the screen mirroring interface and the appearance element. The present application does not limit the division manner of the hot area corresponding to the first window element in the target device.

[0227] It can be understood that in the above example, the display style of the first window element is an oval shape, and the first window element is displayed near the hot area. In actual application, the display style of the first window element can also be a rectangular shape or other styles, and the display position of the first window element can also be set at other positions. The present application does not limit the display style and display position of the first window.

[0228] For example, as shown in FIG. 9, if the cursor 902 is located outside the first window 900, when the user controls the cursor to enter the hot area corresponding to the first window element from an area outside the first window 900, the first window element can be displayed in the first window 900, or the first window element can not be displayed in the first window 900. For example, the cursor 902 is located above the first window 900, as shown in the position 910. When the user operates the mouse to move the cursor 902 from 910 downward to the title bar hot area 910, the PC displays the title bar 903. Or, when the user operates the mouse to move the cursor 902 from 910 downward to the title bar hot area 910, the PC does not display the title bar 903.

[0229] It can be understood that the above example is explained by taking the case that the mobile phone is displayed on the PC through screen mirroring, the user operates the mouse, and the cursor moves in the display interface of the PC. In actual application, the target device can be an electronic device including a touch screen, and the user can operate the display interface of the touch screen of the target device through a finger, a touch pen, a capacitive pen, etc. For example, the user touches the title bar hot area of the first window in the display interface with a finger, and the target device displays the title bar in response to the operation.

[0230] In the embodiment of the present application, the target device can be pre-set with an operation capable of triggering the target device to display the first window element. The target device displays the first window and receives the operation of the user on the first window. If it is determined that the operation is the pre-set operation capable of triggering the target device to display the first window element, the first window element is displayed in response to the operation. The present application does not limit the specific implementation manner of the target device to display the first window element.

[0231] In the present application, the target device displays the first window without displaying the window element of the first window when displaying the first window. The target device can make full use of the size of the first window, maximize the display of the appearance element and the screen projection interface in the window, provide a richer and clearer visual experience, and make the window more concise. Moreover, displaying the window element can reduce visual interference, so that the user can focus more on the screen projection interface and the appearance element. The target device can dynamically display or not display the window element of the first window according to the user operation, display the window element in response to the user operation when the user needs to view or operate the window element, and not display the window element when the user does not need to operate the window element, so as to provide a more flexible user experience.

[0232] In the embodiments of the present application, the target device can also set a corresponding hot area for the function element in the first window. The target device displays the function area of the function element in response to the operation of the user on the hot area corresponding to the function element. The user can manipulate the function area, and the target device performs corresponding functions in response to the user operation.

[0233] In some examples, the target device displays the target element corresponding to the function element in response to the trigger operation input by the user within the preset range of the function element.

[0234] The appearance component includes the function component, and the appearance element includes the function element. The display effect presented by the function element is the same as the visual effect presented by the function component of the source device. That is, the function element is an element in the appearance element that can trigger the source device to perform a corresponding function. For example, a volume key, a power key, a camera, etc. The function area of the function element includes the target element corresponding to the function element.

[0235] The hot area corresponding to the function element is a display area that triggers the target device to display the target element corresponding to the function element. The hot area corresponding to the function element can also be described as being within the preset range of the function element.

[0236] For example, based on the example of FIG. 9, the appearance elements in the first window 900 include the frame element, the volume key element, the power key element, and the front camera element. For example, the PC sets a preset range of display area around the volume key element as a volume key hot area 906, where the volume key hot area 906 covers a part of the display area around the volume key element in the projection interface and a part of the display area to the right of the volume key element in the non-projection interface. When the user moves the cursor 902 from the current position to the volume key hot area 906, the PC displays a function area 907 of the volume key element (i.e., the target element corresponding to the volume key element) in the PC. Subsequently, the user can manipulate the function area 907 to change the volume of the foldable screen mobile phone. For another example, the PC sets a preset range of display area around the power key element as a power key hot area 908, where the power key hot area 908 covers a part of the display area around the power key element in the projection interface and a part of the display area to the right of the power key element in the non-projection interface. When the user moves the cursor 902 from the current position to the power key hot area 908, the PC displays a function area 909 of the power key element (i.e., the target element corresponding to the power key element) in the PC. Subsequently, the user can manipulate the function area 909 to turn off the power. For another example, the PC sets a preset range of display area around the front camera element as a front camera hot area (not shown in FIG. 9), and when the user moves the cursor 902 from the current position to the front camera hot area, the PC displays a function area of the front camera element (not shown in FIG. 9) in the PC, and subsequently the user can manipulate the function area to complete the photographing function.

[0237] In some scenarios, the mobile phone and the PC are placed around the same object (e.g., the mobile phone is placed in front of the object and the PC is placed behind the object), the mobile phone establishes a communication connection with the PC, and the mobile phone sends the information of the appearance components and the information of the first interface to the PC, so that the PC displays the first window in the display screen according to the information. The user can manipulate the camera element in the first window in the PC to start the photographing function of the mobile phone to capture the object from the front. The PC can also start the photographing function to capture the object from the back. In this way, the mobile phone and the PC are manipulated on the PC to capture the same object from different directions, achieving multi-angle photographing.

[0238] In the embodiments of the present application, the target end device can be preconfigured with operations that can trigger the target end device to display a function element. The target end device displays the first window and receives the operation of the user on the first window. If it is determined that the operation is the preconfigured operation that can trigger the target end device to display a function element, the target end device displays the function element in response to the operation. The present application does not limit the specific implementation manner of the target end device to display the function element and the display style of the function element.

[0239] It should be understood that, as described above for the first window element, the present application does not limit the display style and display position of the function element, nor does it limit the division manner of the hot area corresponding to the function element.

[0240] In the present application, the target terminal device sets a hot area and a function area for the function element corresponding to the function component in the appearance component of the source terminal device. The user can manipulate the function component corresponding to the function element in the source terminal device through the hot area and the function area of the function element in the target terminal device to realize the corresponding function. This direct manipulation manner enhances the interactivity and real-time response between the user and the device. It also increases the user's manipulation experience of the appearance component of the source terminal device, expands the functionality and applicability of device collaboration, and enriches the user's use experience.

[0241] Moreover, the target terminal device can dynamically display or not display the function area corresponding to the function element according to the user operation. When the user needs to view or manipulate the function element, the function area corresponding to the function element is displayed in response to the user operation. When the user does not need to manipulate the function element, the function area corresponding to the function element is not displayed, which can provide a more flexible user experience. The user can clearly see the function control of the element being manipulated through the function area, which improves the visibility, controllability and convenience of the operation. It also improves the intelligence and integrity of device collaboration.

[0242] It can be understood that the display area of the above window element and the function area corresponding to the appearance element can be described as a function window. The function window does not display when the corresponding hot area is not triggered (e.g., the cursor is not located in the hot area). When the hot area corresponding to the function window is triggered (e.g., the cursor is located in the hot area), the function window corresponding to the hot area is displayed.

[0243] Optionally, the embodiments of the present application can also dynamically change the display effect of the function window according to the distance from the operation point to the hot area. For example, when the user moves the cursor 902 to approach the title bar hot area 901 from the current position, the closer the distance between the cursor 902 and the title bar hot area 901, the higher the clarity of the title bar 903 when it is displayed. When the cursor 902 is located in the title bar hot area 901, the title bar 903 is always displayed, and the clarity is the highest. When the user continues to move the cursor 902 away from the title bar hot area 901, the farther the distance between the cursor 902 and the title bar hot area 901, the lower the clarity of the title bar 903 when it is displayed, until it is not displayed. Alternatively, the closer the distance between the cursor 902 and the title bar hot area 901, the higher the opacity of the title bar 903 when it is displayed; the farther the distance between the cursor 902 and the title bar hot area 901, the lower the opacity of the title bar 903 when it is displayed.

[0244] In some examples, the window elements of the first window are displayed when the target end device displays the first window. That is, the window elements, the appearance elements and the screen projection interface are displayed in the first window.

[0245] For example, as shown in FIG. 10, the window elements of the first window 1000 displayed on the PC include a window frame 1001, a title bar 1002 and a navigation bar 1003. The first window 1000 also includes the appearance elements corresponding to the foldable phone and the screen projection interface 1004. The title bar 1002 includes a pin control 1005, a minimize control 1006, a maximize control 1007 and a close control 1008. The title bar 1002 also includes a textual description of "Device Collaboration" to briefly introduce the function of the window, or the description can be a name such as "My Mate X5" that reflects the model of the foldable phone. The navigation bar 1003 includes a back control 1009, a home control 1010 and a multi-task control 1011. In response to an operation on the back control 1009, the screen projection interface 1004 displays the previous interface of the current interface; in response to an operation on the home control 1010, the screen projection interface 1004 displays the home screen; and in response to an operation on the multi-task control 1011, the screen projection interface 1004 displays the multi-task interface. The appearance elements corresponding to the foldable phone and the screen projection interface 1004 include appearance elements corresponding to the appearance components of the foldable phone and a screen projection interface corresponding to the desktop of the foldable phone.

[0246] It can be understood that in this example, the PC displays the window elements of the first window when displaying the first window, which can provide visual guidance to help users quickly identify the function and purpose of the current first window and improve the operation efficiency of the window elements. Moreover, the standardized window elements make the display content in the first window look more consistent and regular, which can enhance the consistency of user experience.

[0247] For example, as shown in FIG. 10, the appearance elements displayed in the first window 1000 of the PC include a frame element, a volume key element, a power key element and a front camera element. Taking the volume key element and the power key element as examples, a volume key hot area 1012 is set near the volume key element and a power key hot area 1014 is set near the power key element. In FIG. 10, the display area covered by the dashed frame around the volume key element is the volume key hot area 1012, and the display area covered by the dashed frame around the power key element is the power key hot area 1014. If the user operates the mouse to move the cursor to the volume key hot area 1012, the PC displays a function area 1013 of the volume key element in the screen projection interface 1004. If the user operates the mouse to move the cursor to the power key hot area 1014, the PC displays a function area 1015 of the power key element in the screen projection interface 1004.

[0248] It can be understood that the function area corresponding to the function element in the above FIG. 9 and FIG. 10 includes a target element corresponding to the function element, the target element indicates the function of the function element, and the target element user guides the user. Therefore, the PC can generate the target element corresponding to the function element in response to the operation of the user on the hot area corresponding to the function element, and display the target element.

[0249] It can be understood that the above examples take the target terminal device displaying the target element corresponding to the function element as an example, and in other manners, the target terminal device can also not display the target element corresponding to the function element. The target terminal device interacts with the source terminal device according to the operation of the user on the function element, and updates the screen projection interface in the first window.

[0250] In the embodiment of the present application, after the device cooperation, the target terminal device can also detect the operation of the user on the function element, generate a cooperation event (such as the cooperation event can include operation information, operation object, operation position, etc. Wherein the operation information can include the information of click, long press and the like) in response to the user operation, and send the cooperation event to the source terminal device. Then, the source terminal device can determine the user operation according to the cooperation event, and then send the information of the second interface to the target terminal device. Wherein the second interface is the interface updated by the source terminal device in response to the cooperation event. The target terminal device updates the screen projection interface in the first window according to the information of the second interface.

[0251] In some examples, taking the target terminal device displaying the target element as an example, the target terminal device detects the operation of the user on the hot area of the function element, and displays the target element corresponding to the function element in response to the operation; the target terminal device detects the operation of the user on the target element, and generates a cooperation event in response to the operation. The target terminal device sends the cooperation event to the source terminal device, and the source terminal device determines the information of the second interface according to the cooperation event, and sends the information of the second interface to the target terminal device.

[0252] It can be understood that in this example, the user can see the target element in the target terminal device, and the cooperation event can be generated according to the operation of the user on the target element, which can improve the operation efficiency, improve the operation accuracy, and avoid the problem of accidental touch.

[0253] In other examples, taking the target terminal device not displaying the target element as an example, the target terminal device detects the operation of the user on the function element, and generates a cooperation event in response to the operation. The target terminal device sends the cooperation event to the source terminal device, and the source terminal device determines the information of the second interface according to the cooperation event, and sends the information of the second interface to the target terminal device.

[0254] Optionally, the target device can be pre-configured to trigger the generation of the corresponding collaborative event of the function element by directly manipulating the function element, such as clicking the function element or hovering the cursor over the function element. For example, the target device detects the click operation of the user on the volume key element, determines the position of the operation point, and determines that the operation point is located at the volume increase element of the volume key element. In response to the operation, the target device generates a volume increase collaborative event. The target device sends the volume increase collaborative event to the source device, and the source device determines the information of the volume increase interface according to the volume increase collaborative event and sends the information of the volume increase interface to the target device.

[0255] It can be understood that in this example, the target device does not display the target element, which can save the first window space, maximize the display of the screen projection interface and the appearance element in the first window, and make the display content in the first window more concise and clear, thereby providing a better visual experience. Moreover, the user operation intention can be determined in response to the user operation on the function element, the operation experience of the user on the function component of the source device can be restored, the authenticity of the operation is increased, and the user experience is improved.

[0256] In the embodiments of the present application, the device collaboration process includes mirroring collaboration and push collaboration. In the mirroring collaboration process, the display content of the screen projection interface in the target device is the same as the interface display content of the source device. In the push collaboration process, the display content of the screen projection interface in the target device is different from the interface display content of the source device.

[0257] It can be understood that in the mirroring collaboration process, after the source device determines the user operation according to the collaborative event, the second interface is displayed in response to the user operation (i.e., the source device performs the corresponding data processing and logic of the user operation and displays the processing result on the local interface), and the information of the second interface is sent to the target device. In the push collaboration process, after the source device determines the user operation according to the collaborative event, the source device performs the corresponding data processing and logic of the user operation and sends the information of the second interface to the target device (i.e., the source device does not update the local interface).

[0258] For example, based on the example of FIG. 9, when the user moves the cursor 902 to the volume key hot area 906, the PC displays the function area 907 of the volume key element. The user moves the cursor 902 to the increase volume control of the function area 907 of the volume key element (i.e., the "+" control in the function area 907 in FIG. 9), and clicks the increase volume control. The PC generates a collaborative event 1 (such as the collaborative event 1 including the click operation, the increase volume control, etc.) in response to the operation, and sends the collaborative event 1 to the foldable screen mobile phone. The foldable screen mobile phone receives the collaborative event 1 and determines that the user operation is to increase the volume.

[0259] If the device coordination process is mirror coordination, the foldable phone updates the first interface (i.e., the desktop) currently displayed to the second interface (i.e., the display interface of the foldable phone when the volume is increased, which contains a volume bar, and the volume indication in the volume bar after the foldable device increases the volume in response to the coordination event 1) according to the coordination event 1. The foldable phone sends the information of the second interface to the PC. The PC displays the projection interface corresponding to the second interface according to the information of the second interface, as shown in FIG. 11. The projection interface in FIG. 11 displays a volume bar. If the user continues to click the volume increase control through the cursor, the volume indication (the shaded area in the volume bar in FIG. 11) in the volume bar will increase. For example, if a video is played in the projection interface, the user adjusts the volume of the video in the projection interface by clicking the volume increase control in the PC, such as from 30% to 60%. At this time, the volume of the phone will also be adjusted to 60%. Even if the foldable phone and the PC are out of the device coordination state, the volume of the foldable phone will remain at 60%.

[0260] It can be understood that the display interface of the source device and the projection interface of the target device in the mirror coordination process remain consistent. Whether the user inputs an operation in the source device or in the target device, the source device responds to the operation and updates the display interface, and the projection interface of the target device is also updated synchronously. That is, the display and operation effect of the display interface of the source device and the projection interface of the target device are synchronized, which can improve the smoothness of user operation and the consistency of user experience. Moreover, the operation interaction between the source device and the target device is simplified, the user experience is improved, and the user can use multiple devices more efficiently for work or life.

[0261] If the device cooperation process is push cooperation, the foldable phone executes the processing flow and logic of increasing the volume indicated by the cooperation event 1 according to the cooperation event 1, and determines the information of the second interface (that is, in the push cooperation process, the foldable phone will respond to the cooperation event sent by the PC, but will not affect the current display (without updating the first interface) and operation of the foldable phone. That is, when processing the foldable phone and the PC, the foldable phone regards the foldable phone and the PC as independent systems. For example, the foldable phone can be regarded as a server, and the foldable phone can process the information of the two devices, i.e., the foldable phone and the PC, respectively. The foldable phone has independent processing flow and logic for each device, and does not interfere with each other.) The foldable phone sends the information of the second interface to the PC. The PC displays the second interface corresponding to the projection interface according to the information of the second interface, as shown in FIG. 11. In the projection interface of FIG. 11, a volume bar is displayed. If the user continues to click the volume increase control through the cursor, the volume indication (the shaded area in the volume bar of FIG. 11) in the volume bar will increase. For example, if a video is played in the projection interface, the user adjusts the volume of the video in the projection interface by clicking the volume increase control in the PC, such as from 30% to 60%. At this time, the volume of the phone will not be adjusted and will remain at 30% before the device cooperation.

[0262] It can be understood that in the push cooperation process, the source device asynchronously processes the tasks of the source device and the target device, and does not interfere with each other. The source device can respond to the cooperation event sent by the target device without interfering with the current display and operation of the source device, and process the operation of the target device, such as the volume increase cooperation event sent by the target device. This separated control mode enhances the user's management ability of the device and the content, increases flexibility and personalized selection, and can significantly improve the device cooperation efficiency and user experience. Moreover, the source device can maintain the stability of its own display and will not change its own display due to the cooperation event of the target device. The stability of the source device can determine the continuity and predictability of the user using the source device.

[0263] In the embodiment of the present application, when the source device and the target device are in the device cooperation state, if a video or music is played in the projection interface of the target device, the audio of the video or music is managed by the target device, and the audio is output by the loudspeaker of the target device.

[0264] It can be understood that the essence of device cooperation is that the source device acts as an image output device, and the target device displays the image sent by the source device, but the audio is managed and output by the target device. This design helps to unify the audio output source, simplifies management, and improves the convenience and efficiency of operation. It can also provide consistent audio experience and control experience, avoiding the confusion or repetition that may be caused by multiple devices playing audio at the same time.

[0265] For example, based on the example of FIG. 9, if the user operates the mouse to click the maximize control in the title bar 903, the PC displays the interface as shown in FIG. 12 in response to the operation. As shown in (a) of FIG. 12, the folding screen device corresponding to the screen projection interface is displayed in the same size. The PC can set a title bar hot area, and when it is detected that the user's operation point is located in the title bar hot area, the title bar is displayed. For example, the title bar hot area is set at the upper right corner of the display screen, and when the user operates the cursor to move to the upper right corner of the PC display screen, the title bar is displayed. The title bar includes a plurality of controls, from left to right, a minimize control, a restore control, and a close control. The minimize control is used to minimize the window to the PC taskbar or the like. The restore control is used to restore the window to the size before the maximum display. The close control is used to close the window.

[0266] It can be understood that, as shown in (a) of FIG. 12, in order to ensure the authenticity of the source device display, even if the first window is enlarged to the maximum size of the display screen, the source device is displayed in the first window in the same size, and the screen projection interface displayed in the first window does not increase with the window.

[0267] Alternatively, as shown in (b) of FIG. 12, according to the maximum size of the PC and the aspect ratio of the folding screen device, the screen projection interface is maximized to display, so that the user can clearly view the content in the screen projection interface. When the screen projection interface is maximized, the title bar can also be displayed. For example, the title bar is displayed at the upper right corner of the display screen. The title bar includes a plurality of controls, from left to right, a minimize control, a restore control, and a close control. The minimize control is used to minimize the window to the PC taskbar or the like. The restore control is used to restore the window to the size before the maximum display. The close control is used to close the window.

[0268] It can be understood that in the above example, when the first window is maximized, the title bar can be dynamically displayed according to user operation, or directly displayed in the first window. In actual application, it can also be entered into maximized display or exited from maximized display according to preset gestures, shortcut keys, etc. The present application does not limit the specific implementation mode of the maximized display.

[0269] It can be understood that the target end device can also receive the operation of the user input in the first window to the projection interface, and respond to the user operation to interact with the source end device to update the projection interface in the first window. The specific implementation of the target end device to update the projection interface in response to the user operation to the projection interface is known in the art, and will not be described here.

[0270] It can be understood that when the user operates the first window in the target end device, only the function elements in the projection interface or the appearance elements in the first window can trigger the execution of the corresponding functions. The target end device responds to the user operation to interact with the source end device, and changes the display content of the projection interface in the first window according to the information transmitted by the source end device, and cannot change the appearance elements in the first window. The appearance elements displayed in the target end device will change only when the use state of the source end device changes in the mirroring cooperation process.

[0271] It can be understood that the above examples are all described taking the use state of the folding screen phone as the use state 601 in FIG. 6 as an example. In actual application, in the device cooperation process, the hardware structure of the source end device, the use state of the source end device and the display content of the source end device will change with the user use. Correspondingly, the content displayed in the target end device can change with the information sent by the source end device.

[0272] In some examples, in the cooperation process, if the hardware structure of the source end device changes, such as the user configures a new appearance component for the source end device, the source end device updates the information of the appearance component. The target end device can obtain the updated information of the appearance component from the source end device or the server, and update the display effect of the appearance elements in the first window according to the updated information of the appearance component.

[0273] For example, in the device cooperation process, the user replaces the phone case, and changes the color of the phone case from black to red. The phone automatically modifies the information of the pre-stored appearance component in the phone, and modifies the color of the phone case in the information of the appearance component to red. The phone sends the updated information of the appearance component to the PC. The PC changes the color of the phone case appearance element in the first window from black to red according to the updated information of the appearance component.

[0274] In other examples, in the cooperation process, if the use state of the source end device changes, such as the phone changes from the unfolded state to the folded state, the source end device updates the information of the appearance component. The target end device can obtain the updated information of the appearance component from the source end device, and update the display effect of the appearance elements in the first window according to the updated information of the appearance component.

[0275] In a possible implementation, the source device determines a real-time use state of the source device according to sensor detection data, and sends the real-time use state of the source device to the target device. The target device displays the first window according to the real-time use state of the source device when displaying. That is, the appearance elements of the first window and the display state of the screen mirroring interface in the target device are consistent with the real-time use state of the source device.

[0276] For example, the use state of the folding screen phone when starting the collaborative function is the use state shown in 601 in FIG. 6. After the folding screen phone establishes a communication connection with the PC, the user folds the A screen completely along the folding edge 6011, and the folding screen phone becomes the folding state shown in 603. In the process of folding the folding screen phone by the user, the folding screen phone synchronizes the information of the appearance components and the information of the current display interface of the folding screen phone to the PC in real time. The PC synchronously updates the first window according to the information transmitted by the folding screen phone, and displays the folding process of the folding screen phone. That is, the first window displays the change effect / animation of the folding screen phone from the unfolded state to the folded state.

[0277] It can be understood that the target device displays the state change of the source device in real time, so that the user can quickly know the state change of the source device, and the interactivity between the collaborative devices can be enhanced, the display effect of device collaboration is improved, and the fun of collaborative use and the user experience are improved.

[0278] It can be understood that in this implementation, the hardware structure of the source device, the use state of the source device, and the display content of the source device displayed by the target device can be the same as the actual use of the source device. The target device simulates and restores the display of the source device according to the information transmitted by the source device.

[0279] In another possible implementation, the source device determines a real-time use state of the source device according to sensor detection data, and sends the real-time use state of the source device to the target device. The target device displays the first window according to the real-time use state of the source device and a second preset rule when displaying.

[0280] The second preset rule includes a corresponding relationship between the use state and the display state of the source device. For example, based on the example of FIG. 6, the second preset rule can be that if the folding angle in the use state of the source device is in the range of [x2°, 180°], and the screen of the source device is upward, it is determined that the display state of the appearance elements and the screen mirroring interface in the first window is the unfolded upward state shown in 601. Wherein, x2 is a positive number, 0 < x2 < 180. If the folding angle in the use state of the source device is in the range of [0°, x2°), and the screen of the source device is upward, it is determined that the display state of the appearance elements and the screen mirroring interface in the first window is the C screen upward state shown in 603.

[0281] Specifically, the target terminal device determines a display state corresponding to the real-time use state of the source terminal device according to a second preset rule when receiving the real-time use state of the source terminal device. The target terminal device displays the first window according to the display state.

[0282] For example, the use state of the folding-screen mobile phone when starting the collaborative function is the unfolded state shown in 601 of FIG. 6. After the folding-screen mobile phone and the PC establish collaboration, the user folds the folding-screen mobile phone from the unfolded state shown in 601 to the folded state shown in 603, and the display interface of the folding-screen mobile phone displays the desktop when the user folds it. The PC determines that the folding angle in the use state determined by the PC according to the second rule and the use state sent by the folding mobile phone is located in the range of x2 to 180, and the screen of the folding-screen device is upward, and then the PC determines that the display state of the display content in the first window is the unfolded state shown in 601. The PC displays the appearance element according to the display state. When the PC determines that the folding angle in the use state determined by the PC according to the second preset rule and the use state sent by the folding mobile phone is less than x2 in the second preset rule, and the screen of the source terminal device is upward, the PC determines that the display state of the display content in the first window is the C-screen-up state shown in 603, and the PC updates the display effect of the appearance element in the first window according to the C-screen-up state. That is, during the folding process of the folding-screen mobile phone,

[0283] It can be understood that in this example, the correspondence between the display state and the use state is pre-set, and the target terminal device displays the appearance element according to the display state corresponding to the real-time use state of the source terminal device, without the need to display the change process. This simplifies the operation process and helps users to focus on the screen interface rather than the change process.

[0284] In another possible implementation, the source terminal device determines the use state of the source terminal device according to the sensor detection data and a first preset rule, and sends the use state of the source terminal device to the target terminal device. The target terminal device displays the first window according to the use state of the source terminal device when displaying.

[0285] The first preset rule is described above in S401.

[0286] It can be understood that in this implementation, the source terminal device and the target terminal device interact before the first preset rule determines the use state corresponding to the real-time use state. The target terminal device does not need to process again after receiving the data sent by the source terminal device, and can directly display according to the data sent by the source terminal device, which can speed up the display process and improve the collaboration efficiency.

[0287] It can be understood that the information of the appearance component in the first information in the present application is the parameter information corresponding to the appearance component of the source device, so that when the source device and the target device are displayed cooperatively subsequently, the target device can generate the appearance element corresponding to the appearance component of the source device according to the information of the appearance component of the source device. The information of the first interface in the present application is the information of the current display interface of the source device, so that when the source device and the target device are displayed cooperatively subsequently, the target device can generate the screen projection interface corresponding to the first interface according to the information of the first interface, and the display content in the screen projection interface is the same as that in the first interface.

[0288] In this way, when the target device is displayed cooperatively, the appearance element corresponding to the hardware appearance of the source device is displayed, and the screen projection interface corresponding to the display interface of the source device is also displayed. From the sense of sight, a complete device is displayed, which can enhance the sense of reality and completeness of cooperative display and improve the user experience.

[0289] In the present application, when the devices are displayed cooperatively, the shape and display content of the source device displayed by the target device through screen projection are completely consistent with the real shape and actual content of the source device, the real device is highly restored, the reality, integrity and delicacy of device cooperation are improved, and the sense of sight of the user is improved. Moreover, the appearance element (i.e. virtual key) of the appearance component is displayed in the target device, and when the user controls the appearance element, the hardware function of the source device can be triggered directly in the target device. Through the appearance element, the operation experience of the appearance component on the source device is increased, and the cooperative operation feeling is improved. In addition, different forms of source devices (landscape or portrait, folding or unfolding, etc.) can be displayed, and the form of the source device displayed by the target device through screen projection is consistent with the actual form of the source device, which is more real.

[0290] As shown in FIG. 13, another device cooperation method provided by an embodiment of the present application is applied to a first device and a second device, and the method includes the following steps.

[0291] Among them, the first device and the second device are connected, the first device is the target device described above, the second device is the source device described above, and the second device includes an appearance component.

[0292] S1301, the second device displays a first interface, and the first interface is the interface currently displayed by the second device.

[0293] The specific implementation manner of the second device displaying the first interface in the embodiment of the present application can be referred to the above S401, which will not be described here.

[0294] S1302, the second device sends the information of the first interface to the first device.

[0295] Correspondingly, the first device receives the information of the first interface from the second device.

[0296] The specific implementation of the second device sending the information of the first interface to the first device in the embodiments of the present application can be referred to the foregoing S402, and details are not described herein.

[0297] S1303, the first device displays a first window in response to the information of the first interface; the first window includes a screen projection interface, and the content of the screen projection interface is a mirror image of the first interface; the first window further includes an appearance element, and the display effect presented by the appearance element is the same as the visual effect presented by the appearance component of the second device.

[0298] In the embodiments of the present application, the appearance component includes a function component, and the appearance element includes a function element, and the display effect presented by the function element is the same as the visual effect presented by the function component of the second device.

[0299] In the embodiments of the present application, the first device can further send a collaborative event to the second device in response to the first operation of the user on the function element; the first device receives the information of the second interface from the second device; the second interface is an interface updated by the second device in response to the collaborative event; and the first device updates the screen projection interface in the first window according to the information of the second interface.

[0300] Correspondingly, the second device can also receive a collaborative event from the first device; the collaborative event is generated by the second device in response to the first operation of the user on the function element; and the second device can send the information of the second interface to the first device, the second interface is an interface updated by the second device in response to the collaborative event, and the information of the second interface is used to update the screen projection interface in the first window.

[0301] The first operation can be an operation of triggering the first device to generate a collaborative event corresponding to the function component by the user.

[0302] In some examples, the first operation includes a second operation and a third operation. The second operation is an operation of triggering the display of a target element corresponding to the function element by the user, and the third operation is an operation of the user on the target element.

[0303] In this example, the first device displays the target element corresponding to the function element in response to the second operation of the user inputting within a preset range of the function element; and the first device generates the collaborative event in response to the third operation of the user on the target element.

[0304] It can be understood that in this example, the first device displays the target element corresponding to the function element, and the first device detects the operation of the user on the target element to generate the collaborative event, which can improve the operation accuracy and avoid mis-touch.

[0305] In some examples, the first operation comprises a fourth operation. The fourth operation is preset, and the fourth operation is directly manipulating the function element to trigger the first device to generate the collaborative event corresponding to the function element.

[0306] In this example, the first device generates the collaborative event in response to the fourth operation of the user on the function element.

[0307] It can be understood that in this example, the first device does not need to display the target element corresponding to the function element, and the first device detects the operation of the user on the function element to directly generate the collaborative event, which can accelerate the collaborative efficiency.

[0308] In the embodiments of the present application, the first device can also receive the information of the appearance component of the second device from the second device.

[0309] Correspondingly, the second device can also send the information of the appearance component of the second device to the first device.

[0310] In the embodiments of the present application, the first device can also receive the device information of the second device from the second device, and the first device can also obtain the information of the appearance component of the second device from the server according to the device information of the second device.

[0311] Correspondingly, the second device can also send the device information of the second device to the first device, and the device information of the second device is used by the first device to obtain the information of the appearance component of the second device from the server according to the device information of the second device.

[0312] In the embodiments of the present application, the first device can also generate a screen projection interface according to the information of the first interface; and the first device generates the appearance element according to the information of the appearance component of the second device.

[0313] In the embodiments of the present application, the first device can also obtain the updated information of the appearance component of the second device; and the first device updates the display effect of the appearance element in the first window according to the updated information of the appearance component.

[0314] Correspondingly, the second device can also send the updated information of the appearance component of the second device to the first device, and the updated information of the appearance component of the second device is used by the first device to update the display effect of the appearance element in the first window.

[0315] In the embodiments of the present application, the information of the appearance component of the second device comprises one or more of the following: a specification parameter of the second device, a product drawing of the second device, or an appearance parameter of the second device.

[0316] In the embodiments of the present application, the information of the appearance component of the second device further comprises a use state of the second device.

[0317] In the embodiments of the present application, the appearance component includes one or more of the following: a frame, a volume key, a power key, a camera, a housing, an interface, and a sound outlet.

[0318] The specific implementation of the first device displaying the first window in response to the information of the first interface in the embodiments of the present application can be referred to the foregoing S403, and details are not described herein.

[0319] The above describes the solutions provided by the embodiments of the present application mainly from the perspective of methods. To implement the above functions, the corresponding hardware structure and / or software modules for executing each function are included. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0320] The embodiments of the present application can divide the functions of the above electronic device according to the above method examples, for example, each function module can be divided according to each function, or two or more functions can be integrated in one processing module. The above integrated module can be implemented in the form of hardware or software function module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division manner.

[0321] Based on the same inventive concept, the embodiments of the present application provide a device collaboration apparatus. The device collaboration apparatus provided by the embodiments of the present application is applied to the electronic device 300 shown in FIG. 3. As shown in FIG. 14, it is a structure schematic diagram of a device collaboration apparatus provided by the embodiments of the present application. The device collaboration apparatus can be used to implement the methods recorded in the above various method embodiments. Exemplarily, the device collaboration apparatus can specifically include: a processing module 1401, a transceiver module 1402, and a display module 1403.

[0322] The processing module 1401 is configured to support the device collaboration apparatus to perform the processing functions of any one of FIGS. 4-13. In the embodiments of the present application, taking the case that the device collaboration apparatus is applied to the first device as an example, the processing module 1401 is configured to process a first window in response to information of a first interface, the first window including a screen projection interface, and the content of the screen projection interface being a mirror image of the first interface; the first window further includes an appearance element, and the display effect of the appearance element is the same as the visual effect of an appearance component of a second device.

[0323] The transceiving module 1402 is configured to support the device collaboration apparatus to perform the transceiving function of any one of FIGS. 4-13. In the embodiments of the present application, taking the device collaboration apparatus applied to the first device as an example, the transceiving module 1402 is configured to receive the information of the first interface from the second device. Correspondingly, taking the device collaboration apparatus applied to the second device as an example, the transceiving module 1402 is configured to send the information of the first interface to the first device.

[0324] The display module 1403 is configured to support the device collaboration apparatus to perform the display function of any one of FIGS. 4-13. In the embodiments of the present application, taking the device collaboration apparatus applied to the first device as an example, the display module 1403 is configured to display the first window. In the embodiments of the present application, taking the device collaboration apparatus applied to the second device as an example, the display module 1403 is configured to display the first interface, which is the interface currently displayed by the second device.

[0325] The technical effects of the device collaboration apparatus shown in FIG. 14 can refer to the technical effects of the methods described in the above method embodiments, which will not be repeated here. The processing module 1401 involved in the device collaboration apparatus shown in FIG. 14 can be realized by a processor or a processor-related circuit component, and can be a processor or a processing module, etc. The display module 1403 can be realized by a display screen and other related components.

[0326] The chip system 1500 includes at least one processor 1501 and at least one interface circuit 1502. For example, when the chip system 1500 includes one processor and one interface circuit, the one processor can be the processor 1501 shown in the solid line box in FIG. 15 (or the processor 1501 shown in the dashed line box), and the one interface circuit can be the interface circuit 1502 shown in the solid line box in FIG. 15 (or the interface circuit 1502 shown in the dashed line box). When the chip system 1500 includes two processors and two interface circuits, the two processors include the processor 1501 shown in the solid line box and the processor 1501 shown in the dashed line box in FIG. 15, and the two interface circuits include the interface circuit 1502 shown in the solid line box and the interface circuit 1502 shown in the dashed line box in FIG. 15. This is not limited.

[0327] The processor 1501 and the interface circuit 1502 can be interconnected through a line. For example, the interface circuit 1502 can be configured to receive a signal. For another example, the interface circuit 1502 can be configured to send a signal to other devices (such as the processor 1501). For example, the interface circuit 1502 can read instructions stored in a memory and send the instructions to the processor 1501. When the instructions are executed by the processor 1501, each step in the above embodiments can be performed. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0328] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software codes stored in a memory.

[0329] Optionally, the chip system can further include a memory (not shown in FIG. 15), which can also be one or more. The memory can be integrated with the processor or arranged separately from the processor, which is not limited in the present application. For example, the memory can be a non-transient processor such as a read only memory (ROM), which can be integrated on the same chip as the processor or arranged on different chips, and the type of the memory and the arrangement of the memory and the processor are not limited in the present application.

[0330] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0331] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0332] The embodiments of the present application further provide a computer storage medium, which stores computer instructions. When the computer instructions run on an electronic device, the electronic device executes the method described in the above method embodiments.

[0333] The computer readable storage medium includes, but is not limited to, any one of the following: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0334] The embodiment of the present application provides a computer program product, which comprises computer programs or instructions, and when the computer programs or instructions run on a computer, the computer programs or instructions make the computer execute the method provided in the method embodiment.

[0335] In addition, the embodiment of the present application further provides an apparatus, which can be a chip, a component or a module in particular, and the apparatus can comprise a processor and a memory connected to each other; wherein the memory is used for storing computer execution instructions, and when the apparatus runs, the processor can execute the computer execution instructions stored in the memory, so that the apparatus executes the method in each method embodiment.

[0336] In addition, the embodiment of the present application further provides a system, which can be a chip, a component or a module in particular, and the system can comprise a processor and a memory connected to each other; wherein the memory is used for storing computer execution instructions, and when the system runs, the processor can execute the computer execution instructions stored in the memory, so that the system executes the method in each method embodiment.

[0337] The electronic device, the computer storage medium, the computer program product or the chip provided in the embodiment are used for executing the corresponding method provided in the above, therefore, the beneficial effects that can be achieved by the electronic device, the computer storage medium, the computer program product or the chip can refer to the beneficial effects in the corresponding method provided in the above, and here, the description is not repeated.

[0338] The steps of methods or algorithms described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC.

[0339] Those skilled in the art can clearly understand the method of the present application from the above description of the embodiments. For the convenience and brevity of description, only the above-mentioned division of functional modules is taken as an example. In actual applications, the above-mentioned functions can be completed by different functional modules according to needs; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0340] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. Each embodiment can be combined or referred to each other in the case of no conflict. The device embodiment described above is only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed each other can be through some interface, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0341] The units described as separate components may or may not be physically separate, and the components displayed as units may be a physical unit or multiple physical units, that is, may be located in one place, or also can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0342] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0343] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical scheme of the embodiments of the present application essentially or the part of the prior art that contributes to the technical scheme or all or part of the technical scheme can be embodied in the form of a software product, and the software product is stored in a storage medium, including a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0344] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A device collaboration method, characterized by, The method is applied to a first device connected with a second device, and the second device comprises an appearance component, and the method comprises: receiving information of a first interface of the second device, the first interface being a currently displayed interface of the second device; displaying a first window in response to the information of the first interface; the first window comprises a screen projection interface, and content of the screen projection interface is a mirror image of the first interface; the first window further comprises an appearance element, and a display effect presented by the appearance element is the same as a visual effect presented by the appearance component of the second device.

2. The method of claim 1, wherein, The appearance component comprises a function component, and the appearance element comprises a function element, and a display effect presented by the function element is the same as a visual effect presented by the function component of the second device. The method further comprises: sending a cooperative event to the second device in response to a first operation of the function element by a user; receiving information of a second interface of the second device; the second interface is an interface of the second device updated in response to the cooperative event; updating the screen projection interface in the first window according to the information of the second interface.

3. The method of claim 2, wherein, The method further comprises: displaying a target element corresponding to the function element in response to a second operation input by a user within a preset range of the function element; the second operation is an operation of triggering display of the target element by the user.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: receiving information of the appearance component of the second device from the second device; or, receiving device information of the second device from the second device; obtaining information of the appearance component of the second device from a server according to the device information of the second device.

5. The method of claim 4, wherein, The method further comprises: generating the screen projection interface according to the information of the first interface; generating the appearance element according to the information of the appearance component of the second device.

6. The method according to any one of claims 1-5, characterized in that, The method further comprises: obtaining information of the updated appearance component of the second device; updating a display effect of the appearance element in the first window according to the information of the updated appearance component.

7. The method according to any one of claims 1 to 6, characterized in that, The information of the appearance component of the second device comprises one or more of the following: a specification parameter of the second device, a product drawing of the second device, or an appearance parameter of the second device.

8. The method of claim 7, wherein, The information of the appearance component of the second device further comprises a use state of the second device.

9. The method according to any one of claims 1-8, characterized in that, The appearance component comprises one or more of the following: a frame, a volume key, a power key, a camera, a housing, an interface, and a sound outlet.

10. A device collaboration method, characterized by, The method is applied to a second device connected with a first device, and the second device comprises an appearance component, and the method comprises: displaying a first interface; the first interface being a currently displayed interface of the second device; sending information of the first interface to the first device; the information of the first interface is used for the first device to display a first window, the first window comprises a screen projection interface, and content of the screen projection interface is a mirror image of the first interface; the first window further comprises an appearance element, and a display effect presented by the appearance element is the same as a visual effect presented by the appearance component of the second device.

11. The method of claim 10, wherein, The appearance component comprises a functional component, and the appearance element comprises a functional element, and the display effect presented by the functional element is the same as the visual effect presented by the functional component of the second device; The method further comprises: receiving a cooperative event from the first device; the cooperative event is generated by the second device in response to a first operation of the user on the functional element; sending information of a second interface to the first device, the second interface being an interface updated by the second device in response to the cooperative event, and the information of the second interface being used to update the projection interface in the first window.

12. The method according to claim 10 or 11, characterized in that, The method further comprises: sending information of the appearance component of the second device to the first device; or, sending device information of the second device to the first device, the device information of the second device being used by the first device to obtain information of the appearance component of the second device from a server according to the device information of the second device.

13. The method according to any one of claims 10-12, characterized in that, The method further comprises: sending information of the updated appearance component of the second device to the first device, the information of the updated appearance component of the second device being used by the first device to update the display effect of the appearance element in the first window.

14. The method according to any one of claims 10-13, characterized in that, The information of the appearance component of the second device comprises one or more of the following: a specification parameter of the second device, a product drawing of the second device, or an appearance parameter of the second device.

15. The method of claim 14, wherein, The information of the appearance component of the second device further comprises a use state of the second device.

16. The method according to any one of claims 10-15, characterized in that, The appearance component comprises one or more of the following: a frame, a volume key, a power key, a camera, a shell, an interface, and a sound outlet.

17. An electronic device, comprising: The electronic device comprises a processor and a memory, the memory being coupled to the processor, and the memory being used to store computer readable instructions, when the processor reads the computer readable instructions from the memory, the electronic device executes the method as claimed in any one of claims 1-9 or 10-16.

18. A chip system, characterized by The electronic device comprises at least one processor and at least one interface circuit, the at least one interface circuit being used to perform a transceiving function and send instructions to the at least one processor, the at least one processor executes the instructions, and the at least one processor executes the method as claimed in any one of claims 1-9 or 10-16.

19. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a computer program, when the computer program runs on an electronic device, the electronic device executes the method as claimed in any one of claims 1-9 or 10-16.

20. A computer program product, characterised in that, The computer program product comprises a computer program or instructions, when the computer program or instructions run on a computer, the computer executes the method as claimed in any one of claims 1-9 or 10-16.

Images