Function collaboration method and electronic equipment

Abstract

The embodiment of the invention provides a function collaboration method and electronic equipment, and relates to the technical field of electronic equipment. In the method, a user can operate an application program corresponding to an application program control on a second electronic device by executing operation on the application program control on a first electronic device; the first electronic equipment can display part of contents of a user interface of the application program; and along with the operation of the user on the currently displayed part of the content of the user interface, the first electronic equipment can translate the user interface to display the other part of the content of the user interface. In conclusion, according to the scheme, the function of the application program in the second electronic equipment can be expanded to the first electronic equipment, and the user interface of the application program displayed in the first electronic equipment can display more, richer, more effective or more detailed contents, so that the user experience is higher, and the power consumption overhead and operation delay caused by frequent interaction between the equipment are avoided.

Description

Technical Field

[0001] The present application relates to the technical field of electronic devices, and in particular to a functional collaboration method and electronic device. Background Art

[0002] In recent years, with the development of electronic technology, there are more and more types of smart electronic devices, enriching and facilitating users' lives. For example, various light smart devices (such as smart watches, smart bracelets, smart glasses, etc.) can not only provide the function of checking time, but also provide more abundant other functions, covering sports health records, medical monitoring, instant messaging, payment transactions, and entertainment games. Therefore, people's requirements for the functions of these light smart devices are getting higher and higher.

[0003] However, since light smart devices are constrained by hardware resources such as processors, screens, and batteries, the functions provided by light smart devices are affected compared to ordinary electronic devices. Summary of the invention

[0004] The present application provides a functional collaboration method and an electronic device for extending the functions of one electronic device to another electronic device, thereby improving user experience.

[0005] In a first aspect, an embodiment of the present application provides a function collaboration method. The method can be applied to a first electronic device, where an application control is installed. The method includes:

[0006] The first electronic device sends a run instruction to a second electronic device in response to a first operation by a first user on the application control on the first electronic device, wherein the run instruction is used to instruct the second electronic device to run the application corresponding to the application control; thereafter, the first electronic device displays a first portion of content of a first user interface; wherein the first user interface is the user interface of the application; the first electronic device translates the first user interface in response to a second operation by the first user on the first portion of content to display a second portion of content of the first user interface; wherein the second portion of content is different from the first portion of content.

[0007] This solution only requires the first electronic device to have communication and display capabilities, and does not have strict requirements on its software and hardware resources, and will not cause large power consumption and memory overhead to the first electronic device. Therefore, this solution can extend the functions of the second electronic device to the first electronic device without being constrained by the software and hardware resources of the first electronic device, thereby realizing functional collaboration between the two electronic devices. In addition, the first electronic device can also display part of the user interface content of the collaborative application, and the user interface can be translated through simple local operations by the user, so that the user interface displayed by the first electronic device can display more, richer, more effective or more detailed content. In addition, the above process does not require frequent communication and interaction between electronic devices. The first electronic device can switch the display content through local operations, which also improves the control fluency of the first electronic device over the collaborative application, and avoids power consumption and operation delays caused by frequent interactions between devices.

[0008] In one possible design, the first electronic device can receive a first message from the second electronic device; wherein the first message is used to indicate a second user interface of the application running on the second electronic device, and the first user interface includes the content of a partial interface area or the content of the entire interface area of ​​the second user interface.

[0009] Through this design, the first electronic device and the second electronic device can obtain the second user interface of the application running on the second electronic device through interaction.

[0010] In one possible design, the first electronic device may also scale the first user interface in response to a third operation of the first user on the first user interface to display all content of the first user interface.

[0011] Through this design, the first electronic device can scale the display of the first user interface according to the user's local operation to display the overall content of the first user interface and improve the user experience.

[0012] In one possible design, after translating the first user interface, when the second portion of content displayed by the first electronic device includes the boundary of the second user interface, the first electronic device sends a second message to the second electronic device, where the second message is used to indicate that the second user interface has been slid.

[0013] Since the first electronic device displays part of the content of the first user interface, it is equivalent to displaying part of the content of the second user interface. Therefore, when the first user translates the first user interface through the second operation so that the part of the content of the first user interface displayed by the first electronic device includes the boundary of the second user interface, it can be represented that the first user performs a sliding operation on the second user interface on the first electronic device side. Therefore, in this case, the first electronic device can send a second message to the second electronic device to notify the second electronic device that the second user interface has been slid, so that the second electronic device can perform subsequent operation steps based on the event, such as updating the user interface.

[0014] In one possible design, the second message includes sliding direction information of the second user interface.

[0015] Through this design, the first electronic device can also notify the second user interface of the sliding direction, so that the second electronic device can perform subsequent operation steps according to the sliding direction.

[0016] In a possible design, the sliding direction information includes the position of the boundary in the second user interface or the translation direction of the second operation indication.

[0017] In one possible design, after the first electronic device sends the second message to the second electronic device, the first electronic device may also receive a third message from the second electronic device; wherein the third message is used to indicate a third user interface of the application running on the second electronic device, and the third user interface is generated by the second electronic device based on the second message; thereafter, the first electronic device may display a third portion of content of a fourth user interface; wherein the fourth user interface contains the content of a partial interface area or the content of the entire interface area of ​​the third user interface; and the first electronic device translates the fourth user interface in response to the first user's second operation on the third portion of content to display a fourth portion of content of the fourth user interface; wherein the fourth portion of content is different from the third portion of content.

[0018] Through this design, the second electronic device can notify the first electronic device of the updated user interface, so that the first electronic device can synchronously display the new user interface.

[0019] In one possible design, the first electronic device may also send a fourth message to the second electronic device in response to a fourth operation of the first user on the first user interface; wherein the fourth operation is a control operation on the first user interface, and the fourth message is used to indicate the fourth operation.

[0020] Through this design, the first electronic device can also achieve distributed control of the application on the second electronic device through the first user's control operation on the first user interface.

[0021] In one possible design, the fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time; wherein the operation location information includes a first operation location or a second operation location; the first operation location is the location where the fourth operation is mapped to the second user interface; and the second operation location is the location of the fourth operation in the first user interface.

[0022] Through this design, the second electronic device can perform subsequent operations based on the relevant information of the fourth operation in the fourth message after receiving the fourth message.

[0023] In a possible design, when the operation position information includes the second operation position, the operation position information also includes relative position information of the first user interface with respect to the second user interface.

[0024] Through this design, after receiving the fourth message, the second electronic device can determine the first operation position based on the second operation position and the relative position information, so that it can execute the operation instruction corresponding to the control operation on the second user interface according to the first operation position.

[0025] In one possible design, after the first electronic device sends a fourth message to the second electronic device in response to the first user's fourth operation on the first user interface, the first electronic device may also receive a fifth message from the second electronic device; wherein the fifth message is used to indicate the fifth user interface of the application running on the second electronic device, and the fifth user interface is generated after the second electronic device performs the fourth operation on the second user interface; thereafter, the first electronic device displays the fifth part of the content of the sixth user interface; wherein the sixth user interface contains the content of part of the interface area or the content of the entire interface area of ​​the fifth user interface; the first electronic device translates the sixth user interface in response to the first user's second operation on the fifth part of the content to display the sixth part of the content of the sixth user interface; wherein the fifth part of the content is different from the sixth part of the content.

[0026] Through this design, when the second electronic device executes the operation instruction corresponding to the fourth operation to update the user interface of the application, the second electronic device can also notify the first electronic device of the updated user interface so that the first electronic device can synchronously display the new user interface.

[0027] In one possible design, the size of the second user interface is larger than the size of the first display area in the display screen of the first electronic device, and the first display area is the area used by the first electronic device to display the user interface; the first user interface includes the content of the first interface area in the second user interface; based on this, the first electronic device can display the first part of the content of the first user interface in the first display part of the first display area; wherein the size of the first display part is smaller than the size of the first user interface; the first electronic device can translate the first user interface within the first display part in response to the second operation of the first user on the first part of the content in the first display part.

[0028] Through this design, the first electronic device can display the first user interface in the first display part within the first display area of ​​the display screen.

[0029] In one possible design, after the first electronic device receives the first message from the second electronic device, it can also display part or all of the content of the seventh user interface in the second display part of the first display area; wherein the seventh user interface includes the content of the second interface area in the second user interface, and the second interface area is different from the first interface area.

[0030] Through this design, the first electronic device can respectively display user interfaces of different interface areas including the second user interface in different display parts within the first display area to improve user experience.

[0031] In a possible design, the first interface area or the second interface area is at least one of the following items in the second user interface: a title bar, a menu bar, a toolbar, an address bar, a work area, a status bar, or a scroll bar.

[0032] Through this design, the first electronic device can display different functional areas of the second user interface in different display parts within the first display area.

[0033] In one possible design, when the area of ​​the first electronic device used to display the user interface is adjusted to the second display area in the display screen of the first electronic device, the first electronic device can also display the seventh part of the content of the first user interface in the third display part in the second display area; wherein the size of the second user interface is larger than the size of the second display area, and the size of the third display part is smaller than the size of the first user interface; the first electronic device translates the first user interface in the third display part in response to the first user's second operation on the seventh part of the content in the third display part to display the eighth part of the content of the first user interface; wherein the seventh part of the content is different from the eighth part of the content; and the first electronic device displays part or all of the content of the seventh user interface in the fourth display part in the second display area.

[0034] With this design, when the display area of ​​the first electronic device is adjusted, the first electronic device can use the same display rule to display the user interface of the application in different display areas.

[0035] In one possible design, the first electronic device receives interface segmentation information from the second electronic device, and determines the first interface area or the second interface area in the second user interface based on the interface segmentation information; wherein the interface segmentation information is used to indicate the location of the first interface area or the second interface area; or the first electronic device performs image recognition on the second user interface to determine the first interface area or the second interface area.

[0036] Through this design, the first electronic device can identify different interface areas in the second user interface in a variety of ways to generate different user interfaces for display.

[0037] In one possible design, the second user interface is generated by the second electronic device on a virtual screen.

[0038] In a possible design, the size of the virtual screen is larger than the size of the display screen of the second electronic device; the size of the virtual screen is larger than the size of the display screen of the first electronic device.

[0039] In one possible design, the second operation is a sliding operation.

[0040] In a possible design, the performance of the second electronic device is better than that of the first electronic device. Through this solution, the first electronic device with poor performance can control the application of the second electronic device with better performance in a distributed manner. Furthermore, the first electronic device with poor performance can also realize the functions of the second electronic device with better performance, and the functional coordination or functional expansion between electronic devices with different performance can be realized without optimizing the software and hardware resources of the first electronic device.

[0041] In one possible design, the first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; and the second electronic device is a smart phone.

[0042] In a second aspect, an embodiment of the present application provides a function collaboration method. The method can be applied to a second electronic device, where an application is installed. The method includes:

[0043] The second electronic device receives an operation instruction from the first electronic device; the operation instruction is sent by the first electronic device in response to a first operation of an application control on the first electronic device by a first user; the second electronic device runs the application corresponding to the application control according to the operation instruction; the second electronic device sends a first message to the first electronic device according to the second user interface of the application, so that the first electronic device displays a first part of the content of the first user interface; wherein the first message is used to indicate the second user interface; the first user interface contains the content of a partial interface area or the content of the entire interface area of ​​the second user interface; when the second electronic device receives the second message from the first electronic device, it generates a third user interface of the application according to the second message, and sends a third message to the first electronic device; wherein the second message is used to indicate that the second user interface has been slid, and the third message is used to indicate the third user interface.

[0044] In one possible design, the second message includes sliding direction information of the second user interface.

[0045] In one possible design, when the second electronic device receives a fourth message from the first electronic device, it generates a fifth user interface of the application based on the fourth message, and sends a fifth message to the first electronic device; wherein the fourth message is used to indicate a fourth operation, and the fourth operation is a control operation of the first user on the first user interface; and the fifth message is used to indicate the fifth user interface.

[0046] In a possible design, the fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time;

[0047] The operation position information includes a first operation position or a second operation position; the first operation position is a position where the fourth operation is mapped to the second user interface; and the second operation position is a position where the fourth operation is in the first user interface.

[0048] In a possible design, when the operation position information includes the second operation position, the operation position information also includes relative position information of the first user interface with respect to the second user interface.

[0049] In one possible design, the first user interface includes the content of the first interface area in the second user interface; the second electronic device may also send interface segmentation information to the first electronic device, where the interface segmentation information is used to indicate the first interface area in the second user interface.

[0050] In a possible design, the first interface area is at least one of the following in the second user interface:

[0051] Title bar, menu bar, toolbar, address bar, workspace, status bar, or scroll bar.

[0052] In one possible design, the second electronic device may generate the second user interface on a virtual screen; and the second electronic device generates the third user interface on the virtual screen.

[0053] In one possible design, the size of the virtual screen is larger than the size of the display screen of the second electronic device.

[0054] In one possible design, performance of the second electronic device is better than performance of the first electronic device.

[0055] In one possible design, the first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; and the second electronic device is a smart phone.

[0056] In a third aspect, an embodiment of the present application provides a functional collaboration method. The method can be applied to a communication system consisting of a first electronic device and a second electronic device, wherein the first electronic device is installed with an application control, and the second electronic device is installed with an application corresponding to the application control. The method comprises the following steps:

[0057] The first electronic device sends a run instruction to the second electronic device in response to a first operation of the application control on the first electronic device by a first user; the run instruction is used to instruct the second electronic device to run the application; the second electronic device receives the run instruction from the first electronic device; the second electronic device runs the application according to the run instruction; the first electronic device displays a first part of the content of a first user interface; wherein the first user interface is the user interface of the application; the first electronic device translates the first user interface in response to a second operation of the first user on the first part of the content to display a second part of the content of the first user interface; wherein the second part of the content is different from the first part of the content.

[0058] In a fourth aspect, an embodiment of the present application provides a first electronic device, comprising a unit for executing each step in the method provided in the first aspect above. Optionally, the first electronic device comprises a display screen, a communication unit, and a processing unit; wherein the display screen is used to display a user interface; the communication unit is used to receive and send data; and the processing unit is used to execute the method provided in the first aspect.

[0059] In a fifth aspect, an embodiment of the present application provides a second electronic device, comprising a unit for executing each step in the method provided in the second aspect above. Optionally, the second electronic device comprises a communication unit and a processing unit; wherein the communication unit is used to receive and send data; and the processing unit is used to execute the method provided in the second aspect. Exemplarily, the second electronic device may also include a display screen for displaying a user interface.

[0060] In a sixth aspect, an embodiment of the present application provides a first electronic device, which includes a display screen, at least one processor and a memory; the at least one processor is coupled to the memory, and the at least one processor is used to read the computer program stored in the memory to execute the method provided in the first aspect.

[0061] In a seventh aspect, an embodiment of the present application provides a second electronic device, the second electronic device comprising at least one processor and a memory; the at least one processor is coupled to the memory, and the at least one processor is used to read a computer program stored in the memory to execute the method provided in the second aspect. Optionally, the second electronic device also includes a display screen.

[0062] The computer program code includes computer instructions.

[0063] In an eighth aspect, an embodiment of the present application provides a chip, comprising a processor and a memory, wherein the processor is coupled to the memory, and the processor is used to call a computer program stored in the memory during runtime to implement the method provided in the first or second aspect above.

[0064] In a ninth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is run on an electronic device, the computer executes the method provided in the first or second aspect above.

[0065] In the tenth aspect, an embodiment of the present application provides a computer program product, which includes a computer program; when the computer program is run on a computer, the computer executes the method provided in the first aspect or the second aspect.

[0066] The technical effects that can be achieved in any of the second to tenth aspects mentioned above can refer to the description of the technical effects that can be achieved by any possible design in the first aspect mentioned above, and the repetitions will not be discussed here. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1A An architecture diagram of a communication system provided in an embodiment of the present application;

[0068] Figure 1B A schematic diagram of a communication scenario provided in an embodiment of the present application;

[0069] Figure 2A A structural diagram of an electronic device provided in an embodiment of the present application;

[0070] Figure 2B A schematic diagram of a display mode of an electronic device provided in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0072] Figure 4 A flow chart of a functional collaboration method provided in an embodiment of the present application;

[0073] Figure 5 A flow chart of a functional collaboration method provided in an embodiment of the present application;

[0074] Figure 6 A flowchart of a functional collaboration method provided in an embodiment of the present application;

[0075] Fig. 7A An example diagram of a common user interface displayed in a smart phone provided in an embodiment of the present application;

[0076] Figure 7B An example diagram of a user interface displayed in a smart watch provided in an embodiment of the present application;

[0077] Figure 7C Another example diagram of a user interface displayed in a smart watch provided in an embodiment of the present application;

[0078] Fig.7D An example diagram of another user interface displayed in a smart watch provided in an embodiment of the present application;

[0079] Fig. 7E An example diagram of a user interface displayed in a smart watch provided in an embodiment of the present application;

[0080] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application;

[0081] Fig. 9 A structural diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The present application provides a functional collaboration method and an electronic device, which are used to extend the functions of one electronic device to another electronic device, thereby improving the user experience. The method and the electronic device are based on the same technical concept. Since the principles of the method and the electronic device to solve the problem are similar, the implementation of the electronic device and the method can refer to each other, and the repeated parts will not be repeated.

[0083] Some terms in this application are explained below to facilitate understanding by those skilled in the art.

[0084] 1) Electronic equipment refers to equipment or devices with data connectivity, data calculation and processing functions. Generally, electronic equipment has a display screen.

[0085] For example, the electronic device in the present application may be a personal computer (PC), a laptop, a computer, a netbook, a tablet computer, a car computer, a smart phone, a smart wearable device (such as a smart watch, a smart bracelet, smart glasses, etc.), etc. Of course, the electronic device may also be various portable devices, car terminal devices, personal digital assistants (PDA), smart home devices (such as smart TVs, etc.), and other smart devices. The present application does not limit the specific form of the electronic device.

[0086] The electronic device can realize its functions through the operating system installed and provide services to the user. For example, the electronic device can be equipped with, but not limited to, Or other operating systems.

[0087] In some scenarios, electronic devices can be divided into the following two categories according to their performance:

[0088] Category I electronic devices: have better performance than Category II electronic devices and have more abundant software and hardware resources. Based on these abundant software and hardware resources, they can provide users with more and better services.

[0089] Optionally, the performance of the first category electronic device is better than that of the second category electronic device, which may include but is not limited to: the storage performance, processor performance or other component performance of the first category electronic device is better than that of the second category electronic device, or the overall performance of the first category electronic device is better than that of the second category electronic device.

[0090] For example, such electronic devices may have at least one of the following features: higher processing speed and computing power of the processor, a larger screen, higher battery capacity, and larger memory storage space.

[0091] Category II electronic devices: their performance is inferior to that of Category I electronic devices, and their software and hardware resources are not as rich as those of Category I electronic devices. Due to the constraints of various software and hardware resources, the services they provide to users are also limited.

[0092] Based on the above description, the first type of electronic device can also be called a capability-rich device, a smart-rich device or a rich device. This application will hereinafter refer to the first type of electronic device as a rich device.

[0093] The second type of electronic device can also be called a light (thin) capability device, a light (thin) intelligent device or a light (thin) device. In the following of this application, the first type of electronic device is referred to as a light device.

[0094] It should be noted that the above classification definition for electronic devices is a relative classification of two electronic devices and does not belong to fixed attributes of electronic devices. For example, electronic device 1 may be a first-class electronic device relative to electronic device 2, and may be a second-class electronic device relative to electronic device 3.

[0095] Exemplarily, rich devices may be PCs, laptops, tablets, smart phones, in-vehicle computers, etc. Light devices may be smart wearable devices, smart home devices, such as smart watches, smart bracelets, smart glasses, in-vehicle terminals, etc.

[0096] 2) The virtual screen is proposed relative to the real screen (i.e., display screen) of the electronic device, and is a virtual component created by the electronic device in the memory. The virtual screen is not affected by the display screen size and display resolution of the electronic device. That is, the electronic device can create any size and any number of virtual screens in the memory, and then the electronic device can generate a user interface for each virtual screen (i.e., create / render the user interface on the virtual screen). The electronic device can display part or all of the user interface on a virtual screen on the display screen according to display requirements or user instructions, and can also project the user interface in the virtual screen to other electronic devices for display through screen projection technology.

[0097] 3) Functional collaboration, that is, one electronic device can have the functions of another electronic device. Specifically, an application in one electronic device can display user interface and other information on another electronic device (the other electronic device may not have the application installed), and can also be reversely controlled by the other electronic device to achieve functional expansion and application flow, and ultimately achieve functional collaboration.

[0098] Optionally, in the embodiment of the present application, an electronic device that does not have an application installed but can display and / or control an application on another electronic device can be recorded as a "cooperative electronic device". Then an electronic device that has an application installed and the functions of the application are extended to other electronic devices can be recorded as a "cooperative electronic device".

[0099] 4) Interface layout strategy, which is the display layout rules of the user interface.

[0100] Optionally, in the functional collaboration scheme provided in the embodiment of the present application, the interface layout strategy may include the following contents: dividing the original user interface of the application generated by the collaborative electronic device into multiple interface areas according to set rules, and dividing the display area for displaying the user interface in the display screen of the collaborative electronic device into multiple display parts; the multiple interface areas correspond one-to-one to the multiple display parts, and each display part is used to display the corresponding interface area in a set manner.

[0101] For example, the display portion 1 may display the entire content of the corresponding interface area 1 (optionally, when the size of the interface area 1 is larger than the size of the display portion 1 , the electronic device may display the scaled interface area 1 within the display portion 1 ).

[0102] For another example, when the size of the interface area 2 is larger than the corresponding display part 2, the electronic device may display part of the content of the interface area 2 in the display part 2 (that is, the electronic device may not perform scaling processing on the interface area 2).

[0103] It should be noted that, in order to ensure standardization and consistency, the interface layout strategy of the user interface of an application that needs to be displayed by a collaborative electronic device is generally fixed and is generally set by the manufacturer of the electronic device or the developer of the application.

[0104] 5) Applications are used to be installed in electronic devices and can provide services to users after running. Applications can also be called application software or applications (APP). In the subsequent description, application and application are equivalent and can be replaced with each other.

[0105] The application program may be developed by the manufacturer of the electronic device, or by the supplier of the operating system of the electronic device, or by a third-party application manufacturer, and this application does not limit this.

[0106] For example, the application programs may include: calling application, camera application, text messaging application, chat application, video application, music application, electronic map application, social application, etc.

[0107] 6) Application control, which is a control component installed in an electronic device and encapsulated with data and / or programs, and used to start certain functions or applications of the electronic device. Optionally, in the user interface, the application control can also be presented as a virtual button or icon. In the embodiment of the present application, the application control can be referred to as a "control".

[0108] 7) Screen projection technology is used to achieve screen collaboration between multiple electronic devices. Screen collaboration is a mode of interaction between multiple electronic devices. Common examples include multi-screen interactions between smartphones, tablets, laptops, projectors, etc.

[0109] With this screen projection technology, a screen projection channel can be established between electronic device A and electronic device B. Then, electronic device A can project user interfaces, audio files, video files, pictures and other contents to electronic device B through the screen projection channel, so that electronic device B can play or display the contents projected by electronic device A.

[0110] Exemplarily, currently provided screen projection technologies may include: ScreenCast, Miracast, airplay, hiplay, digital living network alliance (DLNA), etc.

[0111] It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0112] In the embodiments of the present application, descriptions such as "when...", "in the case of...", "if" and "if" all mean that the electronic device will make corresponding processing under certain objective circumstances. It does not limit the time, nor does it require the electronic device to make a judgment when implementing it, nor does it mean that there are other limitations.

[0113] It should be noted that in this application, "used for indication" can include being used for direct indication and being used for indirect indication. When describing that a certain information is used to indicate A, it can include that the information directly indicates A or indirectly indicates A, but it does not mean that the information must carry A. Taking the first information used to indicate the first content as an example, the first information can include the first content, and can also include a part of the first content or an identifier or index of the first content, and can also include an algorithm for determining the first content, a calculation parameter, etc. This application does not limit the method of "indication".

[0114] In the present application, "including / comprising A" may be equivalent to "including / comprising A information". A information is used to indicate A.

[0115] In addition, it should be understood that, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0116] The solution provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0117] The functional collaboration solution provided in the embodiment of the present application can be applied to a communication system in which two electronic devices establish a communication connection. The following is an example of a communication system including a first electronic device and a second electronic device. Figure 1A The first electronic device is a function-cooperating electronic device, and the second electronic device is a function-cooperating electronic device.

[0118] Optionally, the communication connection between the first electronic device and the second electronic device may be a wireless communication connection or a wired communication connection, which is not limited in the present application. Figure 1A The wireless communication connection is taken as an example for exemplary description.

[0119] In one embodiment, the second electronic device can act as a master device and actively initiate a connection establishment process through short-range wireless communication technology to establish a wireless communication connection with the first electronic device, complete the transmission of relevant data when implementing the functional collaboration method provided in the embodiment of the present application, and realize the control and management of the first electronic device. Exemplarily, the connection establishment process may include: device search, pairing, connection establishment and other processes. In addition, the second electronic device as a master device can also be responsible for providing a clock synchronization signal.

[0120] Optionally, the short-range wireless communication technology may include but is not limited to: radio frequency identification (RFID) technology, Bluetooth technology, wireless-fidelity (WI-FI) technology, ZigBee technology, ultra-wideband (UWB) technology, near field communication (NFC) technology, as well as communication technologies evolved based on the above communication technologies, and communication technologies that have the same or similar functions as the above communication technologies and can replace each other.

[0121] For example, Figure 1B As shown, the smart phone can establish a Bluetooth connection with the smart watch through Bluetooth technology, and realize control and management of the smart watch and business communication through the Bluetooth connection.

[0122] In another embodiment, the first electronic device and the second electronic device can be connected to the same wireless local area network (WLAN) through short-range wireless communication technology. In this way, the first electronic device and the second electronic device can establish a wireless communication connection through the wireless local area network and complete the transmission of relevant data when implementing the functional collaboration method provided in the embodiment of the present application.

[0123] For example, the communication system may also include a WI-FI access point. Two electronic devices may first access the WI-FI access point according to the WI-FI technology, and then the WI-FI access point may connect the two electronic devices to the wireless local area network.

[0124] For another example, two electronic devices can be connected to the same Bluetooth beacon via Bluetooth technology, so that the two can establish a wireless communication connection via Bluetooth technology.

[0125] Optionally, in this communication system, after the first electronic device establishes a wireless communication connection with the second electronic device, the two can establish a traditional screen projection channel based on the wireless communication connection, so that the second electronic device can send screen projection data to the first electronic device to realize the traditional screen collaboration function.

[0126] It should also be noted that the present application does not limit the connection method between the first electronic device and the second electronic device. For example, the first electronic device and the second electronic device may also establish a physical communication connection through a cable.

[0127] In such Figure 1A In the communication system shown, in the scenario where the second electronic device is a rich device and the first electronic device is a light device, the second electronic device can provide users with a variety of rich functions due to its rich software and hardware resource support. However, since the first electronic device is constrained by various software and hardware resources, the functions provided by the first electronic device are also limited.

[0128] Continue with Figure 1B The communication system composed of the smart watch and the smart phone shown in the figure is used as an example. A variety of applications can be installed in the smart phone, but the smart watch may not be able to install some applications due to the performance of the processor, such as weather software, chat software, search engine software, wait.

[0129] In addition, even if smart watches can install applications, the services provided by smart watch applications are greatly reduced compared to the applications of the same name installed on smartphones. The application does not support sending red envelopes, real-time voice / video calls, or opening mini-programs, etc. For another example, the navigation function of the map application installed in the smart watch cannot display the electronic map, but only displays an arrow to indicate the direction of travel.

[0130] In order to expand the functions of light devices, the following solutions are currently provided: 1. Forcibly install a mobile phone operating system compatible with the smartphone in the smart watch so that applications compatible with the smartphone can be installed in the smart watch; 2. Based on the feature ability (FA) programming framework, the important information or operations of the smartphone are pre-placed on the card through the atomic service card flow architecture, and the card is displayed on the smart watch, and the background application logic of the card is executed on the smartphone.

[0131] However, due to the huge differences between mobile phone operating systems and watch operating systems in terms of power management, background task scheduling, etc., solution one will result in higher power consumption, larger memory usage, shorter standby time, and poor user experience for smart watches. Solution two also requires developers to design cards based on the display of smart watches. Therefore, the display of smart watches will also impose greater restrictions and constraints on the content displayed on the cards. Therefore, even with solution two, smart watches still cannot realize some functions.

[0132] In summary, how to expand the function of one electronic device to another electronic device without affecting the performance of the electronic device is an issue that technicians in this field need to consider.

[0133] Based on this, the embodiment of the present application provides a function collaboration method and an electronic device. In this method, the first electronic device is installed with an application control, and the second electronic device is installed with an application corresponding to the application control; when the user performs an operation on the application control on the first electronic device, the first electronic device sends a run instruction to the second electronic device to instruct the second electronic device to run the application; and the first electronic device can display part of the user interface of the application; and as the user operates the part of the content of the currently displayed user interface, the first electronic device can translate the user interface to display another part of the content of the user interface. Through this method, with the assistance of the second electronic device, the first electronic device can display the user interface of the application of the second electronic device, that is, realize the function of the application in the second electronic device. Based on this scheme, the function of the second electronic device can be extended to the first electronic device to realize the functional collaboration of the two electronic devices. In summary, the scheme provided by the present application can realize the functional collaboration and functional expansion of the electronic device without considering the software and hardware resource constraints of the electronic device, and avoid causing large power consumption and memory overhead to the electronic device. Further, the first electronic device can display part of the user interface content of the application, and the user can translate the user interface by operating the user interface, so that the user interface of the application displayed in the first electronic device can display more, richer, more effective or more detailed content, and the user experience is higher. In addition, this method does not require frequent communication interactions between electronic devices. The first electronic device can switch the displayed content through local operations. Therefore, this method can avoid power consumption overhead and operation delays caused by frequent interactions between devices.

[0134] Continue with Figure 1B The communication system composed of the smart watch and the smart phone shown in the figure is used as an example. Based on the solution provided by this application, when the smart watch is installed In the case of a control, the smart watch can run And update synchronously real-time user interface. When opening functions such as sending packages, real-time voice / video calls, and mini-programs, the corresponding user interfaces of these functions can also be synchronized to the smart watch, allowing the smart watch to also realize corresponding functions to achieve functional synergy.

[0135] The present application embodiment provides an electronic device, which can form a communication system with other electronic devices to implement the functional coordination method provided by the present application embodiment. Figure 2A , the structure of the electronic device provided in the embodiments of the present application is introduced.

[0136] like Figure 2A As shown, the electronic device 200 may include: a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a SIM card interface 295, etc.

[0137] The processor 210 may include one or more processing units, for example: the processor 210 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. Among them, the controller can be the nerve center and command center of the electronic device 200. The controller can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0138] A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. By providing the memory, the number of times the processor 210 accesses data to the internal memory 221 may be reduced, the waiting time of the processor 210 may be reduced, and thus the efficiency of the system may be improved.

[0139] The functional coordination method provided in the embodiment of the present application can be controlled by the processor 210 or called by other components to complete, such as calling the software program of the embodiment of the present application stored in the internal memory 221 to control the wireless communication module 260 to communicate data with other electronic devices, and control the display screen 294 to display the user interface. The processor 210 may include different devices, such as when the CPU and GPU are integrated, the CPU and GPU can cooperate to execute the functional coordination method provided in the embodiment of the present application, such as part of the algorithm in the functional coordination method is executed by the CPU, and another part of the algorithm for displaying the user interface is executed by the GPU, so as to obtain faster processing efficiency.

[0140] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 200 may include 1 or N display screens 294, where N is a positive integer greater than 1. The display screen 294 can be used to display information input by a user or information provided to a user and various graphical user interfaces (GUIs). For example, the display screen 294 can display windows, photos, videos, web pages, or files, etc. In addition, it can be understood that in some embodiments, the status bar may also include a Bluetooth icon, a Wi-Fi icon, an external device icon, etc. In some embodiments, when the processor 210 detects a touch operation of a user's finger (or a stylus, etc.) on an application icon, in response to the touch operation, the user interface corresponding to the application icon is opened and displayed on the display screen 294. In some embodiments, when the processor 210 detects a click operation of an external input device (such as a mouse, keyboard, etc.) of the electronic device 200 on an application icon, in response to the click operation, a corresponding control operation is performed on the application.

[0141] In the embodiment of the present application, the display screen 294 can be an integrated flexible display screen, or a spliced ​​display screen composed of two or more rigid screens and a flexible screen located between two adjacent rigid screens. After the processor 210 runs the functional coordination method provided in the embodiment of the present application, the processor 210 can control the display screen 294 to display the user interface of the application.

[0142] In some embodiments, part or all of the display area of ​​the display screen 294 can be used to display a user interface. For example, the electronic device 200 can have multiple display modes, and each display mode has a corresponding display area. The shape or size of the display area corresponding to different display modes may be different. It should be noted that the display areas corresponding to different display modes may overlap.

[0143] Take a smart watch as an example. Figure 2B As shown, the display screen of the smart watch is square. In the first display mode, the smart watch displays the user interface through the first display area in the display screen; in the second display mode, the smart watch displays the user interface through the second display area in the display screen. The first display area is square, consistent with the size and shape of the display screen; the second display area is circular. The first display area and the second display area are different in size and / or shape.

[0144] Of course, the electronic device 200 may also have only one display mode. In this case, the electronic device 200 may display the user interface through the entire area of ​​the display screen 294 , or may display the user interface through a partial area of ​​the display screen 294 .

[0145] The camera 293 is used to capture still images or videos. Optionally, the electronic device 200 may include 1 to N cameras 293. For example, the camera 293 may include a front camera or a rear camera, or a camera 293 may be used as both a front camera and a rear camera. Typically, the camera 293 may include a photosensitive element such as a lens group and an image sensor. Among them, the lens group includes a plurality of lenses (convex lenses or concave lenses) for collecting light signals reflected by the object to be photographed, and transmitting the collected light signals to the image sensor. The image sensor generates an original image of the object to be photographed based on the light signal.

[0146] The internal memory 221 can be used to store computer executable program codes, which include instructions. The processor 210 executes various functional applications and data processing of the electronic device 200 by running the instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. The program storage area may store an operating system, a program code of at least one application, etc. The operating system may include, but is not limited to, The data storage area can store data created during the use of the electronic device 200 .

[0147] The internal memory 221 may also store one or more computer programs for executing the functional collaboration method provided in the embodiment of the present application. The one or more computer programs are stored in the internal memory 221 and configured to be executed by one or more processors 210. The one or more computer programs include instructions, which may be used to execute the various steps in the following embodiments.

[0148] In addition, the internal memory 221 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0149] The sensor module 280 may include a fingerprint sensor, a touch sensor, a pressure sensor, a magnetic sensor, an ambient light sensor, an air pressure sensor, a bone conduction sensor, and the like.

[0150] Touch sensor, also known as "touch panel". The touch sensor can be set on the display screen 294. The touch sensor is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 294. In other embodiments, the touch sensor can also be set on the surface of the electronic device 200, which is different from the position of the display screen 294.

[0151] The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor and the baseband processor.

[0152] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0153] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 200. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the same device as at least some of the modules of the processor 210.

[0154] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 270A, a receiver 270B, etc.), or displays an image or video through a display screen 294. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 210 and be set in the same device as the mobile communication module 250 or other functional modules.

[0155] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied to the electronic device 200. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2. In the embodiment of the present application, the electronic device 200 can establish a wireless communication connection with other electronic devices through the wireless communication module 260 to exchange data, or the electronic device 200 can access the access point through the wireless communication module 260, so that instructions and data can be sent to the network device or other electronic devices through the access point. For example, the electronic device 200 can establish a functional collaboration channel with other electronic devices through the wireless communication module 260, and receive user interface data from other electronic devices through the wireless communication module 260, and send control operation information to the other electronic devices; or send user interface data to other electronic devices through the wireless communication module 260, and receive control operation information from other electronic devices.

[0156] In addition, the electronic device 200 can implement audio functions through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor. For example, music playback, recording, etc. The electronic device 200 can receive the key 290 input and generate key signal input related to the user settings and function control of the electronic device 200. The electronic device 200 can generate a vibration prompt using the motor 291. The indicator 292 in the electronic device 200 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 295 in the electronic device 200 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 295, or pulled out from the SIM card interface 295, to achieve contact and separation with the electronic device 200.

[0157] It should be understood that in practical applications, Figure 2A The electronic device 200 shown is only an example and does not constitute a limitation of the electronic device. In addition, the electronic device 200 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0158] It should also be noted that in the embodiments provided in this application, the structures of the first electronic device and the second electronic device can refer to Figure 2A In addition, the structures of the first electronic device and the second electronic device may be different. Figure 2A More or fewer components, or different component configurations are shown, and this application will not repeat them. In addition, even if the two have the same components, the same components in the two electronic devices may also have performance differences.

[0159] The software system of the electronic device 200 provided in the embodiment of the present application can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device. Figure 3 shown.

[0160] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. Figure 3As shown, the software architecture can be divided into four layers, from top to bottom, namely, application layer, application framework layer (framework, FWK), Android runtime (Android runtime) and system library, and kernel layer. In addition, the electronic device may also include a hardware layer.

[0161] The application layer is the top layer of the operating system, including native applications of the operating system, as well as third-party applications such as camera, gallery, calendar, Bluetooth, music, video, information, etc.

[0162] In an embodiment of the present application, the application layer may also include a functional collaboration service application, which is used to implement the functional collaboration method provided in the embodiment of the present application, extend the service functions provided by the electronic device to another electronic device, or extend the functions of other electronic devices to the local area, thereby achieving functional collaboration and functional expansion.

[0163] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer may include some predefined functions. The application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0164] The window manager is used to provide window management services. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The window manager can provide multiple management functions to control the transparency, position, size and other parameters of the window or interface on the display screen.

[0165] Content providers are used to store and retrieve data and make these data accessible to applications. The data may include files (such as documents, videos, images, audio), text and other information.

[0166] The view system includes visual controls, such as controls for displaying text, images, documents, etc. The view system can be used to build applications. The interface in the display window can be composed of one or more controls. For example, a display interface showing a file icon can include controls for displaying text and controls for displaying images.

[0167] The phone manager is used to provide the communication function of the terminal device. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and disappear automatically after a short stay without user interaction.

[0168] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0169] The core library of the Android system consists of two parts: one is the function that the Java language needs to call, and the other is the core library of the Android system. The application layer and the application framework layer run in the virtual machine. Taking Java as an example, the virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0170] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0171] The kernel layer provides the core system services of the operating system, such as security, memory management, process management, network protocol stack and driver model, which are all implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer has many drivers related to electronic devices, the main drivers are: display driver; keyboard driver as input device; Flash driver based on memory technology device; camera driver; audio driver; Bluetooth driver; WiFi driver, etc.

[0172] In some embodiments, the kernel layer may also include an input driver service for obtaining and reporting operation information performed on an interface or window displayed on the display screen from an external input device (such as a mouse, keyboard, touchpad, etc.), a sensor (such as a touch screen, touch sensor, etc.) or other electronic device.

[0173] The hardware layer can include various sensors, display screens, cameras, etc.

[0174] In order to extend the function of one electronic device to another electronic device, the embodiment of the present application provides a function collaboration method. The method is described below with reference to the accompanying drawings. The method can be applied to Figure 1A The communication system shown includes a first electronic device and a second electronic device, for example Figure 1BApplication scenario shown. In an embodiment of the present application, an application is installed in the second electronic device, and an application control corresponding to the application is installed in the first electronic device, that is, the application control in the first electronic device and the application in the second electronic device have an association relationship (or binding relationship). Optionally, the icon of the application control can be the same as the icon of the application on the second electronic device.

[0175] In one implementation, the application control can be created after the user performs a setting operation on the second electronic device, triggering the second electronic device to interact with the first electronic device.

[0176] In another embodiment, the application control can be generated by the user performing a setting operation on the first electronic device, or can be preset by the manufacturer of the first electronic device or the operating system of the first electronic device. After the first electronic device establishes a communication connection with the second electronic device, the first electronic device can interact with the second electronic device to associate the locally set application control with the application in the second electronic device.

[0177] It should also be noted that there may be multiple application controls in the first electronic device, and the applications corresponding to these application controls may be installed in one or more second electronic devices. Each application control in the first electronic device is associated with an application in the second electronic device. The embodiment of the present application is described by taking an application control of the first electronic device as an example.

[0178] Before introducing the solutions provided by the embodiments of the present application, some definitions involved in the embodiments of the present application are first explained.

[0179] The first electronic device is an electronic device for distributedly controlling an application installed on the first electronic device. The first electronic device is installed with a control of the application. The first electronic device can also be called a function-cooperative electronic device.

[0180] The second electronic device is an electronic device installed with an application program, and the application program can be distributedly controlled by the first electronic device. The second electronic device can be called a function-coordinated electronic device.

[0181] A first user is a user who uses and operates the first electronic device.

[0182] The second user is a user who uses and operates the second electronic device.

[0183] It should be noted that the first user and the second user may be the same user or different users, and this application does not limit this.

[0184] The first user interface is the user interface of the application displayed by the first electronic device after the first electronic device runs the application of the second electronic device through the application control.

[0185] The second user interface is a user interface of the application generated after the second electronic device runs the application after receiving the running instruction from the first electronic device.

[0186] The third user interface is a user interface of the application generated after the second electronic device learns through a message sent by the first electronic device that the second user interface has been slid.

[0187] The fourth user interface is the user interface of the application updated by the first electronic device after the second electronic device notifies the first electronic device of the third user interface.

[0188] The fifth user interface is a user interface of an application generated after the second electronic device learns, through a message sent by the first electronic device, that the user has performed a control operation on the first user interface displayed by the first electronic device.

[0189] The sixth user interface is the user interface of the application updated by the first electronic device after the second electronic device notifies the first electronic device of the fifth user interface.

[0190] It should be noted that the first user interface may include the content of a partial interface area or the content of the entire interface area of ​​the second user interface. In the scenario where the first user interface includes the content of a partial interface area (e.g., the first interface area) of the second user interface, the first electronic device may also display the seventh user interface of the application. The seventh user interface includes the content of other partial interface areas (e.g., the second interface area) of the second user interface. Obviously, the first user interface and the seventh user interface include different contents.

[0191] In order for the first electronic device to display the first user interface and the seventh user interface, the display area of ​​the display screen of the first electronic device for displaying the user interface can be divided into multiple display parts, each display part is used to display one user interface. For example, the display area of ​​the first electronic device can be divided into two display parts. In this way, one display part can display the first user interface, and the other display part can display the seventh user interface.

[0192] Similar to the first user interface and the second user interface, the fourth user interface may include the content of a portion of the interface area or the content of the entire interface area of ​​the third user interface, and the sixth user interface may include the content of a portion of the interface area or the content of the entire interface area of ​​the fifth user interface. In addition, the first electronic device may also display the fourth user interface and other user interfaces in multiple display parts within the display area (in this case, the other user interfaces include the content of other portions of the interface area of ​​the third user interface except the portion of the interface area included in the fourth user interface). The first electronic device may also display the sixth user interface and other user interfaces in multiple display parts of the display area content (in this case, the other user interfaces include the content of other portions of the interface area of ​​the fifth user interface except the portion of the interface area included in the sixth user interface).

[0193] The first display area of ​​the first electronic device is an area in the display screen of the first electronic device used to display a user interface.

[0194] The second display area of ​​the first electronic device is an area in the display screen of the first electronic device used to display the user interface.

[0195] Optionally, the first display area and the second display area can be divided into multiple display parts for displaying the content of different interface areas in the user interface of the application generated by the second electronic device. Optionally, the sizes and / or shapes of different display parts divided from a display area can be different.

[0196] It should be noted that, when the first electronic device has multiple display modes, each display mode has a corresponding display area. The shapes or sizes of the display areas corresponding to different display modes are different, such as Figure 2B The user can select the first display area or the second display area on the first electronic device to display the user interface. When the first electronic device has only one display mode, the first display area is the default area of ​​the first electronic device for displaying the user interface.

[0197] In addition, the first display area and the second display area have different sizes and / or shapes.

[0198] In the embodiment of the present application, the size may include but is not limited to: length, width, perimeter, area or radius, etc., which are parameters that can reflect the size of a plane figure.

[0199] The first operation is an operation of the first user on the application control on the first electronic device, which is used to run the application on the second electronic device. Optionally, similar to the traditional operation of starting and running a local application of the electronic device, the first operation can be any of the following operations performed by the first user on the application control: single click, double click, long press, drag, etc.

[0200] The second operation is an operation by the first user on the user interface of the application displayed on the first electronic device, which is used to translate the user interface. Optionally, the second operation can be a sliding operation by the first user, or a click operation by the first user on other blank positions in the user interface other than the display center position (a set area position including the center point of the current display area or display portion). A blank position is a position in the user interface that does not contain a virtual button, menu bar or toolbar, etc., and will not trigger other control operations on the application.

[0201] The third operation is an operation of the first user on the first user interface, which is used to zoom the first user interface. Optionally, the third operation can be the first user clicking a zoom shortcut key, or the first user touching and sliding two fingers on the first user interface towards each other at the same time.

[0202] The fourth operation is a control operation of the first user on the first user interface, which is used to perform distributed control on the application program located on the second electronic device. For example, the fourth operation can be a control operation of the first user on a virtual button, menu bar or toolbar in the first user interface.

[0203] It should be noted that the present application does not limit the specific operation methods of the above operations, which may be exemplified when introducing the embodiments of the present application later.

[0204] See below Figure 4 The flow chart shown illustrates the method provided in the embodiment of the present application.

[0205] S401: The first electronic device sends a run instruction to the second electronic device in response to a first user's first operation on an application control on the first electronic device. The run instruction is used to instruct the second electronic device to run an application corresponding to the application control. The second electronic device receives the run instruction from the first electronic device.

[0206] Optionally, the running instruction may include information of the application, for example, the running instruction may include an identifier of the application, or a running command of the application.

[0207] S402: The second electronic device runs the application according to the running instruction.

[0208] It should also be noted that in the embodiment of the present application, the first electronic device may not have the application installed. Therefore, when the first user performs the first operation on the application control on the first electronic device, the first electronic device does not run the application locally.

[0209] Optionally, in S402, the second electronic device may run the application in the following manners, but is not limited to:

[0210] Method 1: Before S402, the application is running in the background of the second electronic device and is in a dormant mode. When the second electronic device executes S402, it can wake up the application and run the application in the foreground.

[0211] Mode 2: Before S402, the application has been running in the foreground of the second electronic device. When the second electronic device executes S402, the application can continue to run in the foreground.

[0212] Mode 3: Before S402, the application is not started, and the second electronic device does not run the application in the background or foreground. When the second electronic device executes S402, it can open the application and run the application in the foreground.

[0213] Method 4: The second electronic device runs the application in the background.

[0214] Method 5: The second electronic device may run the application program through a third-party application. The third-party application may be an application with a function of functional collaboration.

[0215] It should be noted that, optionally, when the second electronic device is in a black screen state or other applications are running in the foreground, the second electronic device may not display the user interface of the application when executing S402 to run the application; or when the second electronic device is in a black screen state, the second electronic device may light up the screen and display the user interface of the application when executing S402 to run the application; or when the second electronic device is running other applications, the first electronic device may place the other application in the background when executing S402 and switch from displaying the user interface of the other application to displaying the user interface of the application. The specific implementation process is not limited in this application.

[0216] It should be noted that during the running of the application, the process list of the second electronic device includes the process of the application. The second user can perform an operation of viewing the running program on the second electronic device (such as long pressing a function key, or the second user swipes upward from the bottom of the second electronic device with a finger, or the second user swipes from the side of the second electronic device to the opposite side with a finger, etc.), and can also see the running window displaying the user interface of the application.

[0217] S403: When the second electronic device runs the application program in S402, the first electronic device may display a first portion of the content of the first user interface, where the first user interface is the user interface of the application program.

[0218] In the embodiment of the present application, the first electronic device may generate the first user interface through communication interaction with the second electronic device, and the process is as follows:

[0219] The second electronic device generates a second user interface of the application based on the running application. The second electronic device sends a first message to the first electronic device through S403a according to the second user interface of the application. The first electronic device receives the first message from the second electronic device. The first message is used to indicate the second user interface of the application running on the second electronic device. After S403a, the first electronic device generates a first user interface according to the second user interface indicated by the first message. The first user interface includes the content of a part of the interface area or the content of the whole interface area of ​​the second user interface.

[0220] It should be noted that the first user interface may also include other content in addition to the content in the second user interface, which is not limited in this application. For example, the first user interface may also include a battery icon, a message reminder icon, etc. of the first electronic device.

[0221] Optionally, in S403a, the second electronic device may send a first message to the second electronic device by using, but not limited to, screen projection technology or image transmission technology. The first message includes image data of the second user interface.

[0222] In one embodiment, the second electronic device may generate the second user interface on a virtual screen. In this case, the second electronic device may not display the second user interface on the display screen; or it may display part or all of the content of the second user interface through S403b according to the size or shape of the display area (hereinafter referred to as the third display area for ease of description) used to display the user interface on the display screen of the second electronic device, which is not limited in this application. Compared with the user interface generated by the second electronic device in a traditional manner (i.e., generating a user interface according to the settings of the display screen), the second user interface generated by the second electronic device on the virtual screen can display more, richer, more effective or more detailed content, i.e., the size of the virtual screen is larger than the size of the display screen of the second electronic device.

[0223] In another embodiment, the second electronic device may also generate and display the second user interface on the display screen (or the third display area) according to the size of the display screen (or the size of the third display area of ​​the second electronic device) and the display resolution setting.

[0224] Optionally, in an embodiment of the present application, in order to reduce frequent communication interactions between electronic devices, the first electronic device can display more, richer, more effective or more detailed content in the first user interface generated based on the second user interface, that is, the size of the second user interface can be larger than the size of the display screen of the first electronic device to ensure that the size of the first user interface generated by the first electronic device can also be larger than the size of the display screen of the first electronic device.

[0225] Based on this, the second electronic device can generate a second user interface on the virtual screen according to the size of the display screen of the first electronic device, that is, the size of the virtual screen is set according to the size of the display screen of the first electronic device. Or when the size of the display screen of the second electronic device is greater than or equal to the size of the display screen of the first electronic device, the second electronic device can generate a second user interface on a virtual super large screen, and the size of the virtual super large screen is greater than the size of the display screen of the second electronic device. For example, in a scenario where the first electronic device is a smart watch and the second electronic device is a smart phone, the aspect ratio of the virtual super large screen of the second electronic device can be more than 1.5 times the aspect ratio of the display screen of the second electronic device, that is, the length of the virtual super large screen of the second electronic device (the length of the second user interface) is more than 1.5 times the length of the display screen of the second electronic device. Of course, the width of the virtual super large screen of the second electronic device can also be greater than the width of the display screen of the second electronic device.

[0226] In addition to the above method, when the size of the display screen of the second electronic device is larger than the size of the display screen of the first electronic device, the second electronic device can also generate the second user interface according to its own display screen. In short, the embodiment of the present application does not limit the method for the second electronic device to generate the second user interface.

[0227] The size of the second user interface generated by the second electronic device is relatively large, and accordingly, the size of the first user interface generated by the first electronic device based on the second user interface is also relatively large, so the current display area of ​​the first electronic device may not be able to directly display all the contents of the first user interface. Therefore, the first electronic device can display part of the contents of the first user interface (i.e., the first part of the contents in S403).

[0228] S404: The first electronic device translates the first user interface in response to the first user's second operation on the first portion of content to display a second portion of content of the first user interface, wherein the second portion of content is different from the first portion of content.

[0229] It should be noted that in actual applications, the first user may translate the first user interface multiple times through multiple second operations.

[0230] Since the size of the first user interface is relatively large, the first electronic device can only display part of the content of the first user interface at a time. Therefore, the first user can slide the first user interface through S404 to switch the displayed content.

[0231] The solution provided by the above steps S401-S404 only requires the first electronic device to have communication and display capabilities, and there are no strict requirements on its software and hardware resources, and it will not cause large power consumption and memory overhead to the first electronic device. Therefore, this solution can extend the functions of the second electronic device to the first electronic device without being constrained by the software and hardware resources of the first electronic device, thereby realizing functional collaboration between the two electronic devices. In addition, the first electronic device can also display part of the user interface content of the collaborative application, and the user interface can be translated through simple local operations by the user, so that the user interface displayed by the first electronic device can display more, richer, more effective or more detailed content. In addition, the above process does not require frequent communication and interaction between electronic devices. The first electronic device can switch the display content through local operations, which also improves the control fluency of the first electronic device over the collaborative application, and avoids power consumption and operation delays caused by frequent interactions between devices.

[0232] It should also be noted that in the process of the above-mentioned functional collaboration, the first message exchanged between electronic devices carries the image data of the second user interface (i.e., a single frame image). Therefore, this method can reduce the encoders and transmission loads between electronic devices required in traditional functional collaboration methods, significantly reduce the power consumption of both parties of the functional collaboration, and is also beneficial for the first electronic device to collaborate with the application of the second electronic device for a long time.

[0233] In a possible implementation, during the process of the first electronic device translating the first user interface according to the operation of the first user through S404, when the first user slides to the boundary of the second user interface, the first electronic device can also notify the second electronic device of the event through S405, so that the second electronic device can perform subsequent operation steps based on the event. For example, the second electronic device can update the user interface of the application based on the event, as shown in Figures S406-S408.

[0234] S405: After the first electronic device translates the first user interface in S404, when the second portion of content displayed by the first electronic device includes the boundary of the second user interface, the first electronic device sends a second message to the second electronic device, wherein the second message is used to indicate that the second user interface has been slid.

[0235] Currently, in some scenarios, when the user interface of an application displayed locally on an electronic device is swiped, the electronic device is also required to perform subsequent operations and update the user interface. If the main page is swiped up, the phone will update the new Weibo post to the main page. When the main page is swiped upward, the phone will update the chat room records with earlier chat times to the main page.

[0236] Since the first electronic device displays part of the content of the first user interface, it is equivalent to displaying part of the content of the second user interface. Therefore, when the first user translates the first user interface through the second operation so that the part of the content of the first user interface displayed by the first electronic device includes the boundary of the second user interface, it can be represented that the first user performs a sliding operation on the second user interface on the first electronic device side. Therefore, in this case, the first electronic device can send a second message to the second electronic device to notify the second electronic device that the second user interface has been slid, so that the second electronic device can perform subsequent operation steps based on the event to update the user interface, and synchronize the update to the first electronic device through the following S406-S408.

[0237] In some embodiments, the second message in S405 may also include the sliding direction information of the second user interface. In one implementation, the first electronic device may directly determine the sliding direction according to the second operation, and the sliding direction information in the second message is an indication of the sliding direction. In another implementation, the sliding direction information in the second message includes: the position of the boundary of the second user interface contained in the second part of the content in the second user interface, or the translation direction indicated by the second operation. In this way, the second message can also indicate the direction in which the second user interface is slid on the first electronic device side through the above information, so that the second electronic device can accurately perform subsequent operation steps. For example, when the second part of the content displayed by the first electronic device includes the lower boundary of the second user interface, it is equivalent to the second user interface being slid upward; when the second part of the content includes the upper boundary of the second user interface, it is equivalent to the second user interface being slid downward. For another example, when the translation direction of the second operation indication contained in the second message is upward translation, that is, the second message indicates that the second user is slid upward.

[0238] S406: When the second electronic device receives the second message from the first electronic device, it generates a third user interface of the application according to the second message and sends the third message to the first electronic device. The first electronic device receives the third message from the second electronic device. The third message is used to indicate the third user interface.

[0239] In this embodiment, the third user interface, ie, the second electronic device, is generated based on an event in which the second user interface is slid.

[0240] It should be noted that in this embodiment, the size, generation method and transmission method of each user interface of the application generated by the second electronic device during the functional collaboration process are the same. Therefore, the description of the size, generation method and transmission method of the third user interface can refer to the description of the second user interface above and will not be repeated here.

[0241] S407a: Optionally, after generating the third user interface, the second electronic device may display part or all of the content of the third user interface.

[0242] S407: After receiving the third message, the first electronic device may display a third portion of content of a fourth user interface, wherein the fourth user interface includes content of a portion of the interface area or content of the entire interface area of ​​the third user interface.

[0243] Among them, the fourth user interface is generated by the first electronic device based on the third user interface. Similar to the first user interface, the fourth user interface may include other content in addition to the content in the third user interface, and this application does not limit this. For example, the fourth user interface may also include a battery icon, a message reminder icon, etc. of the first electronic device.

[0244] S408: The first electronic device translates the fourth user interface in response to the first user's second operation on the third portion of content to display a fourth portion of content of the fourth user interface, wherein the fourth portion of content is different from the third portion of content.

[0245] The process of translating the fourth user interface may refer to the description of translating the first user interface in S404, which will not be repeated here.

[0246] It should also be noted that, in the process of the first user translating the fourth user interface through the second operation, the first user may also slide to the boundary of the third user interface. At this time, the first electronic device can continue to notify the second electronic device of the event so that the second electronic device can continue to perform subsequent operations based on the event. For details, please refer to the above steps S405-S408.

[0247] In a possible implementation, it can be seen from the description of steps S403-S404 above that, since the size of the first user interface is relatively large, the first electronic device can display part of the content of the first user interface by default or in general. Based on this, the first user can also make the first electronic device scale the first user interface through local operations, as shown in S409, so that the first electronic device can display the entire content of the first user interface, thereby enriching the operability of the first user on the first electronic device for the first user interface, thereby improving the user experience.

[0248] S409: When the first electronic device displays partial content of the first user interface through S403 or S404, the first electronic device may also scale the first user interface in response to a third operation of the first user on the first user interface to display all content of the first user interface.

[0249] Through S409, the first electronic device can zoom in and out to display the first user interface according to the local operation of the user, so as to display the overall content of the first user interface.

[0250] Optionally, when the first electronic device displays the entire content of the first user interface, if the first electronic device detects a sliding operation of the first user interface by the first user, it may also send a second message to the second electronic device to notify the second electronic device that the second user interface has been slid so that the second electronic device can perform subsequent operation steps. The specific process can refer to S405 and subsequent steps above, which will not be repeated here. It should be noted that at this time, the second message may also include the sliding direction information of the second user interface. At this time, the sliding direction information is used to indicate the sliding direction of the sliding operation.

[0251] It should also be noted that, in a scenario where the first electronic device displays the fourth user interface, the first user can also scale the fourth user interface to display the entire content of the fourth user interface by performing a third operation on the fourth user interface.

[0252] In one possible implementation, in order to further improve the user experience, the first user can also control the first electronic device to enable the first electronic device to achieve distributed control of the application, that is, the first electronic device can transmit the first user's control operation on the first user interface on the first electronic device side back to the second electronic device to trigger the second electronic device to execute the operation instruction corresponding to the control operation. The specific process can be referred to the following description of S410-S413.

[0253] S410: When the first electronic device displays part or all of the content of the first user interface through S403, S404 or S409, in response to the first user's fourth operation on the first user interface, sending a fourth message to the second electronic device. The fourth operation is a control operation of the first user interface by the first user, and the fourth message is used to indicate the fourth operation.

[0254] Optionally, in order to enable the second electronic device to accurately execute the operation instruction corresponding to the fourth operation, the fourth message may include at least one of the following: operation location information, operation type, operation instruction, and operation time. The operation location information includes a first operation location or a second operation location. The first operation location is the location where the fourth operation is mapped to the second user interface; the second operation location is the location of the fourth operation in the first user interface.

[0255] In some schemes, the fourth message may include the first operation position. In this scheme, the first electronic device may determine the second operation position after detecting the fourth operation in the first user interface; the first electronic device may determine the first operation position based on the relative position information of the first user interface in the second user interface and the second operation position. In this way, after receiving the fourth message, the second electronic device may directly execute the operation instruction corresponding to the fourth operation on the second user interface according to the first operation position.

[0256] In some other schemes, the fourth message may include the second operation position. In this scheme, after the first electronic device detects the fourth operation in the first user interface, it can determine the second operation position, and directly send the second operation position to the second electronic device in the fourth message. After receiving the fourth message, the second electronic device can determine the first operation position based on the relative position information of the first user interface in the second user interface and the second operation position; finally, execute the operation instruction corresponding to the fourth operation on the second user interface according to the first operation position. Optionally, in this scheme, the second electronic device can determine the relative position information of the first user interface in the second user interface in the following way:

[0257] Method 1: The first electronic device and the second electronic device can maintain the same interface layout strategy for the application.

[0258] In method 1, the first electronic device can generate the first user interface according to the interface layout strategy and the second user interface; the second electronic device can also determine the relative position information of the first user interface in the second user interface according to the interface layout strategy. Optionally, the interface layout strategy in the first electronic device can be sent by the second electronic device.

[0259] Method 2: After generating the first user interface based on the second user interface, the first electronic device can send the relative position information of the first user interface in the second user interface to the second electronic device; or in S410, the operation position information contained in the fourth message sent by the first electronic device to the second electronic device can carry the relative position information.

[0260] Method 3: When the first user interface is the content of a partial interface area of ​​the second user interface, the second electronic device can send interface segmentation information to the first electronic device. The interface segmentation information is used to indicate the position of the first interface area used to segment the first user interface in the second user interface. In this case, when the second electronic device generates the interface segmentation information, it can determine the relative position information of the first user interface in the second user interface.

[0261] Through S410, the first electronic device can feed back the control operation event of the first user on the application on the first electronic device to the second electronic device, so that the second electronic device executes the corresponding operation instruction. It should be noted that if the second electronic device updates the user interface of the application after executing the operation instruction, the second electronic device and the first electronic device can continue to display the updated user interface on the first electronic device through the user interface display process such as S403a-S404, as shown in the following S411-S413.

[0262] S411: When the second electronic device receives the fourth message from the first electronic device, it generates a fifth user interface of the application according to the fourth message and sends a fifth message to the first electronic device. The first electronic device receives the fifth message from the second electronic device. The fifth message is used to indicate the fifth user interface.

[0263] In this embodiment, the fifth user interface is generated after the second electronic device performs the fourth operation on the second user interface.

[0264] It should be noted that the description of the size, generation method and transmission method of the fifth user interface can refer to the description of the second user interface above, and will not be repeated here.

[0265] S412a: Optionally, after generating the fifth user interface, the second electronic device may display part or all of the content of the fifth user interface.

[0266] S412: After receiving the fifth message, the first electronic device may display a fifth portion of content of a sixth user interface, wherein the sixth user interface includes content of a partial interface area or content of the entire interface area of ​​the fifth user interface.

[0267] Among them, the sixth user interface is generated by the first electronic device based on the fifth user interface. Similar to the first user interface, the sixth user interface may include other content in addition to the content in the fifth user interface, and this application does not limit this. For example, the sixth user interface may also include a battery icon, a message reminder icon, etc. of the first electronic device.

[0268] S413: The first electronic device translates the sixth user interface in response to the first user's second operation on the fifth portion of content to display a sixth portion of content of the sixth user interface, wherein the fifth portion of content is different from the sixth portion of content.

[0269] The process of translating the sixth user interface and performing a control operation on the sixth user interface may refer to the above detailed description of translating the first user interface and performing a control operation on the first user interface, which will not be repeated here.

[0270] Through the above description of each step in the embodiment of the present application, in the solution provided in the embodiment of the present application, the first user interface displayed by the first electronic device may include the content of part of the interface area of ​​the second user interface or the content of the entire interface area. For these two different situations, the embodiment of the present application is exemplified by the following three implementation methods.

[0271] In addition, the process of the first electronic device displaying the fourth user interface or the sixth user interface may refer to the following description on displaying the first user interface.

[0272] Taking the area in the display screen of the first electronic device for displaying the user interface as the first display area as an example for description, the size of the second user interface is larger than the size of the first display area.

[0273] Implementation method 1: The first user interface includes the content of a part of the interface area of ​​the second user interface (excluding other content). The first user interface includes the content of the first interface area in the second user interface.

[0274] In the first implementation, in order to enable the first electronic device to display not only the content of the first interface area of ​​the second user interface, but also the content of other interface areas of the second user interface (taking the second interface area as an example), in an embodiment of the present application, the first electronic device may divide the first display area into multiple display parts - a first display part and a second display part. Among them, the first display part corresponds to the first user interface (that is, the first display part is used to display the content of the first interface area in the second user interface), and the second display part corresponds to the seventh user interface (that is, the second display part is used to display the content of the second interface area in the second user interface). Among them, the first interface area is different from the second interface area. The size of the first display part is smaller than the size of the first user interface.

[0275] Based on this, when the first electronic device executes the above steps S403-S404 and S409 to display part or all of the content of the first user interface, the first display part in the first display area displays the corresponding content.

[0276] For example, when the first electronic device executes S403, the first electronic device may display the first part of the content of the first user interface in the first display part of the first display area; when executing S404, the first electronic device may translate the first user interface within the first display part in response to the second operation of the first user on the first part of the content in the first display part; when executing S409, the first electronic device may scale the first user interface in the first display part in response to the third operation of the first user on the first user interface in the first display part.

[0277] In addition, after receiving the first message, the first electronic device may also display part or all of the content of the seventh user interface in the second display portion of the first display area, wherein the seventh user interface includes the content of the second interface area of ​​the second user interface.

[0278] In one display mode, the first electronic device can display part of the seventh user interface on the second display portion. Similar to displaying the first user interface, the first user can also slide, zoom or control the seventh user interface on the second display portion through various operations. The specific process can refer to the display process of the first user interface, which will not be repeated here.

[0279] In another display mode, the first electronic device can display the entire contents of the seventh user interface on the second display portion. For example, when the size of the second display portion is greater than or equal to the seventh user interface, the first electronic device can display the entire contents of the seventh user interface by enlarging the second display portion or directly displaying the entire contents of the seventh user interface; when the size of the second display portion is smaller than the seventh user interface, the first electronic device can scale the third user interface on the second display portion to display the entire contents of the third user interface.

[0280] It should be noted that implementation method 1 is illustrated by taking the second user interface being divided into two interface areas and the first display area of ​​the first electronic device being divided into two display parts as an example, but it does not limit the method provided in the embodiment of the present application. In an actual scenario, according to the actual layout of the second user interface, the second user interface can be divided into two or more interface areas, and the first display area of ​​the first electronic device is also correspondingly divided into two or more display parts. Each display part displays the content of the corresponding interface area, and the specific display process can refer to the above description.

[0281] Implementation method 2: The first user interface includes the content of a part of the interface area of ​​the second user interface (excluding other content). The first user interface includes the content of the first interface area of ​​the second user interface.

[0282] Different from the first implementation, the first electronic device displays the content of the first interface area in the second user interface (i.e., the first user interface) only in the first display portion in the first display area, but does not display the content of other interface areas in the second user interface (e.g., the content of the second interface area in the first implementation). Optionally, in the second implementation, the first display portion may be the entire area of ​​the first display area or a partial area of ​​the first display area, which is not limited in the present application.

[0283] Regarding the process of the first electronic device displaying the first user interface in the first display part in the first display area, reference may be made to the corresponding description in implementation method 1, which will not be repeated here.

[0284] It should be noted that in implementation mode 1 and implementation mode 2, the first interface area or the second interface area may be an area with different display functions in the second user interface. For example, the first interface area or the second interface area may be at least one of the following in the second user interface: a title bar, a menu bar, a toolbar, an address bar, a work area, a status bar, or a scroll bar. Of course, the first interface area or the second interface area may also be divided in other ways, which is not limited in this application.

[0285] Take the user interface of an application on a smart phone as an example. The user interface of a general application includes a title bar, a workspace, and a menu bar. The title bar is located at the top, the workspace is located in the middle, and the menu bar is located at the bottom. Assuming that the second electronic device in the embodiment of the present application is a smart phone, the second user interface is the user interface of the application of the second electronic device. Based on this, the first user interface displayed by the first electronic device can be the workspace of the second user interface, and the seventh user interface can be the title bar and / or menu bar of the second user interface, which is not limited in this application.

[0286] In addition, in implementation mode 1 or implementation mode 2, the first electronic device may determine the first interface area or the second interface area in the following manner, but not limited to:

[0287] Mode 1: The first electronic device receives interface segmentation information from the second electronic device, and determines the first interface area or the second interface area in the second user interface according to the interface segmentation information, wherein the interface segmentation information is used to indicate the position of the first interface area or the second interface area.

[0288] Optionally, the second electronic device may perform image recognition on the second user interface to determine the first interface area and the second interface area, and then determine the interface segmentation information. For example, the second electronic device may perform image recognition on the second user interface according to a set interface layout strategy.

[0289] Method 2: The first electronic device performs image recognition on the second user interface to determine the first interface area and the second interface area.

[0290] Optionally, the first electronic device may perform image recognition on the second user interface according to a set interface layout strategy. The interface layout strategy in the first electronic device may be sent by the first electronic device or may be preset, which is not limited in this application.

[0291] Implementation method three: The first user interface includes the content of all interface areas of the second user interface (excluding other content). For example, the first user interface is the same as the second user interface. The first user interface may include all display function areas of the second user interface.

[0292] Based on this, when the first electronic device executes the above steps S403-S404 and S409 to display part or all of the content of the first user interface, the corresponding content can be displayed in the first display area of ​​the first electronic device without dividing the first display area into different display parts.

[0293] For example, when the first electronic device executes S403, the first electronic device can display the first part of the content of the first user interface in the first display area; when executing S404, the first electronic device can translate the first user interface in the first display area in response to the second operation of the first user on the first user interface in the first display area; when executing S409, the first electronic device can scale the first user interface in the first display area in response to the third operation of the first user on the first user interface in the first display area.

[0294] It should also be noted that the above three implementation methods are described by taking the area used by the first electronic device to display the user interface as the first display area as an example. In a scenario where the first electronic device has multiple display modes, when the first electronic device switches the display mode and the area used to display the user interface in the display screen of the first electronic device is adjusted from the first display area to the second display area, the first electronic device can also use similar steps to display the first user interface in the second display area. Taking implementation method one as an example, the first electronic device can divide the second display area into two display parts—a third display part and a fourth display part, and then display the first user interface and the seventh user interface in the third display part and the fourth display part respectively. The specific steps are as follows:

[0295] The third display portion of the first electronic device in the second display area displays the seventh part of the content in the first user interface; wherein the size of the second user interface is larger than the size of the second display area, and the size of the third display portion is smaller than the size of the first user interface;

[0296] The first electronic device may also translate the first user interface in the third display portion to display an eighth portion of the first user interface in response to the first user's second operation on the seventh portion of the content on the third display portion, wherein the seventh portion of the content is different from the eighth portion of the content;

[0297] The fourth display portion of the first electronic device in the second display area displays part or all of the content of the seventh user interface.

[0298] Of course, the first user can slide, zoom or control the corresponding user interface in the third display part and the fourth display part through various operations. The specific process can refer to the above description of the display process of the first user interface and the seventh user interface, which will not be repeated here.

[0299] In the embodiment of the present application, the performance of the second electronic device may be better than that of the first electronic device. Optionally, the second electronic device is a rich device and the first electronic device is a light device. Figure 1B As shown, the second electronic device may be a smart phone, and the first electronic device may be a smart wearable device (such as a smart watch). For another example, in a smart home scenario, the second electronic device may be a smart phone, and the first electronic device may be a smart home device (such as a smart TV). Through this solution, the first electronic device with poor performance can control the application of the second electronic device with better performance in a distributed manner. Furthermore, the first electronic device with poor performance can also realize the functions of the second electronic device with better performance, and can realize functional collaboration or functional expansion between electronic devices with different performance without optimizing the software and hardware resources of the first electronic device.

[0300] The embodiments of the present application do not limit the performance of the first electronic device and the second electronic device. For example, the second electronic device is a smart phone, and the first electronic device can be a computer. In some scenarios, the first electronic device and the second electronic device can also be the same electronic device.

[0301] In summary, the embodiment of the present application provides a method for functional collaboration. In this method, the first electronic device is installed with an application control, and the second electronic device is installed with an application corresponding to the application control; when the user performs an operation on the application control on the first electronic device, the first electronic device sends a run instruction to the second electronic device to instruct the second electronic device to run the application; and the first electronic device can display part of the user interface of the application; and as the user operates the part of the content of the currently displayed user interface, the first electronic device can translate the user interface to display another part of the content of the user interface. Through this method, with the assistance of the second electronic device, the first electronic device can display the user interface of the application of the second electronic device, that is, realize the function of the application in the second electronic device. Based on this scheme, the function of the second electronic device can be extended to the first electronic device to realize the functional collaboration of the two electronic devices. In summary, the scheme provided by the present application can realize the functional collaboration and functional expansion of electronic devices without considering the software and hardware resource constraints of the electronic device, and avoid causing large power consumption and memory overhead to the electronic device. Further, the first electronic device can display part of the user interface of the application, and the user can translate the user interface by operating the user interface, so that the user interface of the application displayed in the first electronic device can display more, richer, more effective or more detailed content, and the user experience is higher. In addition, this method does not require frequent communication interactions between electronic devices. The first electronic device can switch the displayed content through local operations. Therefore, this method can avoid power consumption overhead and operation delays caused by frequent interactions between devices.

[0302] Combination Figure 4 The embodiment shown is based on Figure 1B The present application also provides an embodiment of the communication scenario shown in the figure. In which, application A is installed in the smart phone, and a control of application A is installed in the smart watch, but application A is not installed.

[0303] Combine the following Figure 5 , the process of this embodiment is described.

[0304] S500: Smartphone creates a virtual super-large screen.

[0305] Optionally, the smartphone can create the virtual extra-large screen after starting the function collaboration function, or after establishing a communication connection with the smart watch, or after the smart watch distributedly controls the smart startup application A.

[0306] Optionally, the size of the virtual super large screen is larger than the size of the display screen of the smartphone. For example, the length of the virtual super large screen is 1.5 times the length of the display screen of the smartphone.

[0307] S501: After the smartphone starts application A, a user interface of application A (hereinafter referred to as user interface a) is created on the virtual super large screen.

[0308] Optionally, in this embodiment, the smartphone can start application A through, but is not limited to, the following steps: after the smartwatch detects that the user has performed a start operation on the control of application A, it sends an instruction message to the smartphone. The instruction message is used to instruct the smartphone to run application A. After receiving the instruction message, the smartphone starts application A. The specific process of the smartphone starting application A can be referred to Figure 4 The description of S402 in the illustrated embodiment will not be expanded here.

[0309] S502: The smart phone performs image recognition on the user interface a, identifies the title bar, menu bar and work area in the user interface a, and determines interface segmentation information. The interface segmentation information is used to indicate the positions of the title bar, menu bar and work area so that the smart watch can segment the user interface a.

[0310] S503: The smart phone sends the image frame corresponding to the user interface a and the interface segmentation information to the smart watch.

[0311] S504: The smart watch receives the image frame and interface segmentation information from the smart phone.

[0312] S505: The smart watch segments user interface a according to the interface segmentation information to obtain user interface b, user interface c and user interface d.

[0313] Assume that user interface b is the title bar at the top of user interface a, user interface c is the work area in the middle of user interface a, and user interface d is the menu bar at the bottom of user interface a.

[0314] S506: The smart watch displays the three segmented user interfaces in the three display parts of the current display area of ​​the display screen respectively, that is, the smart watch displays user interface b in display part 1, user interface c in display part 2, and user interface d in display part 3.

[0315] Since user interface a is created on the virtual super large screen of the smart phone, the size of user interface a is larger than the size of the display screen of the smart watch and also larger than the size of the current display area in the display screen of the smart watch. Generally, the size of at least one user interface divided by user interface a is also larger than the size of the corresponding display part.

[0316] For example, the user interface c (the working area of ​​the user interface a) is larger than the size of the display part 2. When the smart watch displays the user interface c on the display part 2, it can Figure 4In the embodiment shown, the method for displaying the first user interface is implemented. That is, the smart watch can display part of the content of the user interface c on the display part 2; the user of the smart watch can translate the user interface c in the display part 2 through local sliding operation; the user of the smart watch can also zoom the user interface c in the display part 2 through local zoom operation to display the entire content of the user interface c. In addition, the user of the smart watch can also realize the distributed control of the application A in the smart phone through control operation. The specific process can be referred to Figure 4 The relevant description in the illustrated embodiment will not be repeated here.

[0317] Through S506, the smart watch can display the three different interface areas of the user interface a, namely, the title bar, the work area, and the menu bar.

[0318] S507: Based on the above example, the smart watch can Figure 4 The method for displaying the first user interface described in the illustrated embodiment displays the user interface c in the display portion 2. Then, when the smart watch detects a sliding operation of the user in the display portion 2, the user interface c can be translated in the display portion 2 to achieve a visual effect of the user sliding the user interface c.

[0319] It should be noted that the smart watch can display user interface b or user interface d in the display part 1 or the display part 3 in the same manner as displaying user interface c; or the smart watch can display user interface b or user interface d in the display part 1 or the display part 3 in a zoomed manner by default.

[0320] In this embodiment, if Figure 6 As shown, according to the functional logic, the smart phone may include but is not limited to the following modules: application A, interface management module, virtual super large screen, and communication module; the smart watch may include but is limited to: communication module, interface management module, watch display screen. Among them, the above interface management module can be a HelperActivity module, and the communication module can be a wireless communication module integrating communication protocols (such as WebSocket, BT, WLAN, Wi-Fi), for example Figure 2A A mobile communication module or a wireless communication module in.

[0321] In this embodiment, the modules in the smart phone and smart watch can be Figure 6 The process shown implements the functional collaboration method.

[0322] S600: The interface management module in the smart phone creates a virtual super large screen.

[0323] S601: After application A in the smart phone is started, a user interface a of application A is created on the virtual super large screen.

[0324] S602: The interface management module of the smart phone obtains an image frame of the user interface a from the virtual super large screen.

[0325] S603: The interface management module of the smart phone performs image recognition on the user interface a, identifies the positions of the title bar, menu bar and workspace in the user interface a, and determines interface segmentation information.

[0326] S604: The interface management module of the smart phone performs image compression processing on the image frame of the user interface a to obtain image compression data.

[0327] S605: The interface management module of the smart phone sends the image compression data and interface segmentation information to the smart watch through the communication module. After receiving the image compression data and interface segmentation information, the communication module in the smart watch forwards them to the interface management module of the smart watch.

[0328] S606: After receiving the image compression data and the interface segmentation information, the interface management module of the smart watch performs image decompression processing on the image compression data to obtain an image frame of the user interface a.

[0329] S607: The interface management module of the smart watch divides the user interface a according to the interface division information to obtain the user interface b, the user interface c and the user interface d.

[0330] S608: The interface management module of the smart watch displays corresponding user interfaces according to the three display parts in the current display area of ​​the watch display screen, that is, the interface management module displays user interface b in display part 1, user interface c in display part 2, and user interface d in display part 3.

[0331] S609: The user performs a sliding operation on the user interface c on the display part 2 of the watch display.

[0332] S610: After the watch display screen detects the sliding operation, it sends the operation data (such as the real-time coordinates of the touch) to the interface management module of the smart watch.

[0333] S611: The interface management module of the smart watch translates the user interface c in the display part 2 according to the operation data, so as to achieve the visual effect of the user sliding the user interface c.

[0334] Take application A as Take the example of an example. After that, the local display shows The normal user interface of the main page is as follows Fig. 7A Based on the functional collaboration method provided in the embodiment of the present application, the smart phone uses the smart watch Control Startup After that, create a virtual jumbotron The main page user interface is as follows Figure 7B As shown in (a), the length of the user interface a is greater than Fig. 7A The length of the ordinary user interface shown in the figure, the working area of ​​user interface a displays more and richer content. After the smart phone and the smart watch communicate and interact, the smart watch can divide the user interface a into three parts: user interface b (the title bar of user interface a), user interface c (the working area of ​​user interface a), and user interface d (the menu bar of user interface a); then, the smart watch can display the three divided user interfaces in the three display parts of the current display area of ​​the display screen, such as Figure 7B As shown in (b) in FIG. 1 , user interface b is scaled on display portion 1, part of user interface c is displayed on display portion 2, and user interface d is scaled on display portion 3. When the user of the smart watch slides up on user interface c on display portion 2, as shown in FIG. Figure 7B As shown in (c) in the figure, the smart watch translates the user interface c upward in the display part 2, giving the user the effect of sliding the user interface c upward, while the contents displayed in the display part 1 and the display part 2 remain unchanged.

[0335] In summary, as the user performs a sliding operation on the user interface c on the display part 2, the smart watch can switch the display content by translating the user interface c on the display part 2, such as Figure 7C As shown. Among them, Figure 7C The content in the thick black dotted line box is equivalent to part of the content displayed on the display part 2. The user's sliding operation on the user interface c in the display part is equivalent to moving the thick black dotted line box in the opposite direction, causing the user interface c displayed on the display part 2 to shift horizontally.

[0336] As the user performs a sliding operation on the user interface c on the display part 2, when the content displayed after the user interface c displayed on the display part 2 is translated to include the lower boundary of the user interface a, it can be indicated that the user performs a sliding operation on the user interface a on the smart watch side. Afterwards, the smart watch can notify the smartphone side of the event so that the smartphone updates Main page to user interface e, such as Fig.7D As shown in (a) in FIG. 1 , the smartphone then notifies the smartwatch of the user interface e through interaction, and the smartwatch continues to use Figure 7B The display method shown in (b) in FIG. 1 divides and displays the user interface e, such as Fig.7D When the user performs a sliding operation on the user interface displayed on the display portion 2, the smart watch translates the user interface, such as Fig.7D As shown in (c) in .

[0337] It should also be noted that Figure 5-7D The embodiments and examples shown do not constitute any limitation on the methods provided in the embodiments of the present application.

[0338] For example, in some scenarios, the smartphone may not send interface segmentation information to the smart watch, and the smart watch may perform image recognition and interface segmentation on its own.

[0339] For example, in some scenarios, after splitting the user interface, the smart watch may only display the user interface including the workspace in the current display area of ​​the display screen, without displaying other user interfaces. That is, the smart watch may only display part of the content of user interface c on the display screen, without displaying user interface b and user interface d.

[0340] For another example, in some scenarios, the smart watch may not split the user interface sent by the smart phone, but instead display the content of the received user interface in the current display area of ​​the display screen. Take the example of Figure 7B After the user interface a shown in (a) in FIG. 1 is displayed, the smart watch can display part of the content of the user interface a on the display screen, such as Fig. 7E As shown in (a) in FIG. 1 , when the user of the smart watch slides up on the display screen of the user interface a, as shown in FIG. Fig. 7E As shown in (b), the smart watch translates the user interface a upward on the display screen, giving the user the effect of sliding the user interface a upward.

[0341] It should also be noted that each step involved in the above embodiments can be performed by a corresponding device, or by a chip, processor, or chip system in the device, and the embodiments of the present application do not limit them. The above embodiments are only described by taking the execution by the corresponding device as an example. In addition, the specific implementation methods or examples in the above embodiments do not limit the solutions provided in the embodiments of the present application.

[0342] It should be noted that in the above embodiments, some steps can be selected for implementation, and the order of the steps in the diagram can be adjusted for implementation, and this application does not limit this. It should be understood that executing some steps in the diagram, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.

[0343] It is understandable that, in order to implement the functions in the above embodiments, the various devices involved in the above embodiments include hardware structures and / or software modules corresponding to the execution of the various functions. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0344] It can be understood that the above-mentioned network architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new services, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0345] It should be noted that the "steps" in the embodiments of the present application are only for illustration, and are a method of expression used to better understand the embodiments, and do not constitute a substantial limitation on the execution of the scheme of the present application. For example, the "steps" can also be understood as "features". In addition, the steps do not constitute any limitation on the execution order of the scheme of the present application. Any changes in the order of steps, step merging, or step splitting made on this basis that do not affect the implementation of the overall scheme, and the resulting new technical solutions are also within the scope of the disclosure of the present application.

[0346] Based on the same technical concept, the present application also provides an electronic device, which can implement the methods provided in the above embodiments. Figure 8 As shown, the electronic device 800 includes a communication unit 801 and a processing unit 802. Optionally, the electronic device 800 may also include a display screen 803.

[0347] The display screen 803 is used to display a user interface.

[0348] The communication unit 801 is used to receive and send data. Optionally, the communication unit 801 may include a transceiver.

[0349] The processing unit 802 is used to execute the method provided in the above embodiment through the communication unit 801 or the display unit 803 .

[0350] In one embodiment, the electronic device 800 is used to implement Figure 4 The functions of the first electronic device in the embodiment shown. The first electronic device is installed with an application control. The processing unit 802 is used to:

[0351] In response to a first operation of the application control on the first electronic device by the first user, sending a running instruction to the second electronic device through the communication unit 801, wherein the running instruction is used to instruct the second electronic device to run the application corresponding to the application control;

[0352] Displaying a first portion of content of a first user interface on the display screen 803; wherein the first user interface is a user interface of the application;

[0353] In response to the first user's second operation on the first portion of content, the first user interface is translated to display a second portion of content of the first user interface on the display screen 803; wherein the second portion of content is different from the first portion of content.

[0354] Optionally, the processing unit 802 is further configured to:

[0355] A first message is received from the second electronic device through the communication unit 801; wherein the first message is used to indicate a second user interface of the application running on the second electronic device, and the first user interface includes the content of a partial interface area or the content of the entire interface area of ​​the second user interface.

[0356] Optionally, the processing unit 802 is further configured to:

[0357] In response to a third operation of the first user on the first user interface, the first user interface is scaled to display all content of the first user interface on the display screen 803 .

[0358] Optionally, the processing unit 802 is further configured to:

[0359] After translating the first user interface, when the second portion of content displayed on the display screen 803 includes the boundary of the second user interface, a second message is sent to the second electronic device through the communication unit 801, and the second message is used to indicate that the second user interface has been slid.

[0360] Optionally, the second message includes sliding direction information of the second user interface.

[0361] Optionally, the sliding direction information includes the position of the boundary in the second user interface or the translation direction of the second operation indication.

[0362] Optionally, the processing unit 802 is further configured to:

[0363] After sending the second message to the second electronic device through the communication unit 801, receiving a third message from the second electronic device through the communication unit 801; wherein the third message is used to indicate a third user interface of the application running on the second electronic device, and the third user interface is generated by the second electronic device according to the second message;

[0364] Displaying a third portion of content of a fourth user interface on the display screen 803; wherein the fourth user interface includes content of a partial interface area or content of the entire interface area of ​​the third user interface;

[0365] In response to the second operation of the first user on the third portion of content, the fourth user interface is translated to display a fourth portion of content of the fourth user interface on the display screen 803; wherein the fourth portion of content is different from the third portion of content.

[0366] Optionally, the processing unit 802 is further configured to:

[0367] In response to the first user's fourth operation on the first user interface, a fourth message is sent to the second electronic device through the communication unit 801; wherein the fourth operation is a control operation on the first user interface, and the fourth message is used to indicate the fourth operation.

[0368] Optionally, the fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time;

[0369] The operation position information includes a first operation position or a second operation position; the first operation position is a position where the fourth operation is mapped to the second user interface; and the second operation position is a position where the fourth operation is in the first user interface.

[0370] Optionally, when the operation position information includes the second operation position, the operation position information further includes relative position information of the first user interface with respect to the second user interface.

[0371] Optionally, the processing unit 802 is further configured to:

[0372] After sending a fourth message to the second electronic device through the communication unit 801 in response to a fourth operation of the first user on the first user interface, a fifth message from the second electronic device is received through the communication unit 801; wherein the fifth message is used to indicate a fifth user interface of the application running on the second electronic device, and the fifth user interface is generated after the second electronic device performs the fourth operation on the second user interface;

[0373] Displaying the fifth part of the content of the sixth user interface on the display screen 803; wherein the sixth user interface includes the content of a part of the interface area or the content of the whole interface area of ​​the fifth user interface;

[0374] In response to the second operation of the first user on the fifth portion of content, the sixth user interface is translated to display a sixth portion of content of the sixth user interface on the display screen 803; wherein the fifth portion of content is different from the sixth portion of content.

[0375] Optionally, the size of the second user interface is larger than the size of the first display area in the display screen 803 of the first electronic device, the first display area being an area of ​​the first electronic device used to display a user interface; the first user interface includes the content of the first interface area in the second user interface;

[0376] The processing unit 802 is specifically configured to:

[0377] The first display portion in the first display area on the display screen 803 displays the first part of the content of the first user interface; wherein the size of the first display portion is smaller than the size of the first user interface;

[0378] The processing unit 802 is specifically configured to:

[0379] In response to the second operation performed by the first user on the first portion of content in the first display portion, the first user interface is translated within the first display portion on the display screen 803 .

[0380] Optionally, the processing unit 802 is further configured to:

[0381] After receiving the first message from the second electronic device through the communication unit 801, part or all of the content of the seventh user interface is displayed on the second display part in the first display area on the display screen 803; wherein the seventh user interface includes the content of the second interface area in the second user interface, and the second interface area is different from the first interface area.

[0382] Optionally, the first interface area or the second interface area is at least one of the following in the second user interface:

[0383] Title bar, menu bar, toolbar, address bar, workspace, status bar, or scroll bar.

[0384] Optionally, when the area of ​​the first electronic device for displaying the user interface is adjusted to the second display area in the first electronic device display screen 803, the processing unit 802 is further configured to:

[0385] The third display portion in the second display area on the display screen 803 displays the seventh part of the content in the first user interface; wherein the size of the second user interface is larger than the size of the second display area, and the size of the third display portion is smaller than the size of the first user interface;

[0386] In response to the second operation of the first user on the seventh portion of content in the third display portion, the first user interface is translated within the third display portion on the display screen 803 to display an eighth portion of content of the first user interface; wherein the seventh portion of content is different from the eighth portion of content;

[0387] A fourth display portion in the second display area on the display screen 803 displays part or all of the content of the seventh user interface.

[0388] The processing unit 802 is further configured to:

[0389] Interface segmentation information is received from the second electronic device through the communication unit 801, and the first interface area or the second interface area is determined in the second user interface according to the interface segmentation information; wherein the interface segmentation information is used to indicate the position of the first interface area or the second interface area; or image recognition is performed on the second user interface to determine the first interface area or the second interface area.

[0390] Optionally, the second user interface is generated by the second electronic device on a virtual screen.

[0391] Optionally, the size of the virtual screen is larger than the size of the display screen of the second electronic device.

[0392] Optionally, the second operation is a sliding operation.

[0393] Optionally, the performance of the second electronic device is better than the performance of the first electronic device.

[0394] Optionally, the first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; the second electronic device is a smart phone.

[0395] In another embodiment, the electronic device 800 is used to implement Figure 4The functions of the second electronic device in the embodiment shown. The second electronic device has an application installed. The processing unit 802 is used to:

[0396] Receiving an operation instruction from a first electronic device through the communication unit 801; the operation instruction is sent by the first electronic device in response to a first user's first operation on an application control on the first electronic device;

[0397] According to the running instruction, running the application corresponding to the application control;

[0398] According to the second user interface of the application, a first message is sent to the first electronic device through the communication unit 801, so that the first electronic device displays a first part of the content of the first user interface; wherein the first message is used to indicate the second user interface; and the first user interface includes the content of a part of the interface area or the content of the whole interface area of ​​the second user interface;

[0399] When a second message is received from the first electronic device through the communication unit 801, a third user interface of the application is generated according to the second message, and a third message is sent to the first electronic device through the communication unit 801; wherein the second message is used to indicate that the second user interface has been slid, and the third message is used to indicate the third user interface.

[0400] Optionally, the second message includes sliding direction information of the second user interface.

[0401] Optionally, the processing unit 802 is further configured to:

[0402] When the fourth message is received from the first electronic device through the communication unit 801, the fifth user interface of the application is generated according to the fourth message, and a fifth message is sent to the first electronic device through the communication unit 801; wherein the fourth message is used to indicate a fourth operation, and the fourth operation is a control operation of the first user on the first user interface; and the fifth message is used to indicate the fifth user interface.

[0403] Optionally, the fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time;

[0404] The operation position information includes a first operation position or a second operation position; the first operation position is a position where the fourth operation is mapped to the second user interface; and the second operation position is a position where the fourth operation is in the first user interface.

[0405] Optionally, when the operation position information includes the second operation position, the operation position information further includes relative position information of the first user interface with respect to the second user interface.

[0406] Optionally, the first user interface includes the content of the first interface area in the second user interface; the processing unit 802 is further configured to:

[0407] Interface segmentation information is sent to the first electronic device through the communication unit 801, where the interface segmentation information is used to indicate the first interface area in the second user interface.

[0408] Optionally, the first interface area is at least one of the following in the second user interface:

[0409] Title bar, menu bar, toolbar, address bar, workspace, status bar, or scroll bar.

[0410] Optionally, the processing unit 802 is further configured to:

[0411] generating the second user interface on the virtual screen;

[0412] The third user interface is generated on the virtual screen.

[0413] Optionally, the size of the virtual screen is larger than the size of the display screen of the second electronic device.

[0414] Optionally, the performance of the second electronic device is better than the performance of the first electronic device.

[0415] Optionally, the first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; the second electronic device is a smart phone.

[0416] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0417] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0418] Based on the same technical concept, the present application also provides an electronic device, which can implement the methods provided in the above embodiments. Fig. 9 As shown, the electronic device 900 includes: at least one processor 901, a memory 902, a transceiver 903 and a display screen 904. For example, the above components can be connected via one or more communication buses.

[0419] The at least one processor 901 is coupled to the memory 902 and is used to read the computer program stored in the memory to execute the methods provided in the above embodiments.

[0420] Exemplarily, the at least one processor 901 may specifically be Figure 2A The processor 210 shown. The memory 902 may be specifically Figure 2A The internal memory 221 shown and / or the external memory connected to the electronic device. The transceiver 903 can be Figure 2A The mobile communication module 250 and / or the wireless communication module 260 shown. The display screen 904 can be Figure 2A The display screen 294 is shown, and the embodiment of the present application does not impose any limitation on this.

[0421] Based on the above embodiments, the embodiments of the present application further provide a computer program product, which includes a computer program; when the computer program runs on a computer, the computer executes the method provided in the above embodiments.

[0422] Based on the above embodiments, the embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the computer executes the method provided in the above embodiments.

[0423] Optionally, the above-mentioned computer may include, but is not limited to, electronic devices.

[0424] The storage medium may be any available medium that can be accessed by a computer. For example, but not limited to, a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0425] Based on the above embodiments, the embodiments of the present application further provide a chip, the chip is used to read the computer program stored in the memory, and implement the method provided in the above embodiments. Optionally, the chip may include a processor and a memory, the processor is coupled to the memory, and is used to read the computer program stored in the memory, and implement the method provided in the above embodiments.

[0426] Based on the above embodiments, the embodiments of the present application provide a chip system, which includes a processor for supporting a computer device to implement the functions involved in the electronic device in the above embodiments. In a possible design, the chip system also includes a memory, which is used to store the necessary programs and data for the computer device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

[0427] In summary, the embodiment of the present application provides a function collaboration method and an electronic device. In this method, an application control is installed in a first electronic device, and an application corresponding to the application control is installed in a second electronic device; when a user performs an operation on the application control on the first electronic device, the first electronic device sends a run instruction to the second electronic device to instruct the second electronic device to run the application; and the first electronic device can display part of the user interface of the application; and as the user operates the part of the content of the currently displayed user interface, the first electronic device can translate the user interface to display another part of the content of the user interface. Through this method, with the assistance of the second electronic device, the first electronic device can display the user interface of the application of the second electronic device, that is, realize the function of the application in the second electronic device. Based on this scheme, the function of the second electronic device can be extended to the first electronic device to realize the functional collaboration of the two electronic devices. In summary, the scheme provided by the present application can realize the functional collaboration and functional expansion of electronic devices without considering the software and hardware resource constraints of the electronic device, and avoid causing large power consumption and memory overhead to the electronic device. Further, the first electronic device can display part of the user interface content of the application, and the user can translate the user interface by operating the user interface, so that the user interface of the application displayed in the first electronic device can display more, richer, more effective or more detailed content, and the user experience is higher. In addition, this method does not require frequent communication interactions between electronic devices. The first electronic device can switch the displayed content through local operations. Therefore, this method can avoid power consumption overhead and operation delays caused by frequent interactions between devices.

[0428] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0429] The present application is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0430] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0431] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0432] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A functional collaboration method, characterized in that: Applied to a first electronic device, the first electronic device is installed with an application control, the method includes: The first electronic device sends a running instruction to a second electronic device in response to a first operation of the application control on the first electronic device by a first user, wherein the running instruction is used to instruct the second electronic device to run an application corresponding to the application control; The first electronic device displays a first portion of content of a first user interface; wherein the first user interface is a user interface of the application; In response to the first user's second operation on the first portion of content, the first electronic device translates the first user interface to display a second portion of content of the first user interface; wherein the second portion of content is different from the first portion of content.

2. The method according to claim 1, characterized in that The method further comprises: The first electronic device receives a first message from the second electronic device; wherein the first message is used to indicate a second user interface of the application running on the second electronic device, and the first user interface includes the content of a partial interface area or the content of the entire interface area of ​​the second user interface.

3. The method according to claim 2, characterized in that The method further comprises: In response to a third operation of the first user on the first user interface, the first electronic device zooms the first user interface to display all content of the first user interface.

4. The method according to claim 2 or 3, characterized in that After translating the first user interface, the method further includes: When the second portion of content displayed by the first electronic device includes a boundary of the second user interface, the first electronic device sends a second message to the second electronic device, where the second message is used to indicate that the second user interface has been slid.

5. The method according to claim 4, characterized in that The second message includes sliding direction information of the second user interface.

6. The method according to claim 5, characterized in that The sliding direction information includes the position of the boundary in the second user interface or the translation direction of the second operation instruction.

7. The method according to any one of claims 4 to 6, characterized in that: After the first electronic device sends the second message to the second electronic device, the method further includes: The first electronic device receives a third message from the second electronic device; wherein the third message is used to indicate a third user interface of the application running on the second electronic device, and the third user interface is generated by the second electronic device according to the second message; The first electronic device displays a third portion of content of a fourth user interface; wherein the fourth user interface includes content of a partial interface area or content of the entire interface area of ​​the third user interface; The first electronic device translates the fourth user interface in response to the second operation of the first user on the third portion of content to display a fourth portion of content of the fourth user interface; wherein the fourth portion of content is different from the third portion of content.

8. The method according to any one of claims 2 to 7, characterized in that: The method further comprises: The first electronic device sends a fourth message to the second electronic device in response to a fourth operation of the first user on the first user interface; wherein the fourth operation is a control operation on the first user interface, and the fourth message is used to indicate the fourth operation.

9. The method according to claim 8, characterized in that The fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time; The operation position information includes a first operation position or a second operation position; the first operation position is a position where the fourth operation is mapped to the second user interface; and the second operation position is a position where the fourth operation is in the first user interface.

10. The method according to claim 9, characterized in that When the operation position information includes the second operation position, the operation position information further includes relative position information of the first user interface to the second user interface.

11. The method according to any one of claims 8 to 10, characterized in that: After the first electronic device sends a fourth message to the second electronic device in response to a fourth operation of the first user on the first user interface, the method further includes: The first electronic device receives a fifth message from the second electronic device; wherein the fifth message is used to indicate a fifth user interface of the application running on the second electronic device, and the fifth user interface is generated after the second electronic device performs the fourth operation on the second user interface; The first electronic device displays a fifth portion of content of the sixth user interface; wherein the sixth user interface includes content of a partial interface area or content of the entire interface area of ​​the fifth user interface; The first electronic device translates the sixth user interface in response to the second operation of the first user on the fifth portion of content to display a sixth portion of content of the sixth user interface; wherein the fifth portion of content is different from the sixth portion of content.

12. The method according to any one of claims 2 to 11, characterized in that: The size of the second user interface is larger than the size of a first display area in the display screen of the first electronic device, and the first display area is an area of ​​the first electronic device used to display the user interface; The first user interface includes the content of the first interface area in the second user interface; The first electronic device displays a first portion of content of a first user interface, including: The first electronic device displays the first part of the content of the first user interface in a first display part in the first display area; wherein the size of the first display part is smaller than the size of the first user interface; The first electronic device translates the first user interface in response to the second operation of the first user on the first portion of content, including: The first electronic device translates the first user interface within the first display portion in response to the second operation performed by the first user on the first portion of content on the first display portion.

13. The method according to claim 12, characterized in that The method further comprises: After the first electronic device receives the first message from the second electronic device, it displays part or all of the content of the seventh user interface in the second display part of the first display area; wherein the seventh user interface includes the content of the second interface area in the second user interface, and the second interface area is different from the first interface area.

14. The method according to claim 13, characterized in that The first interface area or the second interface area is at least one of the following in the second user interface: Title bar, menu bar, toolbar, address bar, workspace, status bar, or scroll bar.

15. The method according to claim 13 or 14, characterized in that When the area of ​​the first electronic device used to display the user interface is adjusted to a second display area in the display screen of the first electronic device, the method further includes: The first electronic device displays the seventh part of the first user interface in the third display part of the second display area; wherein the size of the second user interface is larger than the size of the second display area, and the size of the third display part is smaller than the size of the first user interface; In response to the second operation of the first user on the seventh portion of content on the third display portion, the first electronic device translates the first user interface within the third display portion to display an eighth portion of content of the first user interface; wherein the seventh portion of content is different from the eighth portion of content; The fourth display part of the first electronic device in the second display area displays part or all of the content of the seventh user interface.

16. The method according to any one of claims 13 to 15, characterized in that: The method further comprises: The first electronic device receives interface segmentation information from the second electronic device, and determines the first interface area or the second interface area in the second user interface according to the interface segmentation information; wherein the interface segmentation information is used to indicate a position of the first interface area or the second interface area; or The first electronic device performs image recognition on the second user interface to determine the first interface area or the second interface area.

17. The method according to any one of claims 2 to 16, characterized in that: The second user interface is generated by the second electronic device on a virtual screen.

18. The method according to claim 17, characterized in that The size of the virtual screen is larger than the size of the display screen of the second electronic device.

19. The method according to any one of claims 1 to 18, characterized in that: The second operation is a sliding operation.

20. The method according to any one of claims 1 to 19, characterized in that The performance of the second electronic device is better than the performance of the first electronic device.

21. The method according to any one of claims 1 to 20, characterized in that The first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; the second electronic device is a smart phone.

22. A functional collaboration method, characterized in that: Applied to a second electronic device, the second electronic device having an application installed thereon, the method comprising: The second electronic device receives an operation instruction from the first electronic device; the operation instruction is sent by the first electronic device in response to a first user's first operation on an application control on the first electronic device; The second electronic device runs the application corresponding to the application control according to the running instruction; The second electronic device sends a first message to the first electronic device according to the second user interface of the application, so that the first electronic device displays a first part of the content of the first user interface; wherein the first message is used to indicate the second user interface; the first user interface includes the content of a part of the interface area or the content of the whole interface area of ​​the second user interface; When the second electronic device receives a second message from the first electronic device, it generates a third user interface of the application based on the second message, and sends a third message to the first electronic device; wherein the second message is used to indicate that the second user interface has been slid, and the third message is used to indicate the third user interface.

23. The method of claim 22, wherein: The second message includes sliding direction information of the second user interface.

24. The method according to claim 22 or 23, characterized in that The method further comprises: When the second electronic device receives a fourth message from the first electronic device, it generates a fifth user interface of the application according to the fourth message, and sends a fifth message to the first electronic device; wherein the fourth message is used to indicate a fourth operation, which is a control operation of the first user on the first user interface; and the fifth message is used to indicate the fifth user interface.

25. The method of claim 24, wherein: The fourth message includes at least one of the following: operation location information, operation type, operation instruction, and operation time; The operation position information includes a first operation position or a second operation position; the first operation position is a position where the fourth operation is mapped to the second user interface; and the second operation position is a position where the fourth operation is in the first user interface.

26. The method of claim 25, wherein: When the operation position information includes the second operation position, the operation position information further includes relative position information of the first user interface to the second user interface.

27. The method according to any one of claims 22 to 26, characterized in that The first user interface includes the content of the first interface area in the second user interface; the method further includes: The second electronic device sends interface segmentation information to the first electronic device, where the interface segmentation information is used to indicate the first interface area in the second user interface.

28. The method of claim 27, wherein: The first interface area is at least one of the following in the second user interface: Title bar, menu bar, toolbar, address bar, workspace, status bar, or scroll bar.

29. The method according to any one of claims 22 to 28, characterized in that The method further comprises: The second electronic device generates the second user interface on the virtual screen; The second electronic device generates the third user interface on the virtual screen.

30. The method of claim 29, wherein: The size of the virtual screen is larger than the size of the display screen of the second electronic device.

31. The method according to any one of claims 22 to 30, characterized in that The performance of the second electronic device is better than the performance of the first electronic device.

32. The method according to any one of claims 22 to 31, characterized in that The first electronic device is any one of the following: a computer, a smart wearable device or a smart home device; the second electronic device is a smart phone.

33. A functional collaboration method, characterized in that: Applied to a communication system consisting of a first electronic device and a second electronic device, wherein the first electronic device is installed with an application control, and the second electronic device is installed with an application corresponding to the application control, the method comprising: The first electronic device sends a running instruction to the second electronic device in response to a first operation of the application control on the first electronic device by a first user; the running instruction is used to instruct the second electronic device to run the application; The second electronic device receives the running instruction from the first electronic device; the second electronic device runs the application according to the running instruction; The first electronic device displays a first portion of content of a first user interface; wherein the first user interface is a user interface of the application; In response to the first user's second operation on the first portion of content, the first electronic device translates the first user interface to display a second portion of content of the first user interface; wherein the second portion of content is different from the first portion of content.

34. A first electronic device, characterized in that: include: Display screen, communication unit and processing unit; The display screen is used to display a user interface; The communication unit is used to receive and send data; The processing unit is used to execute the method according to any one of claims 1-21.

35. A second electronic device, characterized in that: include: a communication unit and a processing unit; The communication unit is used to receive and send data; The processing unit is used to execute the method according to any one of claims 22 to 32.

36. A first electronic device, characterized in that: include: a display screen, at least one processor and a memory; The at least one processor is coupled to the memory, and the at least one processor is configured to read a computer program stored in the memory to execute the method according to any one of claims 1 to 21.

37. A second electronic device, characterized in that: include: at least one processor and memory; The at least one processor is coupled to the memory, and the at least one processor is configured to read the computer program stored in the memory to execute the method according to any one of claims 22 to 32.

38. A communication system, characterized in that: include: The first electronic device as claimed in claim 36, and the second electronic device as claimed in claim 37.

39. A chip, characterized in that: include: Processor and memory; The processor is coupled to the memory, and the processor is used to call the computer program stored in the memory during operation to implement the method according to any one of claims 1 to 32.

40. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 32.

41. A computer program product, characterized in that When the computer program product runs on an electronic device, the electronic device executes the method according to any one of claims 1 to 32.

Images