Collaboration method between electronic devices, collaboration system and electronic device

Abstract

The present application discloses a method for collaboration between electronic devices, which is applied to a system including at least a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device establish a communication connection, and the method includes: the first electronic device displays a target interface, which is an interface for a user of the first electronic device to input information; the first electronic device sends interface view data corresponding to the target interface to the second electronic device, so that the second electronic device generates and displays a collaborative interface corresponding to the target interface according to the interface view data; the second electronic device can obtain input information by receiving the information input operation of the user of the second electronic device on the collaborative interface, and sends the input information to the first electronic device, so as to realize the collaboration of the second electronic device with the information input of the first electronic device, so as to improve the user experience. The present application also discloses a collaborative system and an electronic device.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a collaboration method, a collaboration system and an electronic device between electronic devices. Background Art

[0002] With the popularization of electronic devices such as televisions and smart screens, there are more and more application scenarios for electronic devices. In some scenarios where users interact with electronic devices, the way in which users directly input information to electronic devices cannot meet the user's usage needs well. For example, in the current online education scenario, the TV (or smart screen) plays online video courses (also called online class videos), and users study online video courses through TV. In the learning process, there is usually a scenario where the teacher asks a question and the user needs to answer the question online. The current user uses the TV remote control to enter the answer to the answer sheet displayed on the TV. However, the way to enter the answer to the answer sheet through the remote control is limited to two methods: text input and option selection, which prevents the user from entering more information. On the other hand, the operation of text input through the remote control is very cumbersome, which affects the user experience. Summary of the invention

[0003] The present application provides a collaboration method, a collaboration system and an electronic device between electronic devices. During the use of an electronic device, information input to the electronic device can be collaborated with by another electronic device to achieve convenient and fast information input to the electronic device, thereby improving the user experience.

[0004] To solve the above technical problems, in the first aspect, the embodiment of the present application provides a collaborative method between electronic devices, which is applied to a system including at least a first electronic device and a second electronic device, wherein the first electronic device and the second electronic device establish a communication connection, and the method includes: the first electronic device displays a target interface, which is an interface for the user of the first electronic device to input information; the first electronic device sends the interface view data corresponding to the target interface to the second electronic device; the second electronic device receives the interface view data, and generates and displays a collaborative interface corresponding to the target interface according to the interface view data. The interface elements such as text and graphics included in the collaborative interface are respectively the same as the interface elements such as text and graphics included in the target interface, and the layout of each element on the collaborative interface is the same as the layout of each element on the target interface. The collaborative interface is used by the user of the second electronic device to input information on the second electronic device, and the input information is the information that needs to be input by the user of the first electronic device on the target interface. The second electronic device receives the information input operation of the user of the second electronic device on the collaborative interface to obtain input information, and the second electronic device sends the input information to the first electronic device. That is, the first electronic device cooperates with the information input of the first electronic device through the second electronic device to realize convenient and fast information input of the first electronic device, thereby improving the user experience.

[0005] The user of the first electronic device and the user of the second electronic device may be the same user or different users.

[0006] The first electronic device may be a television, and the second electronic device may be a mobile phone. The collaborative method between electronic devices provided in the present application may be applied to scenarios in which users watch online course videos through television, and users answer questions online on answer sheets displayed on television through their mobile phones.

[0007] In a possible implementation of the first aspect, the method further includes: the first electronic device runs an application and displays an application service interface; the target interface is a partial interface of the application service interface. That is, the target interface is a partial interface of the application service interface. For example, the application may be an online video course application in an online video course playback scenario, and the application service interface is a video playback interface of the online video course application playing the online video course. The target interface may be an answer sheet on the video playback interface, or may be an interface for the user to input information on other video playback interfaces.

[0008] In a possible implementation of the first aspect, the method further includes: the first electronic device obtains application service data corresponding to the application from the management device for displaying the application service interface, and obtains interface view data corresponding to the target interface, so that when the first electronic device displays the target interface, the first electronic device directly sends the interface view data of the target interface to the second electronic device, so that the second electronic device constructs a collaborative interface corresponding to the target interface according to the interface view data. The application service data may be, for example, the video streaming data of the aforementioned online video course, and the interface view data may be the data of the aforementioned answer sheet.

[0009] In addition, the management device can be a server, or other electronic devices such as a computer or tablet.

[0010] In a possible implementation of the first aspect, the method further includes: the first electronic device obtains application service data corresponding to the application from the management device for displaying the application service interface, and the first electronic device performs data analysis on the application service data, and obtains interface view data corresponding to the target interface from the application service data. For example, the application service data may be the video streaming media data of the aforementioned online video course, and information such as interface elements and interface layout of the answer sheet may be analyzed from the video streaming media data by image analysis.

[0011] In a possible implementation of the first aspect, the application is a video playback application, and the method further includes: when the first electronic device displays the target interface, the first electronic device stops playing the video; after the first electronic device receives the input information, the first electronic device plays the video. That is, the video can be paused when the user inputs information to prevent the user from missing video-related content or affecting the user's information input.

[0012] In a possible implementation of the first aspect above, the video playback application may be the aforementioned online video course application. In addition, the application may also be a video conferencing application, or other video applications.

[0013] In a possible implementation of the first aspect above, the information input operation includes any one of the following: a text input operation; an operation of selecting an option; an operation of adding a file; and a voice input operation. The text input operation may be text input through an input keyboard, or text input by handwriting or other methods. The operation of selecting an option may be a click operation, or an operation of drawing (e.g., underlining), circling, or the like. The operation of adding a file may be the addition of a picture file, a Word file, an audio file, or other files. The voice input operation may be a user of the second electronic device performing voice input through a recording component of the second electronic device.

[0014] In a possible implementation of the first aspect above, the method further includes: the second electronic device detects a trigger operation of the user of the second electronic device on the collaborative interface, generates and displays an information input control; the second electronic device receives the information input operation of the user of the second electronic device on the collaborative interface through the information input control. The information input control can be an input keyboard corresponding to the aforementioned text input operation, or it can be a graffiti pen tool corresponding to the aforementioned operations such as outlining and circling, or it can be a voice input control corresponding to the aforementioned voice input operation. Of course, it can also be other types of controls for users to input information.

[0015] In a possible implementation of the first aspect, after the second electronic device sends the input information to the first electronic device, the method further includes: the second electronic device updates the collaborative interface and displays the second interface. The second interface may display a prompt message that the user of the second electronic device has input information to remind the user of the second electronic device that the information input has been completed. The next interface for the user of the second electronic device to input information may also be displayed.

[0016] In a possible implementation of the first aspect above, after the first electronic device receives the input information, the method further includes the first electronic device performing at least one of the following operations: the first electronic device displays a first prompt message that the input information has been received, to remind the user of the first electronic device that the first electronic device has received the input information. The first electronic device displays the input information so that the user of the first electronic device can view the input information on the first electronic device. The first electronic device sends the input information to the management device, and the first electronic device displays a second prompt message that the input information has been sent to the management device, to remind the user of the first electronic device that the input information has been sent to the management device. The first electronic device sends the identification information of the second electronic device to the management device so that the management device knows the input device of the input information. The identification information of the second electronic device can be device information such as the product number of the second electronic device, or can be user information such as the user account of the second electronic device, or can be network information of the network where the second electronic device is located.

[0017] In a possible implementation of the first aspect, the method further includes: the first electronic device sends a collaboration request to the second electronic device, the collaboration request is used to request the second electronic device to collaborate on information input; the second electronic device receives the collaboration request, generates and sends a response message of agreeing to collaborate to the first electronic device; the first electronic device receives the response message, and sends the interface view data to the second electronic device according to the response message. That is, after the first electronic device and the second electronic device complete the collaborative negotiation according to the collaboration request, the first electronic device sends the interface view data to the second electronic device.

[0018] In a possible implementation of the first aspect above, the first electronic device sends the collaboration request to the second electronic device, which may be that after the first electronic device displays the target interface, the first electronic device directly sends the collaboration request to the second electronic device.

[0019] In a possible implementation of the first aspect above, the second electronic device may directly generate response information according to the collaboration request.

[0020] In a possible implementation of the first aspect, the method further includes: after the first electronic device displays the target interface, the first electronic device displays a collaborative start control; the first electronic device detects a confirmation start trigger operation of the user of the first electronic device on the collaborative start control, and sends a collaborative request to the second electronic device. That is, after receiving the confirmation start trigger operation of the user on the collaborative start control, the first electronic device sends a collaborative request to the second electronic device.

[0021] In a possible implementation of the first aspect above, the method also includes: the second electronic device receives the collaboration request, generates and displays a collaboration confirmation control; the second electronic device detects a confirmation trigger operation of the collaboration confirmation control by a user of the second electronic device, and generates a response message.

[0022] In a possible implementation of the first aspect, the collaborative request includes identification information of the first electronic device; the second electronic device authenticates the first electronic device based on the identification information, and if the authentication is successful, the second electronic device generates response information. The identification information of the first electronic device may be device information such as a product number of the first electronic device, or user information such as a user account of the first electronic device, or network information of the network where the first electronic device is located.

[0023] In a possible implementation of the first aspect above, the method further includes: the second electronic device receives a confirmation sending instruction, and sends the input information to the first electronic device.

[0024] In a possible implementation of the first aspect above, after obtaining the input information, the second electronic device displays a send selection control; the second electronic device receives a confirm-send instruction through a confirm-send trigger operation of the send selection control by a user of the second electronic device.

[0025] In a possible implementation of the first aspect above, the method further includes: the first electronic device and the second electronic device transmit information such as the aforementioned interface view data between the two via their respective distributed communication modules.

[0026] In a second aspect, an embodiment of the present application provides a collaborative method between electronic devices, which is applied to a first electronic device, and the first electronic device and a second electronic device establish a communication connection, the method comprising: the first electronic device displays a target interface, and the target interface is an interface for a user of the first electronic device to input information; the first electronic device sends interface view data corresponding to the target interface to the second electronic device, so that the second electronic device generates and displays a collaborative interface corresponding to the target interface according to the interface view data, and the second electronic device receives the information input operation of the user of the second electronic device on the collaborative interface to obtain input information; the first electronic device receives the input information sent by the second electronic device.

[0027] In a possible implementation of the second aspect above, the method further includes: the first electronic device runs an application and displays an application service interface; and the target interface is a partial interface of the application service interface.

[0028] In a possible implementation of the second aspect above, the method further includes: the first electronic device obtains application service data corresponding to the application from the management device for displaying the application service interface, and obtains interface view data corresponding to the target interface.

[0029] In a possible implementation of the second aspect above, the application is a video playback application, and the method further includes: when the first electronic device displays the target interface, the first electronic device stops playing the video; after the first electronic device receives the input information, the first electronic device plays the video.

[0030] In a possible implementation of the second aspect above, the information input operation includes any one of the following: a text input operation; an operation of selecting an option; a file adding operation; or a voice input operation.

[0031] In a possible implementation of the second aspect above, after the first electronic device receives the input information, the method also includes the first electronic device performing at least one of the following operations: the first electronic device displays a first prompt message that the input information has been received; the first electronic device displays the input information so that the user can display the input information on the first electronic device; the first electronic device sends the input information to the management device, and the first electronic device displays a second prompt message that the input information has been sent to the management device; the first electronic device sends the identification information of the second electronic device to the management device.

[0032] In a possible implementation of the second aspect above, the method also includes: the first electronic device sends a collaboration request to the second electronic device, the collaboration request is used to request the second electronic device to collaborate on information input; the first electronic device receives response information generated according to the collaboration request and sent by the second electronic device, and sends interface view data to the second electronic device according to the response information.

[0033] In a possible implementation of the second aspect above, the collaboration request includes identification information of the first electronic device.

[0034] In a possible implementation of the second aspect above, the method also includes: after the first electronic device displays the target interface, the first electronic device displays a collaborative start-up control; the first electronic device detects a user's confirmation start-up trigger operation on the collaborative start-up control, and sends a collaborative request to the second electronic device.

[0035] The collaborative method between electronic devices provided in this embodiment is a collaborative method between electronic devices executed by a first electronic device corresponding to the collaborative method between electronic devices provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect, and therefore can also achieve the beneficial effects (or advantages) possessed by the collaborative method between electronic devices provided by the first aspect.

[0036] In a third aspect, an embodiment of the present application provides a collaborative method between electronic devices, which is applied to a second electronic device, and the second electronic device establishes a communication connection with a first electronic device, the method comprising: the second electronic device receives interface view data sent by the first electronic device, the interface view data is interface view data corresponding to a target interface displayed by the first electronic device, and the target interface is an interface for a user of the first electronic device to input information; the second electronic device generates and displays a collaborative interface corresponding to the target interface based on the interface view data; the second electronic device receives the information input operation of the user of the second electronic device on the collaborative interface to obtain input information; the second electronic device sends the input information to the first electronic device.

[0037] In a possible implementation of the third aspect above, the information input operation includes any one of the following: a text input operation; an operation of selecting an option; a file adding operation; or a voice input operation.

[0038] In a possible implementation of the third aspect above, the method also includes: the second electronic device detects a trigger operation of the user on the collaborative interface, generates and displays an information input control; and the second electronic device receives the user's information input operation on the collaborative interface through the information input control.

[0039] In a possible implementation of the third aspect above, the method further includes: the second electronic device receives a collaboration request sent by the first electronic device, and generates and sends a collaboration response to the first electronic device according to the collaboration request.

[0040] In a possible implementation of the third aspect above, the second electronic device authenticates the first electronic device according to the identification information included in the collaboration request, and if the authentication is successful, the second electronic device generates response information.

[0041] In a possible implementation of the third aspect above, the method also includes: the second electronic device receives the collaboration request of the first electronic device, generates and displays a collaboration confirmation control; the second electronic device detects the user's confirmation trigger operation on the collaboration confirmation control, and generates a response message.

[0042] In a possible implementation of the third aspect above, the method further includes: the second electronic device receives a confirmation sending instruction from a user of the second electronic device, and sends the input information to the first electronic device.

[0043] In a possible implementation of the third aspect, after obtaining the input information, the second electronic device displays a send selection control; the second electronic device receives a confirm-send instruction through a confirm-send trigger operation of the send selection control by a user of the second electronic device.

[0044] The collaborative method between electronic devices provided in this embodiment is a collaborative method between electronic devices executed by a second electronic device corresponding to the collaborative method between electronic devices provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect, and therefore can also achieve the beneficial effects (or advantages) possessed by the collaborative method between electronic devices provided by the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a collaborative system, which includes at least a first electronic device and a second electronic device, and the first electronic device and the second electronic device establish a communication connection, wherein: the first electronic device is used to display a target interface; the target interface is an interface for a user of the first electronic device to perform information input operations; the first electronic device is used to send interface view data corresponding to the target interface to the second electronic device; the second electronic device is used to receive the interface view data, and generate and display a collaborative interface corresponding to the target interface based on the interface view data; the second electronic device is used to receive the information input operation of the user of the second electronic device on the collaborative interface to obtain input information; the second electronic device is used to send the input information to the first electronic device.

[0046] The collaborative system provided in the present application is used to execute the collaborative method between electronic devices provided by the above-mentioned first aspect and / or any possible implementation method of the first aspect, and thus can also achieve the beneficial effects (or advantages) of the collaborative method between electronic devices provided by the first aspect.

[0047] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing a computer program, the computer program including program instructions; a control component for executing the program instructions so that the electronic device performs a collaborative method between electronic devices as provided in the above-mentioned first aspect and / or any possible implementation of the first aspect; or a collaborative method between electronic devices as provided in the above-mentioned second aspect and / or any possible implementation of the second aspect; or a collaborative method between electronic devices as provided in the above-mentioned third aspect and / or any possible implementation of the third aspect.

[0048] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a computer to enable the computer to execute a collaborative method between electronic devices as provided in the above-mentioned first aspect and / or any possible implementation of the first aspect; or a collaborative method between electronic devices as provided in the above-mentioned second aspect and / or any possible implementation of the second aspect; or a collaborative method between electronic devices as provided in the above-mentioned third aspect and / or any possible implementation of the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solution of the present application, the drawings used in the description of the implementation methods are briefly introduced below.

[0050] Figure 1 is a system schematic diagram showing an application scenario of the collaboration method between electronic devices provided by the present application according to some implementation modes of the present application;

[0051] Figure 2A is a schematic diagram showing the structure of a mobile phone according to some implementation modes of the present application;

[0052] Figure 2B is a schematic diagram showing the structure of a television according to some implementation modes of the present application;

[0053] Figure 2C According to some implementation methods of the present application, a software framework structure of a mobile phone and a television, and a schematic diagram of information flow between the mobile phone, the television, and a server are shown;

[0054] Figures 3A-3I is a schematic diagram showing interfaces of some mobile phones and televisions according to some implementation modes of the present application;

[0055] Figures 4A-4C is a schematic diagram showing interfaces of other mobile phones and televisions according to some implementation modes of the present application;

[0056] Figure 5 is a schematic diagram showing another interface between a mobile phone and a television according to some implementation modes of the present application;

[0057] Figure 6 is a system schematic diagram showing another application scenario of the collaboration method between electronic devices provided by the present application according to some embodiments of the present application;

[0058] Figure 7 is a schematic diagram of an interface of a mobile phone and a television according to some embodiments of the present application, showing another application scenario of the collaboration method between electronic devices provided by the present application;

[0059] Figure 8 is a schematic diagram showing the structure of an electronic device according to some embodiments of the present application;

[0060] Fig. 9 A schematic diagram of the structure of a system on chip (SoC) is shown according to some embodiments of the present application. DETAILED DESCRIPTION

[0061] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0062] In the process of using electronic devices, there is a problem that the way in which users directly input information into the electronic devices cannot well meet the user's usage needs. The present application provides a collaborative method between electronic devices. In the process of using a first electronic device, the information input of the first electronic device can be coordinated by a second electronic device to realize convenient and fast information input into the first electronic device, thereby improving the user's experience.

[0063] The first electronic device can typically be a large-screen device such as a television or a smart screen. In addition, the first electronic device can also be other devices such as an air conditioner or a refrigerator.

[0064] The second electronic device may typically be a mobile phone, a tablet computer, a laptop computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), or the like.

[0065] In order to illustrate the collaborative method between electronic devices provided by the present application, the present application provides the following Figure 1 The collaborative system shown is a carrier of the collaborative method between electronic devices provided by the present application.

[0066] See also Figure 1The collaborative system includes a mobile phone 100, a TV 200 and a server 300, wherein the TV 200 and the mobile phone 100 can establish a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network) communication connection through a router (not shown in the figure). The TV 200 and the server 300 can establish a communication connection through the router, and the TV 200 can access the server 300 to obtain video data such as video streaming data required by the TV 200 from the server 300.

[0067] In the present application, the server 300 may be a cloud server. In addition, the server 300 may also be other devices such as a PC.

[0068] The above-mentioned method of establishing a communication connection between the television 200 and the mobile phone 100 may also be to establish a communication connection through Bluetooth (BT) or other methods.

[0069] In the collaborative method between electronic devices provided in the present application, taking the scenario of watching online video courses as an example, after the TV 200 and the mobile phone 100 establish a communication connection, the TV 200 and the mobile phone 100 can collaborate in information input. For example, during the process of the TV 200 playing the online video course, the user is required to interact with online questions and answers, then the TV 200 displays the answer sheet related to the online video course on the video playback interface, and the user can input information on the answer sheet. The interface of the answer sheet is an example of the target interface, which is an interface that can be used for the user to input information. After the TV 200 displays the target interface on the video playback interface, the TV 200 sends the interface view data corresponding to the target interface to the mobile phone 100. The mobile phone 100 receives the interface view data, generates a collaborative interface corresponding to the target interface according to the interface view data, and the mobile phone 100 displays the collaborative interface. The collaborative interface is a collaborative answer sheet corresponding to the answer sheet. The interface elements (such as text, pictures, etc.) included in the collaborative answer sheet are the same as the interface elements included in the answer sheet, and the layout of the interface elements on the collaborative answer sheet is the same as the layout of the interface elements on the answer sheet. The proportion of the collaborative answer sheet is smaller than that of the answer sheet to adapt to the screen of the mobile phone 100. The user can perform information input operations on the collaborative answer sheet. The mobile phone 100 receives the user's information input operation on the collaborative answer sheet to obtain input information, and then the mobile phone 100 sends the input information to the television 200 to achieve the coordination of the mobile phone 100 with the information input of the television 200, so as to achieve convenient and fast information input to the television 200 and improve the user experience.

[0070] It should be noted that the application scenarios of this application are not limited to Figure 1The system and scenario shown may also be a collaborative process of information input by other devices other than the mobile phone 100 and the TV 200 or other electronic devices using the collaborative method according to the present application. The target interface may be an answer sheet displayed by the TV 200 in a scenario where the user is watching an online video course, or an information input interface displayed by the TV 200 for the user to input information in a scenario where the user is watching other videos.

[0071] See also Figure 2A , Figure 2A A schematic diagram of the structure of a mobile phone 100 in one implementation of the present application is shown.

[0072] The mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0073] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than those shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0074] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0075] The processor can generate operation control signals based on instruction opcodes and timing signals to complete the control of instruction fetching and execution.

[0076] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0077] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0078] The mobile communication module 150 can provide wireless communication functions including 2G / 3G / 4G / 5G etc. applied on the mobile phone 100 .

[0079] The wireless communication module 160 can provide wireless communication solutions including WLAN (Wi-Fi), BT, etc. applied on the mobile phone 100.

[0080] In some embodiments, the antenna 1 of the mobile phone 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0081] The mobile phone 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0082] The display screen 194 is used to display images, videos, etc. In some embodiments, the mobile phone 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0083] The mobile phone 100 can realize the shooting function and the video call function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. In some embodiments, the mobile phone 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0084] Video codecs are used to compress or decompress digital videos. Mobile phone 100 can support one or more video codecs. In this way, mobile phone 100 can play or record videos in multiple coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc. In addition, mobile phone 100 can realize the packaging and playback of audio and video data and video data.

[0085] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, the intelligent cognition of the mobile phone 100 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0086] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile phone 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0087] For example, the instructions stored in the internal memory 121 can enable the mobile phone 100 to perform the following steps: after receiving the interface view data corresponding to the aforementioned answer sheet, the mobile phone 100 generates a collaborative answer sheet corresponding to the answer sheet as a collaborative interface according to the interface view data, and the mobile phone 100 displays the collaborative answer sheet through the display screen 194. In addition, the mobile phone 100 receives the answer input by the user on the collaborative answer sheet, and sends the input answer to the television 200 through the wireless communication module 160, so as to realize the collaboration of the mobile phone 100 with the answer input of the television 200.

[0088] The mobile phone 100 can implement audio functions such as video playback, music playback, etc. through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0089] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. When a touch operation is applied to the display screen 194, the mobile phone 100 detects the touch operation intensity according to the pressure sensor 180A. The mobile phone 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A.

[0090] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the mobile phone 100, which is different from the position of the display screen 194.

[0091] In the embodiment of the present application, the mobile phone 100 can detect the user's trigger operation on the display screen 194 based on the pressure sensor 180A and the touch sensor 180K, which is not limited in this embodiment.

[0092] See also Figure 2B , Figure 2B A schematic diagram of the structure of a television 200 in one implementation of the present application is shown.

[0093] The TV 200 may include a processor 210 , a communication module 220 , an audio module 230 , a display screen 240 , a power module 250 , and an internal memory 260 .

[0094] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, etc. Different processing units may be independent devices or integrated into one or more processors. The processor may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0095] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0096] The communication module 220 can provide wireless communication solutions including WLAN (Wi-Fi), BT, etc. applied on the TV 200.

[0097] The mobile phone 100 implements the display function through a GPU, a display screen 240, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 240 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

[0098] The TV 200 can implement audio functions such as video playback and music playback through the audio module 230 and the application processor.

[0099] The TV 200 can power itself through the power module 250 .

[0100] The internal memory 260 can be used to store computer executable program codes, which include instructions. The internal memory 260 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data (such as video data) created during the use of the TV 200, etc. In addition, the internal memory 260 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 260 executes various functional applications and data processing of the TV 200 by running instructions stored in the internal memory 260 and / or instructions stored in a memory provided in the processor.

[0101] For example, the instructions stored in the internal memory 121 can enable the TV 200 to perform the following steps: when the TV 200 includes a target interface on the video playback interface of the TV 200, the interface view data of the target interface is sent to the mobile phone 100 through the communication module 220, and the TV 200 receives the input information sent by the mobile phone 100 through the communication module 220 to obtain the input information.

[0102] It is understandable that the structure shown in the embodiment of the present invention does not constitute a limitation of the television 200. In other embodiments of the present application, the television 200 may include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently.

[0103] See also Figure 2C , Figure 2CThe software structure block diagram of the mobile phone 100 and the television 200 in one implementation of the present application is shown.

[0104] It should be noted that, in the present application, the systems of the mobile phone 100 and the television 200 may be Android systems or other types of systems.

[0105] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system of the mobile phone 100 and the TV 200 may include an application layer and a system layer, wherein the application layer may include a series of application packages; the system layer may provide an application programming interface (API) and a programming framework for the application programs in the application layer, as well as other modules for implementing system functions.

[0106] The application layer of the mobile phone 100 includes a collaborative interface module 1100 and a first distributed communication module 1200, and the application layer of the television 200 includes an online video course application 2100 and a second distributed communication module 2200. The online video course application 2100 may also be other video playback applications.

[0107] It should be noted that the mobile phone 100 and the television 200 have distributed communication capabilities by respectively setting up distributed communication modules, and the mobile phone 100 and the television 200 can transmit information between them based on the distributed communication capabilities.

[0108] In one implementation of the present application, the collaborative method between electronic devices can be applied to a scenario in which a user watches an online video course through a TV 200. During the process of playing an online video course on the TV 200, if the teacher asks a question and the user needs to answer it online, the answer sheet is displayed on the video playback interface of the online video course of the TV 200 for the user to enter the answer, that is, the answer sheet is used as a target interface for the user to enter information. To facilitate the user to enter the answer, the TV 200 sends the interface view data of the answer sheet corresponding to the answer sheet to the mobile phone 100. The mobile phone 100 receives the interface view data, generates a collaborative answer sheet corresponding to the answer sheet as a collaborative interface according to the interface view data, and the mobile phone 100 displays the collaborative answer sheet. The collaborative answer sheet has the same content as the answer sheet, and the user can enter the answer on the collaborative answer sheet. The mobile phone 100 receives the answer entered by the user on the collaborative answer sheet, and then the mobile phone 100 sends the answer to the TV 200 to achieve the collaboration of the mobile phone 100 with the answer input of the TV 200.

[0109] Please continue to see Figure 2CIn one implementation of the present application, in a scenario where a user watches an online video course through a TV 200, the user implements a collaborative process of inputting answers to the TV 200 through a mobile phone 100, including:

[0110] S101, the television 200 establishes a communication connection with the server 300, the user opens the online video course application 2100 in the television 200 to watch the online video course, and the online video course application 2100 accesses the server 300. The server 300 sends the streaming media data of the online video course video and the interface view data of all answer sheets included in the online video course to the online video course application 2100. The online video course application 2100 plays the online video course according to the streaming media data.

[0111] The streaming media data is the video data of the online video course (or can also be called application service data, which is used for the mobile phone 100 to display the video playback interface of the corresponding online video course). The streaming media data may include the image information of the teacher giving the lecture and the courseware information (such as PPT), etc. The interface view data includes the information of various elements such as text information and picture information on the answer sheet, as well as the layout information of each element, etc.

[0112] The user may open the online video course application 2100 in the TV 200 by using a remote control.

[0113] S102, the online video course application 2100 presents the video playback interface view of the online video course during the playback of the online video course according to the streaming media data. The video playback interface view has different contents corresponding to different time points during the video playback process. In the scenario of the online video course, for example, the teacher asks a question at a certain moment and the user needs to answer the question online. The teacher can trigger the display of the answer sheet through the function provided by the online video course application. Then the TV 200 displays the answer sheet for the user to input information on the video playback interface of the online video course. The online video course application 2100 sends the interface view data of the answer sheet to the second distributed communication module 2200 of the TV 200. The interface view data of the answer sheet can be sent as a data packet independent of the aforementioned streaming media data.

[0114] In this implementation, after the TV 200 establishes a communication connection with the mobile phone 100, it can be assumed that the TV 200 and the mobile phone 100 have enabled the collaborative processing function of information input. When the TV 200 displays the answer sheet, the TV 200 needs the mobile phone 100 to perform collaborative processing of information input, and the TV 200 and the mobile phone 100 can directly perform collaborative processing of information input, that is, when the TV 200 displays the answer sheet, the online video course application 2100 can send the interface view data of the answer sheet to the second distributed communication module 2200 in the TV 200, and after step S102, the TV 200 can directly execute step S103.

[0115] In addition, after the answer sheet is displayed on the online video course video playback interface, the online video course application 2100 can temporarily stop playing the online video course.

[0116] S103, the second distributed communication module 2200 in the TV 200 sends the interface view data of the answer sheet to the first distributed communication module 1200 in the mobile phone 100 through the distributed communication between the TV 200 and the mobile phone 100. The interface view data includes the text information, picture information and other interface element information of the answer sheet, as well as the layout information of each interface element, so that the mobile phone 100 can reconstruct the collaborative answer sheet corresponding to the answer sheet on the mobile phone 100 side according to the interface view data. The interface elements (such as text, pictures, etc.) included in the collaborative answer sheet are the same as the interface elements included in the answer sheet, and the layout of the interface elements on the collaborative answer sheet is the same as the layout of the interface elements on the answer sheet, and the proportion of the collaborative answer sheet is smaller than that of the answer sheet to adapt to the screen of the mobile phone 100, and the user can answer questions on the collaborative answer sheet.

[0117] S104, the first distributed communication module 1200 in the mobile phone 100 sends the interface view data of the answer sheet to the collaborative interface module 1100 in the mobile phone 100. The collaborative interface module 1100 performs view rendering processing according to the interface view data of the answer sheet to generate a collaborative answer sheet corresponding to the answer sheet, and displays the collaborative answer sheet on the screen of the mobile phone 100. The collaborative interface module 1100 receives the user's information input operation on the collaborative answer sheet, that is, performs the answer input operation, and obtains the answer as the input information.

[0118] When generating a collaborative answer sheet, the collaborative interface module 1100 may render a view based on the text information, image information and other element information of the answer sheet in the interface view data, as well as the layout information of each element, to obtain a collaborative answer sheet that is the same as the answer sheet. When generating a collaborative answer sheet, the collaborative interface module 1100 may also adjust the proportion or layout of the collaborative answer sheet relative to the answer sheet according to the screen size of the mobile phone 100, so that the mobile phone 100 can fully display the text information, image information and other interface element information on the answer sheet. For example, the collaborative interface module 1100 may reduce the collaborative answer sheet to one-tenth of the answer sheet, or adjust the text displayed on the same line on the answer sheet to be displayed in three lines on the collaborative answer sheet, etc. It can be adjusted as needed.

[0119] If the answer sheet is a multiple-choice type answer sheet including multiple options, the mobile phone 100 can receive a user's click operation on the option, and the mobile phone 100 can obtain the answer input by the user according to the trigger position of the user's click operation. The answer can be, for example, English letters such as "A" (or B, C, D), or numbers such as "1". Alternatively, the mobile phone 100 can receive a user's check operation on the option, and obtain the answer input by the user according to the trigger position of the user's check operation, or obtain the answer input by the user through image recognition by parsing the interface of the collaborative answer sheet. In addition, the check operation can also be a drawing operation.

[0120] If the answer sheet is a short-answer type answer sheet including an input box, the mobile phone 100 can receive the text input operation of the user in the input box, and the mobile phone 100 can obtain the text answer input by the user.

[0121] After obtaining the answer as input information, the mobile phone 100 stores the answer in text form.

[0122] S105 , the collaborative interface module 1100 in the mobile phone 100 sends the answer to the input information to the first distributed communication module 1200 in the mobile phone 100 .

[0123] After receiving the user's confirmation sending instruction, the collaborative interface module 1100 in the mobile phone 100 sends the answer to the input information to the first distributed communication module 1200 in the mobile phone 100.

[0124] S106 , the first distributed communication module 1200 in the mobile phone 100 sends the answer to the input information to the second distributed communication module 2200 in the TV 200 .

[0125] S107 , the second distributed communication module 2200 in the TV 200 sends the answer as the input information to the online video course application 2100 in the TV 200 .

[0126] After receiving the input information, the online video course application 2100 may display the answer to the input information on the screen of the TV 200, for example, displaying "the answer of the mobile phone 100 is "A"".

[0127] In addition, after receiving the answer as input information, the online video course application 2100 may further execute step S108.

[0128] S108 , the online video course application 2100 sends the answer as the input information to the server 300 .

[0129] After the server 300 obtains the answer as the input information, the answer can be stored for viewing by teachers and other relevant personnel.

[0130] For step S108, the online video course application 2100 in the TV 200 may only send the answer input by the user to the server 300. In other implementations of the present application, the online video course application 2100 in the TV 200 may also send the answer and the identification information of the mobile phone 100 to the server 300, so that the teacher can know who the answerer is through the identification information of the mobile phone 100. The identification information of the mobile phone 100 may be, for example, device identification information such as the International Mobile Equipment Identity (IMEI) of the mobile phone, or user information such as the user account of the mobile phone 100, for example, the device-associated user account of the mobile phone 100, or the account information of the online video course.

[0131] The television 200 may determine the identification information of the mobile phone 100 according to the transmission link of the answer. In addition, the mobile phone 100 may also send the identification information of the mobile phone 100 to the television 200 when sending the answer to the television 200.

[0132] In this implementation, the mobile phone 100 and the television 200 establish a communication connection, which can be that after the mobile phone 100 and the television 200 are turned on and the device discovery function is turned on, the mobile phone 100 and the television 200 respectively perform device discovery to establish a communication connection. In addition, the mobile phone 100 and the television 200 establish a communication connection, which can also be that after the aforementioned step S102, after the second distributed communication module 2200 in the television 200 receives the interface view data of the answer sheet, the television 200 sends a connection request to the mobile phone 100 through the second distributed communication module 2200 to establish a communication connection. After the first distributed communication module 1200 in the mobile phone 100 receives the connection request, if it is determined to establish a communication connection with the television 200, it sends a response to agree to establish communication to the second distributed communication module 2200 in the television 200. After receiving the response to agree to establish communication, the television 200 establishes a communication connection with the mobile phone 100.

[0133] In another implementation of the present application, after the TV 200 establishes a communication connection with the mobile phone 100, if the TV 200 and the mobile phone 100 are not enabled by default for collaborative processing of information input, when the TV 200 displays an answer sheet and the TV 200 needs the mobile phone 100 to perform collaborative processing of information input, the TV 200 and the mobile phone 100 need to negotiate to determine whether to perform collaborative processing of the information input operation.

[0134] That is, for step S103, before the second distributed communication module 2200 in the TV 200 sends the interface view data of the answer sheet to the first distributed communication module 1200 in the mobile phone 100 through the distributed communication between the TV 200 and the mobile phone 100, the TV 200 also sends a coordination request to the mobile phone 100, the coordination request is used to request the mobile phone 100 to perform coordinated processing of information input, and the coordination request includes the identification information of the TV 200. The identification information can be the device identification information of the TV 200, such as the model, product serial number and other information, or the network identification information of the network where the TV 200 is located, or it can be the account information of the user of the TV 200, etc.

[0135] The mobile phone 100 receives the collaboration request and authenticates the TV 200 according to the identification information. If the authentication is successful, the mobile phone 100 determines to collaborate, and the mobile phone 100 sends a response message of agreeing to collaborate to the TV 200. After the TV 200 receives the response message of agreeing to collaborate, it sends the interface view data of the answer sheet to the mobile phone 100 according to the response message. If the authentication is not successful, the mobile phone 100 determines that it does not agree to collaborate, and the mobile phone 100 sends a response message of not agreeing to collaborate to the TV 200. After the TV 200 receives the response message of not agreeing to collaborate, it does not need to send the interface view data of the answer sheet to the mobile phone 100.

[0136] Exemplarily, the mobile phone 100 authenticates the TV 200 according to the identification information of the TV 200, and the mobile phone 100 compares the device identification information of the TV 200 with the trusted device identification information of the trusted device stored in the mobile phone 100, and determines whether the device identification information exists in the trusted device identification information list, so as to determine whether the TV 200 is a trusted device of the mobile phone 100. If the device identification information of the TV 200 exists in the trusted device identification information list, the TV 200 is a trusted device and the authentication is passed. If the device identification information of the TV 200 does not exist in the trusted device identification information list, the TV 200 is not a trusted device of the mobile phone 100, and the authentication is not passed.

[0137] The mobile phone 100 may authenticate the television 200 by other means, which may be selected as needed.

[0138] In another implementation of the present application, the mobile phone 100 confirms whether to collaborate, which may be that after receiving the collaboration request sent by the television 200, if the mobile phone 100 receives a user's instruction to confirm collaboration, the mobile phone 100 confirms collaboration; if the mobile phone 100 receives a user's instruction to confirm not to collaborate, the mobile phone 100 confirms not to collaborate. The mobile phone 100 receives the user's instruction to confirm whether to collaborate, which may be through the user's confirmation of the triggering operation of the collaboration confirmation control displayed on the screen of the mobile phone 100, or through the confirmation of the received user's voice input or the like.

[0139] The collaborative method between electronic devices provided by this implementation is that in the scenario where a user needs to answer questions online while watching an online video course through a TV 200, the TV 200 can send the answer sheet for the user to input the answer to the mobile phone 100 for display, and the user can enter the answer on the collaborative answer sheet corresponding to the answer sheet displayed on the mobile phone 100, and then the mobile phone 100 sends the answer input by the user to the TV 200 to complete the collaborative processing of the answer input of the TV 200 by the mobile phone 100. Compared with the method in the prior art where the user inputs the answer to the TV 200 through the remote control, the user can input the answer through the mobile phone 100 without the need for additional remote control. With the popularization of mobile phones 100, users can directly input information and control the TV 200 through the mobile phone 100 without the need to configure a remote control separately, which is more in line with the development needs of current smart electronic devices and the needs for more convenient and fast interaction between devices.

[0140] In addition, if the question on the answer sheet is a multiple-choice question type, the answer sheet includes multiple options and answers corresponding to each option. The user can click on the option corresponding to the answer through the mobile phone 100 to enter the answer, or the user can draw a line or circle the option corresponding to the answer to enter the answer. Compared with the method of selecting the answer to the question through the direction keys and the confirmation key of the remote control (the principle is similar to selecting a TV program through the remote control), the operation is more convenient.

[0141] If the question on the answer sheet is a fill-in-the-blank question type, the answer sheet includes the question and an answer input area (such as an input box) for the user to input the answer, and the user can enter text or picture input to enter the answer by clicking the answer input area. The speed at which the user enters the answer in the form of text or the like through the mobile phone 100 is faster than the speed at which the user enters the answer in the form of text or the like through the remote control, and the operation is more convenient, which can effectively improve the user experience.

[0142] The answer sheet may also include other types of questions. The user may input various types of input information by triggering or touching the mobile phone 100 , which may provide support for enriching the types of questions in online video courses.

[0143] In another implementation of the present application, for S101, the server 300 may only send the streaming media data of the online video course video to the TV 200. For S102, while the online video course application 2100 plays the online video course according to the streaming media data, the TV 200 may perform image recognition on the video playback interface of the online video course through an image recognition method to determine whether there is an answer sheet. If there is an answer sheet, the TV 200 obtains the interface view data of the answer sheet from the video playback interface of the online video course through image recognition.

[0144] In another implementation of the method for collaboration between electronic devices provided in the present application, the collaboration of the mobile phone 100 and the television 200 for information input includes the following process:

[0145] See also Figure 3A , the user turns on the TV 200 , and the screen of the TV 200 displays the online video course application 2100 and other applications such as games, music, etc.

[0146] If the user needs to learn online video courses, the user opens the online video course application 2100 through the remote control. After detecting the opening operation of the online video course application 2100 by the remote control, the TV opens (or starts) the online video course application 2100.

[0147] At this time, the mobile phone 100 can display Figure 3A The desktop interface is shown.

[0148] After the TV 200 opens the online video course application 2100, Figure 3B As shown, the TV 200 displays a collaborative start control 20 on the screen. The collaborative start control 20 is used for the user to select whether the mobile phone 100 needs to cooperate in information input. The collaborative start control 20 may include an explanatory message "Turn on collaboration with the mobile phone 100", as well as a "Cancel" control and a "Confirm" control, and may also include other information, which may be set as needed.

[0149] It should be noted that the user can use the remote control to trigger the “cancel” control and the “confirm” control included in the collaborative opening control 20 .

[0150] If the TV 200 detects that the user triggers the "confirm" control through the remote control, the TV 200 sends a collaboration request to the mobile phone 100. In addition, after the TV 200 detects that the user triggers the "confirm" control through the remote control, the online video course may be played or not.

[0151] If the TV 200 detects that the user triggers the "Cancel" control, the TV 200 does not send a collaboration request to the mobile phone 100, and the TV 200 directly plays the online video course.

[0152] After the mobile phone 100 receives the cooperation request sent by the TV 200, Figure 3B The interface shown displays a collaboration confirmation control 10, which may include an explanatory message "Agree to collaborate with TV 200", as well as a "Confirm" control and a "Cancel" control. Of course, other information may also be displayed, which may be selected as needed.

[0153] If the mobile phone 100 detects that the user triggers the "confirm" control, the mobile phone 100 sends a response message of agreeing to the collaboration to the TV 200. If the mobile phone 100 detects that the user triggers the "cancel" control, the mobile phone 100 sends a response message of not agreeing to the collaboration to the TV 200.

[0154] In addition, after the mobile phone 100 detects the user's triggering operation on the "confirm" control or the "cancel" control, the mobile phone 100 displays Figure 3C The desktop interface described.

[0155] The negotiation between the mobile phone 100 and the TV 200 can be performed when the online video course starts playing, or it can be triggered during the playing of the online video course. In the case where the negotiation is performed when the online video course starts playing, after the TV 200 receives the response information of agreeing to cooperate, the TV 200 immediately starts playing the online video course; in the case where the negotiation is triggered during the playing of the online video course, the TV 200 is currently playing the online video course. After the TV 200 triggers the negotiation and completes the negotiation, the TV 200 can continue to play the online video course.

[0156] When the TV 200 plays the online video course, all the information of the online video course is displayed on the screen of the TV 200. Figure 3C As shown, the screen of the TV 200 can display courseware information 21 (such as a PPT file image) and image information 22 of the teacher giving the lecture.

[0157] In addition, during the playback of online video courses, if the teacher asks a question that requires the user to answer, Figure 3D As shown, the video playback interface of the online video course displays an answer sheet 23. The answer sheet 23 includes the question "Which of the following is a triangle ()", as well as options A, B, C, D and graphics corresponding to each option.

[0158] To facilitate the user to input answers, after the answer sheet 23 appears on the screen of the TV 200, the TV 200 sends the answer sheet 23 to the mobile phone 100 for display, so that the user can input answers on the mobile phone 100, and then the mobile phone 100 sends the answers to the TV 200.

[0159] The television 200 sends the answer sheet 23 to the mobile phone 100 for display. The television 200 may send the interface view data of the answer sheet 23 to the mobile phone 100, so that the mobile phone 100 generates (or may also be referred to as constructs) the interface view data according to the interface view data. Figure 3D The collaborative answer sheet 11 is shown on the screen of the mobile phone 100. The information and layout displayed on the collaborative answer sheet 11 and the answer sheet 31 can be exactly the same, and the collaborative answer sheet 11 also includes the question "Which of the following is a triangle ()", as well as the options A, B, C, D and the graphics corresponding to each option.

[0160] The user can click on the corresponding area of ​​each option included in the collaborative answer sheet 11 displayed on the screen of the mobile phone 100. When the mobile phone 100 detects the user's touch operation, it is regarded as a selection operation of clicking on the option, and the answer is completed. Figure 3E As shown, after the mobile phone 100 detects the user's click operation on the option "C", the answer "C" is obtained, and the mobile phone 100 displays the answer "C" in the "()" of the collaborative answer sheet 11.

[0161] In one implementation, after the mobile phone 100 detects the answer “C” input by the user, the answer “C” may be directly sent to the television 200 .

[0162] In another implementation, after the mobile phone 100 detects the answer "C" input by the user, it can also display the following Figure 3F The send selection control 12 shown may include a "confirm" control and a "cancel" control. If the mobile phone 100 detects that the user clicks the "confirm" control, the mobile phone 100 receives the confirmation sending instruction, and the mobile phone 100 sends the answer "C" input by the user to the television 200. The processing details of the selection options will be explained later.

[0163] After the mobile phone 100 sends the answer "C" to the television 200, the mobile phone 100 can display the following information: Figure 3G The interface shown includes prompt information 13, which can be "You have completed the test" to remind the user that the test has been completed. Of course, prompt information 13 can also be other information such as "You have selected answer C", which can be selected as needed.

[0164] After the mobile phone 100 sends the answer "C" to the television 200, it can also continue to display Figure 3D In the interface shown, if the TV 200 further sends a new answer sheet to the mobile phone 100, the mobile phone 100 displays the new answer sheet.

[0165] If the mobile phone 100 detects that the user clicks the "Cancel" control in the sending selection control 12, the mobile phone 100 will not send the answer "C" to the TV 200, and on the other hand, the mobile phone 100 can delete the answer "C" entered by the user in "()" so that the user can re-enter a new answer.

[0166] In addition, the user can also directly click on other options to input a new answer. Or the user can click "()" to delete the answer filled in "()". The mobile phone 100 can receive the user's deletion operation on the filled answer and the user's input operation of re-entering a new answer to obtain the user's new answer.

[0167] In addition, if the mobile phone 100 detects that the user has not clicked the "OK" control within the specified time, the mobile phone 100 may also send a notification message to the television 200 indicating that the user has not completed the information input for this question, and Figure 3H As shown, a prompt message 14 may be displayed on the screen of the mobile phone 100 , and the prompt message 14 may be “This question is not completed” to remind the user that he has not answered the question through the mobile phone 100 .

[0168] The prescribed time may be 30 seconds or 60 seconds, or any other time, which may be set as required.

[0169] In one implementation of the present application, after receiving the answer "C" sent by the mobile phone, the TV 200 can directly send the answer "C" to the server 300. The teacher can view the answer input by the user through the server 300.

[0170] In addition, after the TV 200 sends the answer "C" to the server, Fig. 3I As shown, the television 200 may display a prompt message 25, such as "the answer has been sent to the server 300", to inform the user that the answer has been successfully completed.

[0171] In another implementation of the present application, after the TV 200 receives the answer "C" input by the user, the answer may also be displayed on the screen of the TV 200 so that the user can view the answer through the TV 200. Fig. 3I As shown, the TV 200 can display a prompt message 25, which is "the mobile phone 100 selects C". Of course, the TV 200 can also play "you have selected C" by voice, which can be set as needed.

[0172] In the present application, the TV 200 may send only the answer input by the user to the server 300, or the TV 200 may send the answer and the identification information of the mobile phone 100 to the server 300 so that the teacher can know who the answerer is through the server 300.

[0173] It should be noted that in some other implementations of the present application, the user triggers the aforementioned Figure 3B In the scene of the collaborative opening control 10 displayed on the screen of the TV 200 shown, if the screen of the TV 200 is a touch screen, the user can also click the "Cancel" control and the "Confirm" control on the TV 200 to trigger the operation. Of course, if the mobile phone 100 includes a remote control application that can realize the remote control function, the user can also trigger the operation through the remote control application in the mobile phone 100.

[0174] In some other implementations of the present application, the user clicks Figure 3E The collaborative answer sheet 11 displayed on the screen of the mobile phone 100 shown in the figure can also be answered by the user clicking the bracket "()", and the "()" provides a text field for the user to enter text. After the mobile phone 100 detects the user's click operation on the "()", as shown in FIG. Figure 4AAs shown, the mobile phone 100 can display the input keyboard 15, and the user can input the answer through the input keyboard 15. For example, when the user clicks "C" on the input keyboard 15, "C" is displayed in the "()" on the mobile phone 100. The mobile phone 100 transmits the text data "C" input by the user to the TV 200, and the TV 200 then transmits the text data "C" to the server 300.

[0175] In some other implementations of the present application, the user clicks Figure 3E The collaborative answer sheet 11 displayed on the screen of the mobile phone 100 shown in the figure can also be answered by the user clicking the bracket "()". After the mobile phone 100 detects the user's click operation on the "()", as shown in FIG. Figure 4B As shown, the mobile phone 100 can display the scribble pen tool 16. After the user clicks the scribble pen tool 16, the user can use the scribble pen to outline the selection, such as outlining the answer "C". The mobile phone 100 can detect the user's triggering operation of the answer "C" through the scribble pen tool 16, and obtain the input text data "C". The mobile phone 100 then transmits the text data "C" to the server 300 through the television 200.

[0176] In addition, you can Figure 4C As shown, the answer "C" is directly input by handwriting in "()" using the graffiti pen. The mobile phone 100 uses the image recognition method to perform image recognition on the interface of the collaborative answer sheet 11 to obtain the newly added text "C", that is, the text "C" of the answer input by the user through the graffiti pen tool 16 can be obtained, and the mobile phone 100 then transmits the text data "C" to the server 300 through the television 200.

[0177] The image recognition technology may be optical character recognition (OCR) technology or other technologies.

[0178] In some other implementations of the present application, the answer sheet 23 may also be a fill-in-the-blank type answer sheet for the user to input text. Figure 5 As shown, the collaborative answer sheet 11 is displayed on the mobile phone 100. If the mobile phone 100 detects a click operation of the user in the "" area, the mobile phone 100 displays an input keyboard 17 for the user to input text. After the mobile phone 100 obtains the text content input by the user, it sends the text content in the form of text data as an answer to the television 200. The television 200 then transmits the text content in the form of text data to the server 300.

[0179] It should be noted that in the present application, input keyboard 15 and input keyboard 17 are input keyboards corresponding to the built-in input method of mobile phone 100, and their types can be set as needed. Users can switch between multiple modes such as Chinese, English, and handwriting mode through input keyboard 15 and input keyboard 17.

[0180] In other implementations of the present application, the answer sheet 23 may also be an answer sheet for users to add files. Then the user can add files to the collaborative answer sheet 11 through the mobile phone 100, and the mobile phone 100 sends the files to the television 200. The files may be picture files, and the mobile phone 100 sends the picture files to the television 200. The files may also be other files such as Word.

[0181] In some other implementations of the present application, the answer sheet 23 may also be an answer sheet for user voice input. Then a voice input control may be displayed on the answer sheet, and the user may input voice by long pressing the voice input control. After the input is completed, the user releases the voice input control. Then the mobile phone 100 may send the voice data input by the user to the server 300 through the television 200. In addition, the mobile phone 100 may also parse the voice input by the user to obtain text data, and the mobile phone 100 sends the text data to the server 300 through the television 200.

[0182] It should be noted that in the present application, if there is only one user watching the online video course through the TV 200, the TV 200 can only collaborate with the mobile phone 10 for information input of a single device. If there are multiple users watching the online video course through the TV 200, the TV 200 can also establish communication connections with multiple devices at the same time and collaborate on information input.

[0183] like Figure 6 As shown, the TV 200 can establish communication connections with the mobile phone 100, the mobile phone 400, and the PC 500 at the same time, and coordinate information input. That is, the TV 200 can send the answer sheet 23 to the mobile phone 100, the mobile phone 400, and the PC 500 respectively, and the mobile phone 100, the mobile phone 400, and the PC 500 respectively display the collaborative answer sheet 11, and the users of the mobile phone 100, the mobile phone 400, and the PC 500 respectively answer the questions on the mobile phone 100, the mobile phone 400, and the PC 500, and the mobile phone 100, the mobile phone 400, and the PC 500 respectively send the answers to the TV 200. The TV 200 sends all the answers to the server 300, so that the teacher can obtain the answers of all users who watch the online video course.

[0184] It should be noted that, when the TV 200 sends all the answers to the server 300, it also sends the identification information of the device corresponding to each answer to the server 300, so that the teacher can know the answerer and the corresponding answer.

[0185] The collaborative method between electronic devices provided in the present application is applied to the scenario where the TV 200 collaborates with multiple devices. When multiple users use the TV 200 to watch online video courses at the same time, the multiple users can complete the questions through their respective devices, avoiding the problem that multiple users cannot answer questions through the remote control, thereby improving the user experience.

[0186] The collaboration method between electronic devices provided in the present application can also be applied to a scenario where a user conducts a video conference through the TV 200.

[0187] like Figure 7 As shown, the TV 200 opens the video conference application, and the TV 200 displays the image information of the participants (such as participant 1, participant 2, participant 3, etc.), and displays the opinion collection input box 27 as the target interface. Then, the TV 200 sends the interface view data of the opinion collection input box 27 to the mobile phone 100, and the mobile phone 100 generates and displays the collaborative opinion collection box 18. The mobile phone 100 receives the text input operation of the user in the collaborative opinion collection box 18, obtains the user's input information, and sends the input information to the TV 200. After receiving the input information, the TV 200 can also send the input information to the server 300.

[0188] The collaborative method between electronic devices provided in the present application can also be applied to scenarios where users watch other videos such as movies and TV series through TV. For example, when TV 200 plays a movie, and the movie playback interface includes a bullet screen input box as the target interface, TV 200 can send the interface view data corresponding to the bullet screen input box to mobile phone 100, and mobile phone 100 generates a collaborative bullet screen input box corresponding to the bullet screen input box, so that the user can input bullet screen information in the collaborative bullet screen input box, and mobile phone 100 receives the user's input operation in the collaborative bullet screen input box, obtains the bullet screen information input by the user, and sends the bullet screen information to TV 200. After receiving the bullet screen information, TV 200 can also send the bullet screen information to server 300.

[0189] The collaboration method between electronic devices provided in the present application can also be applied to scenarios where the TV 200 needs to input passwords, select channels, and other information that requires the user to input. The present application will not elaborate on the collaboration process between the mobile phone 100 and the TV 200 in these scenarios.

[0190] The collaboration method between electronic devices provided in the present application can realize cross-device collaboration and online interaction between the mobile phone 100 and the TV 200, and can make full use of the convenience of the trigger or touch function of the mobile phone 100 and the superiority of the large-screen display of the TV 200, and achieve better interactive effects through multi-device collaboration.

[0191] The application scenarios of the present application are not limited to the aforementioned collaborative systems and scenarios, and may also be other devices other than the mobile phone 100 and the TV 200 or other electronic devices that use the collaborative method according to the present application to perform collaborative processing of information input. The collaborative method between electronic devices provided in the present application is applied to a scenario where the first electronic device needs to input information, but it is inconvenient for the user to perform information input operations on the first electronic device. The first electronic device can display the target interface for the user to input information on the first electronic device to the second electronic device where the user can conveniently perform input operations, so that the user can input information on the second electronic device, and then the second electronic device sends the input information to the first electronic device, so as to conveniently and quickly complete the information input of the first electronic device.

[0192] In some embodiments of the present application, an electronic device is also provided. Figure 8 The electronic device in the embodiment of the present application is introduced.

[0193] Figure 8 A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. For at least one embodiment, the controller hub 804 communicates with the processor 801 via a multi-drop bus such as a front-end bus (FSB), a point-to-point interface such as a quick path interconnect (QPI), or a similar connection. The processor 801 executes instructions that control general types of data processing operations. In one embodiment, the controller hub 804 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown in the figure) and an input / output hub (IOH) (which can be on a separate chip) (not shown in the figure), wherein the GMCH includes a memory and a graphics controller and is coupled to the IOH.

[0194] The electronic device 800 may also include a coprocessor 806 and a memory 802 coupled to the controller hub 804. Alternatively, one or both of the memory 802 and the GMCH may be integrated within the processor 801 (as described in this application), with the memory 802 and the coprocessor 806 being directly coupled to the processor 801 and the controller hub 804, with the controller hub 804 being in a single chip with the IOH.

[0195] In one embodiment, the memory 802 may be, for example, a dynamic random access memory (DRAM), a phase change memory (PCM), or a combination of the two. The memory 802 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, and more specifically, stores temporary and permanent copies of the instructions.

[0196] In one embodiment, coprocessor 806 is a special purpose processor, such as, for example, a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPU, or an embedded processor, etc. The optional nature of coprocessor 806 is indicated by dashed lines in FIG. Figure 8 middle.

[0197] In one embodiment, the electronic device 800 may further include a network interface (NIC) 803. The network interface 803 may include a transceiver for providing a radio interface for the electronic device 800 to communicate with any other suitable device (such as a front-end module, an antenna, etc.). In various embodiments, the network interface 803 may be integrated with other components of the electronic device 800. The network interface 803 may implement the functions of the communication unit in the above-mentioned embodiments.

[0198] In one embodiment, Figure 8 As shown, the electronic device 800 may further include an input / output (I / O) device 805. The input / output (I / O) device 805 may include: a user interface designed to enable a user to interact with the electronic device 800; a peripheral component interface designed to enable peripheral components to interact with the electronic device 800; and / or a sensor designed to determine environmental conditions and / or location information related to the electronic device 800.

[0199] It is worth noting that Figure 8 This is for illustrative purposes only. Figure 8 It is shown that the electronic device 800 includes multiple devices such as a processor 801, a controller hub 804, and a memory 802. However, in actual applications, the devices using the methods of the present application may only include a part of the devices of the electronic device 800, for example, it may only include the processor 801 and the NIC 803. Figure 8 The properties of optional devices are shown with dotted lines.

[0200] In some embodiments of the present application, the instructions stored in the computer-readable storage medium of the electronic device 800 may include: instructions that, when executed by at least one unit in the processor, cause the device to implement the aforementioned collaborative method between electronic devices. When the instructions are executed on the computer, the computer executes the aforementioned collaborative method between electronic devices.

[0201] See also Fig. 9 , Fig. 9 FIG. 1 is a schematic diagram of the structure of a SoC (System on Chip) 1000 provided according to an embodiment of the present application. Fig. 9 In the embodiment, similar components have the same reference numerals. In addition, the dashed box is an optional feature of a more advanced SoC 1000. The SoC 1000 can be used in any electronic device according to the present application. Depending on the device in which it is located and the instructions stored therein, the corresponding functions can be implemented.

[0202] exist Fig. 9 In the embodiment, SoC 1000 includes: an interconnect unit 1002, which is coupled to a processor 1001; a system agent unit 1006; a bus controller unit 1005; an integrated memory controller unit 1003; a group or one or more coprocessors 1007, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; an SRAM (static random access memory) unit 1008; and a DMA (direct memory access) unit 1004. In one embodiment, the coprocessor 1007 includes a special-purpose processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0203] The SRAM unit 1008 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of the instructions. The instructions may include: when executed by at least one unit in the processor, causing the electronic device to implement the collaborative method between electronic devices as mentioned above.

[0204] Each embodiment of the mechanism disclosed in this application can be implemented in software, hardware, firmware, or a combination of these implementation methods. The embodiments of this application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, memory (or storage system, including volatile and non-volatile memory and / or storage unit).

[0205] It should be noted that the terms “first”, “second”, etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0206] It should be noted that in the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0207] Although the present application has been illustrated and described by referring to certain preferred embodiments of the present application, it should be understood by those skilled in the art that the above contents are further detailed descriptions of the present application in combination with specific embodiments, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. Those skilled in the art may make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. A method for collaboration between electronic devices, applied to a system including a first electronic device and a second electronic device, characterized in that: The method comprises: The first electronic device runs an application and displays an application service interface, wherein the application service interface includes a target interface, and the target interface is a partial interface of the application service interface and is an interface portion of the application service interface for a user to input information; The first electronic device sends the interface view data corresponding to the target interface to the second electronic device; The second electronic device receives the interface view data, generates and displays a collaborative interface corresponding to the target interface according to the interface view data; The second electronic device receives the user's information input operation on the collaborative interface to obtain input information; The second electronic device sends the input information to the first electronic device, so that the second electronic device cooperates with the information input of the first electronic device.

2. The method for collaboration between electronic devices according to claim 1, characterized in that: The interface view data includes interface element information of each interface element on the target interface and layout information of each interface element, and the second electronic device generates a collaborative interface corresponding to the target interface according to the interface view data, including: The second electronic device performs view rendering processing according to the interface view data to generate a collaborative interface corresponding to the target interface, wherein the interface elements included in the collaborative interface are the same as the interface elements included in the target interface, and the layout of the interface elements included in the collaborative interface is the same as the layout of the interface elements included in the target interface.

3. The method for collaboration between electronic devices according to claim 1, characterized in that: The information input operation includes any of the following: Text input operation; Selection operations on selected items; File adding operation; Voice input operation.

4. The method for collaboration between electronic devices according to any one of claims 1 to 3, characterized in that: The method further comprises: The second electronic device detects a triggering operation of the collaborative interface by a user, and generates and displays an information input control; The second electronic device receives an information input operation of a user on the collaborative interface through the information input control.

5. The method for collaboration between electronic devices according to any one of claims 1 to 3, characterized in that: The method further comprises: The second electronic device updates the collaborative interface and displays a second interface.

6. The method for collaboration between electronic devices according to any one of claims 1 to 3, characterized in that: The method further comprises: After the first electronic device displays the target interface, the first electronic device displays a collaborative start control; The first electronic device detects a user's confirmation start trigger operation on the collaborative start control, and sends a collaborative request to the second electronic device.

7. The method for collaboration between electronic devices according to claim 6, characterized in that: The method further comprises: The second electronic device receives the collaboration request, generates and sends a response message of agreeing to collaborate to the first electronic device; The first electronic device receives the response information and sends the interface view data to the second electronic device.

8. A method for collaboration between electronic devices, applied to a first electronic device, characterized in that: The method comprises: The first electronic device runs an application and displays an application service interface, wherein the application service interface includes a target interface, and the target interface is a partial interface of the application service interface and is an interface portion of the application service interface for a user to input information; The first electronic device sends the interface view data corresponding to the target interface to the second electronic device, so that the second electronic device generates and displays the collaborative interface corresponding to the target interface according to the interface view data, and the second electronic device receives the user's information input operation on the collaborative interface to obtain input information; The first electronic device receives the input information sent by the second electronic device.

9. The method for collaboration between electronic devices according to claim 8, characterized in that: The information input operation includes any of the following: Text input operation; Selection operations on selected items; File adding operation; Voice input operation.

10. The method for collaboration between electronic devices according to claim 8 or 9, characterized in that: After the first electronic device displays the target interface, the first electronic device displays a collaborative start control; The first electronic device detects a user's confirmation start trigger operation on the collaborative start control, and sends a collaborative request to the second electronic device.

11. A method for collaboration between electronic devices, applied to a second electronic device, characterized in that: The method comprises: The second electronic device receives the interface view data sent by the first electronic device, where the interface view data is interface view data corresponding to a target interface displayed by the first electronic device, where the target interface is a partial interface of an application service interface displayed by the first electronic device according to a running application, and is an interface portion of the application service interface for a user to input information; The second electronic device generates and displays a collaborative interface corresponding to the target interface according to the interface view data; The second electronic device receives the user's information input operation on the collaborative interface to obtain input information; The second electronic device sends the input information to the first electronic device, so that the second electronic device cooperates with the information input of the first electronic device.

12. The method for collaboration between electronic devices according to claim 11, characterized in that: The information input operation includes any of the following: Text input operation; Selection operations on selected items; File adding operation; Voice input operation.

13. The method for collaboration between electronic devices according to claim 11, characterized in that: The method further comprises: The second electronic device detects a triggering operation of the collaborative interface by a user, and generates and displays an information input control; The second electronic device receives an information input operation of a user on the collaborative interface through the information input control.

14. The method for collaboration between electronic devices according to any one of claims 11 to 13, characterized in that: The method further comprises: The second electronic device updates the collaborative interface and displays a second interface.

15. A first electronic device, characterized in that: include: A memory for storing one or more programs, wherein the one or more programs include program instructions; The processor is used to execute the program instructions so that the first electronic device executes the collaboration method between electronic devices as described in any one of claims 8 to 10.

16. A second electronic device, characterized in that: include: A memory for storing one or more programs, wherein the one or more programs include program instructions; The processor is used to execute the program instructions so that the second electronic device executes the collaboration method between electronic devices as described in any one of claims 11 to 14.

17. A collaborative system, characterized in that: The device comprises the first electronic device as claimed in claim 15 and the second electronic device as claimed in claim 16.

18. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes program instructions, and the program instructions are executed by an electronic device to enable the electronic device to perform a collaborative method between electronic devices as described in any one of claims 8 to 10; or to enable the electronic device to perform a collaborative method between electronic devices as described in any one of claims 11 to 14.

Images