Multi-end interaction methods, electronic device and computer-readable storage medium

Abstract

Disclosed in the present application are multi-end interaction methods, an electronic device, and a computer-readable storage medium. The present application relates to the technical field of communications. A control end is in communication connection with a remote end. After triggering an interaction task with the remote end, the control end will further determine an interaction scenario of the interaction task and, when the type of the interaction scenario is data processing result interaction, generate, on the basis of the interaction task, an interaction instruction for controlling the remote end to process local data, and receive a processing result obtained by processing the local data on the basis of the interaction instruction.

Description

Multi-terminal interaction method, electronic device and computer readable storage medium

[0001] Related applications

[0002] The present application claims priority to Chinese Patent Application No. 202410860585.8, filed on June 28, 2024, the contents of which are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the field of communication technology, and in particular to a multi-terminal interaction method, an electronic device and a computer readable storage medium. BACKGROUND

[0004] With the development of communication technology, the types of intelligent terminal devices are increasingly diverse, and the collaborative work and interaction scenarios of multiple electronic devices are increasingly rich. However, in the prior art, the implementation efficiency of various interaction functions is low for the use scenarios of multi-screen interaction. For example, the file sharing function needs to transmit the original file in the network, and the transmission of the original file will take a certain amount of time. The file transmission time is actually also taken as the response time of the interaction, thereby causing slow multi-terminal interaction response or low interaction efficiency, especially in the case of large files, the interaction response delay is more obvious, which seriously affects the actual experience of the user. SUMMARY

[0005] The main purpose of the present application is to provide a multi-terminal interaction method, an electronic device and a computer readable storage medium.

[0006] To achieve the above-mentioned purpose, the present application provides a multi-terminal interaction method applied to a control terminal, the control terminal being in communication connection with a remote terminal, the multi-terminal interaction method comprising: determining an interaction scenario of an interaction task triggered with the remote terminal; in the case that the type of the interaction scenario belongs to data processing result interaction, generating an interaction instruction for processing local data of the remote terminal based on the interaction task; sending the interaction instruction to the remote terminal and receiving a processing result obtained by the remote terminal based on the interaction instruction for processing the local data.

[0007] To achieve the above-mentioned purpose, the present application further provides a multi-terminal interaction method applied to a remote terminal, the remote terminal being in communication connection with a control terminal, the multi-terminal interaction method comprising: receiving an interaction instruction sent by the control terminal, wherein the interaction instruction is generated by the control terminal based on an interaction task in the case that the type of an interaction scenario of the triggered interaction task belongs to data processing result interaction; processing local data pointed to by the interaction instruction based on the interaction instruction to obtain a processing result; and feeding back the processing result to the control terminal.

[0008] In addition, to achieve the above object, the present application also provides an electronic device, comprising a memory, a processor, and a multi-terminal interaction program stored in the memory and executable on the processor, wherein the multi-terminal interaction program is executed by the processor to implement the multi-terminal interaction method as described above.

[0009] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a multi-terminal interaction program, and the multi-terminal interaction program is executed by a processor to implement the multi-terminal interaction method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings incorporated in and forming a part of the specification illustrate embodiments consistent with the present application and serve to explain the principles of the present application, together with the description.

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in the drawings without creative labor.

[0012] Fig. 1 is a flowchart of a first embodiment of the multi-terminal interaction method of the present application;

[0013] Fig. 2 is a system framework diagram of multi-terminal interaction in the multi-terminal interaction method of the present application;

[0014] Fig. 3 is a flowchart of a second embodiment of the multi-terminal interaction method of the present application;

[0015] Fig. 4 is an interaction diagram of a local copy scenario in the multi-terminal interaction method of the present application;

[0016] Fig. 5 is a flowchart of a third embodiment of the multi-terminal interaction method of the present application;

[0017] Fig. 6 is an interaction diagram of a remote data thumbnail display scenario in the multi-terminal interaction method of the present application;

[0018] Fig. 7 is a flowchart of a fourth embodiment of the multi-terminal interaction method of the present application;

[0019] Fig. 8 is a flowchart of a fifth embodiment of the multi-terminal interaction method of the present application;

[0020] Fig. 9 is a device structure diagram of a hardware running environment involved in the multi-terminal interaction method in the embodiments of the present application.

[0021] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] Currently, file sharing is an essential function in multi-device interactive services. To improve user experience, most current file sharing solutions employ a combination of virtual file systems and network file systems. This approach allows any application on the receiving end to access remote files as easily as accessing local files. Taking a multi-screen collaborative projection scenario as an example, when projecting a screen from a mobile phone to a PC (Personal Computer), the PC may need to access files on the mobile phone. The PC can be considered the receiving end, and the mobile phone the remote end. In practical applications, besides accessing files on the mobile phone, the PC may also perform control operations on related files. However, in the aforementioned file sharing solutions, if the PC needs to operate on any file on the mobile phone, the original file must be transferred over the network. For example, suppose data needs to be copied from folder A on the mobile phone to folder B. If the aforementioned technical solutions are adopted, both folder A and folder B will be considered local folders for the PC. Therefore, the PC will retrieve the data to be copied from folder A and copy it to folder B. However, the actual operation involves the mobile phone sending the original data from folder A to the PC, the PC then sending the original data back to the mobile phone, and the mobile phone copying the received original data to folder B. Since the original data is transmitted over the network or other means, it takes longer than local data transfer. For the user, the transmission time of the original data is also considered as the interaction response time, resulting in slow multi-device interaction response or low interaction efficiency. This is especially noticeable with large files, severely impacting the user experience.

[0025] The main solution of the present application is: after triggering an interaction task of interacting with the remote end, determining an interaction scene of the interaction task; in the case that the type of the interaction scene belongs to data processing result interaction, generating an interaction instruction for controlling the remote end to process local data based on the interaction task; sending the interaction instruction to the remote end, and receiving a processing result obtained by the remote end processing local data based on the interaction instruction.

[0026] That is, in the embodiment of the present application, the control end and the remote end are in communication connection. After the control end triggers an interaction task with the remote end, the control end will further determine the interaction scene of the interaction task, and in the case that the type of the interaction scene belongs to data processing result interaction, generate an interaction instruction for controlling the remote end to process local data according to the interaction task, and receive a processing result obtained by the remote end processing local data based on the interaction instruction. That is, in the interaction process between the control end and the remote end, when the control end needs to process a certain data of the remote end, the control end will not obtain the original data stored in the remote end, but send the above-mentioned interaction instruction to the remote end, so that the remote end processes the data locally. For the remote end, only the processing result of the data processing is needed. Compared with the related technical solution, in the process of interaction, the original data needs to be transmitted. In the case that the interaction scene belongs to data processing result interaction, only the data processing result is transmitted, the data transmission amount is less, so the time spent for data transmission is reduced, the multi-end interaction efficiency is improved, and the user experience is guaranteed.

[0027] The execution subject of the embodiment can include, but is not limited to, mobile terminals such as mobile phones (i.e., cell phones), tablet computers, mobile robots, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and the like, and fixed terminals such as digital TVs, desktop computers, and the like, or any electronic device capable of implementing the above functions, and the embodiment does not specifically limit this. The following will take the terminal as an execution subject as an example to describe the embodiment and each of the following embodiments.

[0028] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0029] Please refer to FIG. 1, which is a flowchart provided by the first embodiment of the multi-end interaction method of the present application.

[0030] The multi-terminal interaction method of the first embodiment is proposed in the application and is applied to a control terminal. The control terminal is in communication connection with a remote terminal. The multi-terminal interaction method comprises steps S100-S300.

[0031] In step S100, after triggering an interaction task of interacting with the remote terminal, an interaction scenario of the interaction task is determined.

[0032] In actual application, the multi-terminal interaction method can be applied to a multi-terminal screen sharing scenario or a multi-terminal screen collaborative projection scenario. Generally, the content in the remote terminal is projected or shared to the control terminal. The control terminal and the remote terminal can be in wired or wireless communication connection. For example, in some conference scenarios, the remote terminal can be a mobile terminal (such as a smart phone), and the control terminal can be a large-screen device (such as a PC). The mobile terminal and the large-screen device can be in communication connection through a network. Generally, the screen content of the mobile terminal (or the content stored in the mobile terminal) needs to be projected (or projected) to the large-screen device to obtain better visual experience. In the multi-terminal screen sharing scenario or the multi-terminal screen collaborative projection scenario, file sharing is an essential function. Based on the file sharing function, the control terminal and the remote terminal can derive various interaction tasks, such as an interaction task of obtaining remote terminal data from the control terminal, an interaction task of viewing remote terminal files from the control terminal, and an interaction task of copying, deleting, or cutting remote terminal data from the control terminal. In different application scenarios, there can be different interaction tasks, which will not be described here.

[0033] Exemplarily, the above-mentioned interaction task is usually generated by the control terminal in response to a user operation. For example, after file sharing is performed between the control terminal and the remote terminal, the folder in the remote terminal can be regarded as a local folder of the control terminal for the application of the control terminal. Taking a file manager as an example, the folder in the remote terminal can be opened by the file manager in the control terminal. Correspondingly, when the user clicks the folder of the remote terminal in the control terminal, the interaction task of viewing the file in the remote terminal can be triggered. When the user selects a certain data in the folder of the remote terminal in the control terminal and drags the data to another folder, the interaction task of copying the data in the remote terminal can be triggered. The specific content of the above-mentioned interaction task or the triggering condition of the above-mentioned interaction task can be set based on the actual scene, which will not be described herein. After the interaction task is triggered, the control terminal can further determine the interaction scene of the interaction task. Exemplarily, it can include determining whether the type of the interaction scene belongs to data processing result interaction. For the control terminal, in the case of data processing result interaction, the control terminal does not need to obtain the original or all data from the control terminal. For example, for the scene of copying a certain data in the remote terminal to a certain folder in the remote terminal, the control terminal does not need to obtain the data to be copied, and only needs to determine that the remote terminal has completed the copying. For the scene of data search, the control terminal only needs to obtain the search result, and does not need to obtain all the data of the remote terminal and search from the obtained all data. For the interaction scene of the type belonging to data processing result interaction, it will not be exemplified herein. The interaction scene of the interaction task can be determined by the content of the interaction task. For example, the judgment condition of the interaction scene can be established in advance, and the content of the interaction task is compared with the judgment condition of the interaction scene to determine the interaction scene of the interaction task. The judgment condition can be set according to the characteristics of the data processing result interaction scene, which will not be described herein.

[0034] Referring to FIG. 2, it is a schematic diagram of the system framework of the multi-terminal interaction of the present application. As shown in the figure, it includes a control terminal and a remote terminal, and a VFS (Virtual File System) virtual file system and a scene recognition module are arranged in the control terminal. The network file system (client side) is deployed in the VFS virtual file system, and the network file system (service side) corresponding to the network file system (client side) is deployed in the remote terminal. The network file system (client side) and the network file system (service side) can realize file sharing between the control terminal and the remote terminal. The scene recognition module is used for determining the interaction scene of the interaction task.

[0035] In a feasible implementation, the step of determining the interaction scene of the interaction task includes step S101.

[0036] In a case where the interaction content in the interaction task matches the interaction feature of the preset scene, it is determined that the type of the interaction scene belongs to data processing result interaction, wherein the preset scene includes at least one of a remote local copy scene, a remote data thumbnail display scene, and a remote data search scene.

[0037] For example, when determining the interaction scene of the interaction task, the interaction content in the interaction task can be compared and matched with the interaction feature of the preset scene. In a case where it is determined that the interaction content matches the interaction feature, it is determined that the type of the interaction scene of the interaction task belongs to data processing result interaction. The preset scene can be set in multiple ways, such as a remote local copy scene, a remote data thumbnail display scene, and a remote data search scene. Different preset scenes can be set with different interaction features. For example, in the remote local copy scene, in a case where the data source address and the destination address of the copied data in the interaction content both belong to the remote end, it is determined that the interaction content in the interaction task matches the interaction feature of the remote local copy scene. In a case where the input keyword is included in the interaction content and the address of the interface at the time of inputting the keyword belongs to the remote end, it is determined that the interaction content in the interaction task matches the interaction feature of the remote data search scene. The specific interaction feature can be set based on the corresponding scene.

[0038] In a case where the type of the interaction scene belongs to data processing result interaction, the interaction instruction for controlling the remote end to process the local data is generated based on the interaction task.

[0039] For example, in a case where it is determined that the type of the interaction scene belongs to data processing result interaction, the control end can generate the interaction instruction for controlling the remote end to process the local data according to the interaction task. In the related technical solution described above, the application of the control end regards the folder of the remote end as a local folder. Therefore, in a case where the interaction scene is not determined and the interaction instruction is not generated, the control end will default to obtain the remote data and complete the corresponding operation. The interaction instruction generated above can make the remote end process the local data, thereby replacing the control end to perform the default operation. The default operation can be an operation that is performed by the control end by default when performing a task related to the folder of the remote end in a case where the control end regards the folder of the remote end as a local folder.

[0040] Different interaction instructions can be generated for different interaction tasks. For example, for the interaction task corresponding to the remote local copy scene, the interaction instruction generated based on the interaction task can include the address of the copied data, the target address of the copy, and the operation of copying the copied data to the target address. After receiving the interaction instruction, the remote end completes the operation of processing the local data corresponding to the interaction instruction, such as copying the copied data to the target address instead of the control end.

[0041] Step S300, sending the interaction instruction to the remote end, and receiving a processing result obtained by processing the local data based on the interaction instruction.

[0042] For example, the control end sends the interaction instruction to the remote end. After receiving the interaction instruction, the remote end processes the local data to obtain a processing result. The control end receives the processing result sent by the remote end. The processing result can be notification information representing the execution of the processing operation, or data generated after the execution of the processing operation. For example, in the data copying scenario, the processing result can be a copy completion or a copy failure. In the data searching scenario, the processing result can be data searched based on a keyword. If no data is searched, the processing result can also be notification information indicating that no search result is obtained.

[0043] In the embodiment of the application, the control end is in communication connection with the remote end. After triggering the interaction task with the remote end, the control end further determines the interaction scene of the interaction task. If the type of the interaction scene belongs to the data processing result interaction, the control end generates the interaction instruction for controlling the remote end to process the local data according to the interaction task, and receives the processing result obtained by processing the local data based on the interaction instruction. That is, in the interaction process between the control end and the remote end, when the control end needs to process a certain data of the remote end, the control end does not obtain the original data stored in the remote end, but sends the above interaction instruction to the remote end, so that the remote end processes the data locally. For the remote end, only the processing result of the data processing completion needs to be received. Compared with the related technical solution, in the interaction process, the original data needs to be transmitted. In the case where the interaction scene belongs to the data processing result interaction, only the data processing result needs to be transmitted, the data transmission amount is smaller, the time spent for data transmission is reduced, the multi-end interaction efficiency is improved, and the user experience is ensured.

[0044] Referring to FIG. 3, a flowchart of a second embodiment of the multi-end interaction method of the application is provided based on the first embodiment.

[0045] In the second embodiment of the application, the same or similar content as the above embodiments can be referred to the above introduction, and will not be described in detail hereinafter.

[0046] The method further includes step S111.

[0047] Step S111, the interaction content of the interaction task includes to-be-read data, to-be-written data, a read address of the to-be-read data, and a write address of the to-be-written data. In the case where the to-be-read data and the to-be-written data are the same, and the read address and the write address both belong to the remote end, it is determined that the interaction content in the interaction task matches the local copying scene interaction feature.

[0048] In the present embodiment, the remote local copy scenario will be identified. After the control terminal device and the remote terminal are connected and file sharing is performed, the files on the remote terminal will be mounted to a certain disk or directory of the control terminal, i.e., the control terminal can regard the folder of the remote terminal as a local folder. In the above remote local copy scenario, the user can open the files stored on the remote terminal through the file manager on the control terminal, and can copy a file from one directory on the remote terminal to another directory on the remote terminal. For example, if the remote terminal is a mobile phone and a USB flash disk or an OTG (USB On-The-Go device, a storage device connected to a mobile device such as a smart phone or a tablet computer through a USB) device is inserted into the mobile phone, and it is required to copy the files in the USB flash disk or the OTG device to the mobile phone, for the control terminal, since the USB flash disk or the OTG device is inserted into the mobile phone, the control terminal will attribute the files stored in the USB flash disk or the OTG device to the mobile phone.

[0049] For example, for the copy type of interaction task, the interaction task usually includes parameters such as to-be-read data, to-be-written data, a read address of the to-be-read data, and a write address of the to-be-written data. The control terminal or the scenario recognition module arranged in the control terminal will perform scenario recognition based on the interaction content, and the scenario recognition is performed based on the pre-set recognition rule. For example, it is determined whether the to-be-read data and the to-be-written data are the same, and whether the read address and the write address both belong to the remote terminal. If the to-be-read data and the to-be-written data are the same, and the read address and the write address both belong to the remote terminal, it can be determined that the interaction content in the interaction task matches the interaction feature of the remote local copy scenario.

[0050] Further, the interaction content of the above interaction task can be generated based on the operation of the user on the control terminal. As in the copy scenario, the user can open a folder stored on the remote terminal through a file manager on the control terminal, and select a file under the folder for the copy operation, at which time the selected file is the above to-be-read data, and the path of the file is the read address of the to-be-read data. Further, the user opens another folder and performs a paste operation in the folder, at which time the file in the pasteboard is the to-be-written data, and the path of the folder in which the paste operation is performed is the write address. Whether the to-be-read data and the to-be-written data are the same can be determined by comparing the names of the two data, and if the names are the same, it can be determined that the two data are the same. As for the read address and the write address, it can be determined whether the read address and the write address both belong to the remote terminal by judging whether the path names of the remote terminal are included in the paths corresponding to the two addresses, and if the path names of the remote terminal are included in the paths, it can be determined that the read address and the write address both belong to the remote terminal. Accordingly, when it is determined that the to-be-read data and the to-be-written data are the same, and the read address and the write address both belong to the remote terminal, it can be considered that the interaction content in the current interaction task matches the remote local copy scenario interaction feature. In actual application, to ensure the accuracy of the determination result of the remote local copy scenario, a corresponding prompt information can also be output, such as a prompt information whether to perform rapid copy, and after the user determines and selects to agree, a corresponding copy instruction can be generated.

[0051] In a feasible implementation, the type of the interaction instruction includes a copy instruction, and the step of generating the interaction instruction for controlling the remote terminal to process the local data based on the interaction task includes steps S211-S212.

[0052] In step S211, the to-be-read data in the interaction content is taken as local to-be-copied data, and the write address in the interaction content is taken as a target copy address.

[0053] In step S212, a copy instruction for controlling the remote terminal to copy the local to-be-copied data to the target copy address is generated.

[0054] For example, for the remote local copy scenario, the control terminal generates a corresponding copy instruction. For example, the control terminal can take the to-be-read data in the interaction task as the local to-be-copied data, and take the write address in the interaction content as the target copy address. Then, a copy instruction for controlling the local to-be-copied data to be copied to the target copy address is generated. In addition, the address of the local to-be-copied data can also be carried in the copy instruction.

[0055] After the control end sends the copy instruction to the remote end, the remote end copies the local data to be copied to the target copy address according to the copy instruction. After the copying is completed, the remote end sends the processing result of the copying to the control end. Compared with the related solution in which the original data needs to be transmitted, only the processing result of the copying is transmitted, so that the data transmission amount can be greatly reduced, the response speed of the overall interaction is accelerated, and the multi-end interaction efficiency is improved.

[0056] Referring to FIG. 4, an interaction diagram of a local copying scenario in the application is shown. The diagram includes a control end and a remote end. When the two ends share files, the remote end sends local file attributes to the control end, and the virtual file system of the control end can complete mapping of the remote end file based on the local file data sent by the remote end. The control end opens a source file A (data to be read) that needs to be copied in response to a user operation, and maps the source file A to the remote end. The control end records the name and path (read address) of the source file A when the source file A is opened. The control end writes data B (data to be written) to a target file (write address) in response to a user operation, and maps the data B to the remote end. The control end writes the data B to the target file, and records the name of the data B and the path of the target file. Whether the interaction scenario is identified, that is, whether the multi-end interaction method in the application is executed, if not, normal copying is directly performed, that is, the control end reads (receives or obtains) the source file A, and writes (or sends) the data B to the target file. If yes, the interaction scenario is identified, that is, the step of determining the interaction scenario of the interaction task is performed. When it is determined that the source file A and the data B are the same (that is, the name of the source file A and the name of the data B are the same), and the path of the source file A and the path of the target file both belong to the remote end, it is determined that the local copying scenario is used, and whether to open fast copying is prompted. If the user selects yes, fast copying is performed, that is, the remote end copies the source file A to the target file. If the user selects no, normal copying is performed.

[0057] Referring to FIG. 5, a flowchart of a third embodiment of the multi-end interaction method in the application based on the first embodiment and the second embodiment is shown.

[0058] In the third embodiment of the application, the same or similar contents as those in the above embodiments can be referred to the above description, and will not be described hereinafter.

[0059] The method further includes steps S121-S122.

[0060] In step S121, when the interaction content of the interaction task includes opening a folder, and the folder belongs to the remote end, a trigger execution operation in a preset time period after the folder is opened is monitored.

[0061] In a case where the trigger execution operation includes opening a file under the folder, it is determined that the interaction content matches the remote data thumbnail display scenario interaction feature.

[0062] In this embodiment, the remote data thumbnail display scenario will be identified. As in an application scenario, a user opens a certain folder through a file manager of a control terminal, and if the folder includes pictures or videos, after opening the folder, the file manager usually generates a thumbnail for the pictures or videos (for example, a thumbnail is used to view picture files or video files by default), and displays the thumbnail as a file preview effect on the current interface. When the file manager generates the thumbnail, it reads the corresponding original image data or original video data, so if the above-mentioned related technical solutions are used in a case where the opened folder is remote, the control terminal will obtain the original image data and the original video data under the folder in the remote for generating the thumbnail, and the remote will send the original image data and the original video data under the folder to the control terminal. However, it will take a lot of time to transmit the original image data and the original video data, especially when the folder includes a large number of pictures and videos, the time taken for transmission is more obvious. Therefore, in this embodiment, the remote data thumbnail display scenario will be identified.

[0063] For example, in a case where the interaction task includes opening a text folder, and the folder is remote, the trigger execution operation after opening the folder in a preset time period is monitored. It is mainly monitored whether the trigger execution operation includes opening a file under the folder. The ownership of the folder can also be determined by the path name of the folder. In a case where the file is viewed by default in a thumbnail, when a user opens a folder including images or videos through a file manager of a control terminal, the file manager will automatically open the images or videos under the folder, and generate corresponding thumbnails for display. That is, when the user opens the folder, the file manager will automatically open the images or videos under the folder. The above-mentioned preset time period is used to distinguish whether the file under the folder is opened by the user or automatically opened by the file manager. That is, in a normal case, the time required to trigger the file manager to automatically open the file is very short, and the time required to manually open the file by the user is relatively long. Correspondingly, the above-mentioned preset time period can be set to be relatively short to distinguish the two opening cases. Therefore, in a case where the trigger execution operation after opening the folder in a preset time period includes opening a file under the folder, it is determined that the interaction content matches the remote data thumbnail display scenario interaction feature, that is, the current interaction scenario is the remote data thumbnail display scenario.

[0064] In addition, to ensure the accuracy of the determination result of the remote data thumbnail display scene, the interaction content can be determined to match the remote data thumbnail display scene interaction feature in a case where the trigger execution operation includes opening multiple files under the folder. Alternatively, the interaction content can be determined to match the remote data thumbnail display scene interaction feature in a case where the trigger execution operation includes simultaneously opening multiple files under the folder. That is, in actual application, the scenario can be that the folder includes multiple images or videos. In the scenario where multiple images or videos exist, if the file manager automatically opens the images or videos under the folder, the file manager will use multiple threads to simultaneously open multiple images or videos. Obviously, the user cannot complete the operation of simultaneously opening multiple images or videos in a short time, and therefore, the remote data thumbnail display scene interaction feature is increased by “multiple”, “simultaneously”, and “multiple”, so as to further accurately distinguish whether the opening is artificial or automatic.

[0065] In another possible implementation, the method further includes steps S131-S133.

[0066] In step S131, when the interaction content of the interaction task includes opening a folder and the folder belongs to the remote end, a current interface feature is obtained, where the current interface feature includes at least one of an interface operation feature and an interface element feature.

[0067] In step S132, the current interface feature is classified and recognized based on a preset classification model to obtain a classification result.

[0068] Exemplarily, in the embodiment, the remote data thumbnail display scenario is determined by a classification model. Similarly, in the case that the interaction content of the interaction task includes opening a folder and the folder belongs to the remote, the current interface features are obtained, which can include at least one of interface operation features and interface element features. The interface operation features can capture the operation steps of the user on the interface, such as capturing the software used by the user to open the file on the control end. Generally, specific image software or applications are used when opening images or videos. The interface element features can include elements existing on the current interface, which can be in the form of an image, such as using a file manager to open an image, and the operation interface of the file manager as the interface element feature, or in the case of displaying image or video files, the text in the format of the file name representation (such as the file name suffix) on the interface can be used as the interface element feature, or the element representing the icon preview mode in the operation interface can be used as the interface element feature. The current interface features are then input into a preset classification model, and the preset classification model outputs a classification result. In the case that the classification result is a data preview mode, it is determined that the interaction content matches the interaction features of the remote data thumbnail display scenario. The preset classification model can be an image recognition library and method such as opencv and tensorflow through training of a convolutional neural network, or a traditional classification model such as Bayes. The preset classification model is pre-trained to have the ability to distinguish data preview modes. The training samples used for training can include operations performed on the interface and interface elements, and the types of sample labels can include data preview modes and non-data preview modes.

[0069] In a feasible implementation, the types of the interaction instructions further include thumbnail instructions, and the step of generating the interaction instructions for controlling the remote to process the local data based on the interaction task further includes steps S221-S222.

[0070] In step S221, the data under the folder in the interaction content is taken as local data to be thumbnailized.

[0071] In step S222, thumbnail instructions for controlling the remote to thumbnail the local data to be thumbnailized to obtain thumbnail content are generated.

[0072] Exemplarily, the type of the interaction instruction can further include a thumbnail instruction. For a scenario of thumbnail display of remote data, the control terminal can take the data under the folder in the interaction content as local data to be thumbnailized, and generate a thumbnail instruction for controlling the remote to perform thumbnail processing on the local data to be thumbnailized to obtain thumbnail content. Correspondingly, the remote performs thumbnail processing on the local data to be thumbnailized pointed to by the thumbnail instruction after receiving the thumbnail instruction, to obtain the thumbnail content. The thumbnail processing can be compression processing on an image or a video to obtain a thumbnail, and details of the compression processing are not described herein. The remote sends the thumbnail content to the control terminal.

[0073] Compared with the step of transmitting an original image in the related solution, the embodiment only transmits thumbnail content, so that the data transmission amount can be greatly reduced, the response speed of the overall interaction is accelerated, and the multi-terminal interaction efficiency is improved.

[0074] Referring to FIG. 6, an interaction schematic diagram of a thumbnail display scenario of remote data in the application is shown. As shown in the figure, the interaction schematic diagram includes a control terminal and a remote terminal. The control terminal opens a remote gallery in a preview mode, and a current viewing file is identified as being in the preview mode by using a preset classification model. A prompt information about whether to display in a thumbnail mode is output. If the user selects no, the remote terminal opens a source file. If the user selects yes, the remote terminal is notified to open a thumbnail of the file. Correspondingly, the remote terminal can open the thumbnail of the source file and send the thumbnail to the control terminal.

[0075] Referring to FIG. 7, a flowchart of a fourth embodiment of the multi-terminal interaction method in the application based on the first embodiment, the second embodiment and the third embodiment is shown.

[0076] In the fourth embodiment of the application, the same or similar content as in the above embodiments can be referred to the above description, and will not be described hereinafter.

[0077] The method further includes steps S141 to S143.

[0078] In step S141, when the interaction content of the interaction task includes opening a folder and inputting a keyword, and the folder belongs to the remote terminal, a current interface feature is obtained. The current interface feature includes at least one of an interface operation feature and an interface element feature.

[0079] In step S142, the current interface feature is classified and identified based on a preset classification model, to obtain a classification result. In step S143, when the classification result is a data search mode, it is determined that the interaction content matches a remote data search scenario interaction feature.

[0080] In the embodiment, the remote data search scenario will be identified. For example, in an application scenario, a user opens a certain folder through a file manager of a control terminal and inputs a keyword in a search bar for searching. The file manager compares data under the folder with the input keyword one by one to find data matching the keyword and obtains a search result. If the above related technical solution is used in the case where the opened folder belongs to a remote end, the control terminal will obtain all original data under the folder from the remote end. Correspondingly, the remote end needs to send all original data to the control terminal for searching by the control terminal. The transmission process will take a lot of time. Therefore, in the embodiment, the remote data search scenario will be identified.

[0081] For example, in the case where the interaction content of the interaction task includes opening a folder and inputting a keyword and the folder belongs to a remote end, a current interface feature is obtained, wherein the input keyword can be obtained by capturing an operation of a user on a current interface. Similarly, the current interface feature can include at least one of an interface operation feature and an interface element feature. The interface operation feature can capture an operation step of a user on an interface, such as capturing an input operation of a user on a search bar of an interface. The interface element feature can be a distributed element on a current interface. For example, when a file is opened using a file manager and a search operation is performed, a search bar expanded on an interface of the file manager or text content existing in the search bar can be used as an interface element feature. The above interface feature is input into a preset classification model. The preset classification model outputs a classification result. In the case where the classification result is a data search mode, it is determined that the interaction content matches the interaction feature of the remote data search scenario, that is, the corresponding interaction scenario is a remote data search scenario.

[0082] The preset classification model outputting the data search mode and the preset classification model outputting the data preview mode can be the same preset classification model or different preset classification models. For the case where they are the same preset classification model, a training sample with a label of a data search mode is added for training during training. At this time, the training sample can include a data preview mode, a data search mode and no mode. The specific process can be referred to the above embodiment, which will not be described here.

[0083] In a feasible implementation, the type of the interaction instruction further includes a search instruction. The step of generating the interaction instruction for controlling the remote end to process the local data based on the interaction task further includes steps S231-S232.

[0084] In step S231, data under the folder in the interaction content is taken as local data to be searched.

[0085] Step S232, generating a search instruction for controlling the remote end to search the local data to be searched based on the keyword to obtain a search result.

[0086] For example, the type of the interaction instruction further includes a search instruction. In the remote data search scenario, the control end takes the data under the opened folder in the interaction content as the local data to be searched, and generates a search instruction for controlling the remote end to search the local data to be searched based on an input keyword to obtain a search result. After receiving the search instruction, the remote end searches the local data to be searched pointed to by the search instruction based on the keyword carried in the search instruction to obtain a search result. The remote end sends the search result to the control end.

[0087] Compared with the step of transmitting all the data under the folder in the related solution, the embodiment of the present application only transmits the search result, so that the data transmission amount can be greatly reduced, thereby accelerating the response speed of the overall interaction and improving the multi-end interaction efficiency.

[0088] Referring to FIG. 8, a flowchart of a fifth embodiment of the multi-end interaction method of the present application based on the first embodiment, the second embodiment, the third embodiment, and the fourth embodiment is shown.

[0089] In the fifth embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above description, and will not be repeated hereinafter.

[0090] In the present embodiment, the multi-end interaction method is applied to a remote end, the remote end is in communication connection with a control end, and the multi-end interaction method includes steps A100-A300.

[0091] Step A100, receiving an interaction instruction sent by the control end, wherein the interaction instruction is generated by the control end based on the interaction task under the condition that the type of the interaction scene of the triggered interaction task is determined to belong to a data processing result interaction.

[0092] Step A200, processing the local data pointed to by the interaction instruction based on the interaction instruction to obtain a processing result. Step A300, feeding back the processing result to the control end.

[0093] In the present embodiment, the multi-end interaction method is applied to a remote end, and the remote end and the control end can be in communication connection through a wired or wireless manner.

[0094] Exemplarily, in the process of the remote end interacting with the control end, the remote end receives an interaction instruction sent by the control end, wherein the interaction instruction is generated by the control end according to the triggered interaction task in a case where the control end determines that the type of the interaction scene of the interaction task belongs to data processing result interaction. For the process of the control end determining the type of the interaction scene of the interaction task and generating the interaction instruction, reference can be made to the above embodiments, which will not be repeated here. The local data pointed to by the interaction instruction is processed according to the interaction instruction, and a processing result is obtained. The processing operation can be a copy operation, a thumbnail operation, a search operation, etc., and the obtained processing result can be a copy completion or copy failure notification, a thumbnail content obtained by performing a thumbnail (or compression) operation on the local data, a search result obtained by performing a search operation on the local data, etc. After obtaining the processing result, the remote end sends the data processing result to the control end.

[0095] Compared with the related technical solution, in the process of interaction, the original data needs to be transmitted. In the case where the interaction scene belongs to data processing result interaction, only the result of processing the local data by the remote end needs to be transmitted (without transmitting the complete or all local data), so that the data transmission amount is smaller, the time spent on data transmission is reduced, the multi-end interaction efficiency is improved, and the user experience is ensured.

[0096] In a feasible implementation, the step of processing the local data pointed to by the interaction instruction according to the interaction instruction to obtain a processing result includes step A210.

[0097] In a case where the interaction instruction is a copy instruction, the local data to be copied pointed to by the copy instruction is copied to a target copy address carried in the copy instruction, and a copy completion processing result is generated after the copying.

[0098] Exemplarily, in a case where the interaction instruction is a copy instruction, the copy instruction can carry a data name or a data address of the local data to be copied, and the data name and the data address can both be used to point to the local data to be copied. The copy instruction also carries a target copy address, and after the remote end successfully copies the local data to be copied to the target copy address, a copy completion processing result is generated.

[0099] In addition, if the remote end is sending the local data to be copied to the control end when receiving the copy instruction, the operation of sending the local data to be copied to the control end can be interrupted.

[0100] In a feasible implementation, the step of processing the local data pointed to by the interaction instruction according to the interaction instruction to obtain a processing result includes step A220.

[0101] Step A220, in the case that the interaction instruction is a thumbnail instruction, performing thumbnail processing on the local data to be thumbnai led pointed by the thumbnail instruction to obtain a thumbnail content of the local data to be thumbnai led, and sending the thumbnail content as the processing result.

[0102] For example, in the case that the interaction instruction is a thumbnail instruction, the thumbnail instruction can carry a data name or a data address of the local data to be thumbnai led, and the data name and the data address can be used to point to the local data to be thumbnai led. It is worth noting that the local data to be thumbnai led is actually the data under the folder opened by the user on the control end when the thumbnail instruction is generated on the control end. The remote end performs thumbnail processing, such as image compression processing, on the local data to be thumbnai led to obtain a thumbnail content of the local data to be thumbnai led. The thumbnail content is sent as the processing result to the control end.

[0103] In addition, if the remote end is sending the local data to be thumbnai led to the control end when the thumbnail instruction is received, the operation of sending the local data to be thumbnai led to the control end can be interrupted.

[0104] In a possible implementation, the step of processing the local data pointed by the interaction instruction based on the interaction instruction to obtain a processing result further includes step A230.

[0105] Step A230, in the case that the interaction instruction is a search instruction, performing search on the local data to be searched pointed by the search instruction based on a keyword carried in the search instruction to obtain a search result, and sending the search result as the processing result.

[0106] For example, in the case that the interaction instruction is a search instruction, the search instruction can carry a keyword, and a data name or a data address of the local data to be searched, and the data name and the data address can be used to point to the local data to be searched. It is worth noting that the local data to be searched is actually the data under the folder opened by the user on the control end when the search instruction is generated on the control end. The remote end performs search on the local data to be searched based on the keyword to obtain a search result. The search result is sent as the processing result to the control end.

[0107] In addition, if the remote end is sending the local data to be searched to the control end when the search instruction is received, the operation of sending the local data to be searched to the control end can be interrupted.

[0108] In addition, please refer to FIG. 9, which is a device structure diagram of a hardware running environment involved in the multi-end interaction method in the embodiments of the present application.

[0109] The application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the multi-end interaction method in the above embodiments.

[0110] Reference is made below to FIG. 9, which shows a structural diagram of an electronic device suitable for implementing embodiments of the application. The electronic device in embodiments of the application can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Portable Application Description), a PMP (Portable Media Player), a vehicle terminal (e.g., a vehicle navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in FIG. 9 is merely an example and should not impose any limitation on the functions and use range of embodiments of the application.

[0111] As shown in FIG. 9, the electronic device can include a processing device 1001 (e.g., a central processor, a graphic processor, or the like) that can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for operation of the electronic device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An I / O (Input / Output) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, and the like; an output device 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, and the like; the storage device 1003 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device having various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or provided. More or less systems can be alternatively implemented or provided.

[0112] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0113] The electronic device provided by the present application adopts the multi-terminal interaction method in the above-mentioned embodiments, and can solve the technical problems of slow multi-terminal interaction response and low interaction efficiency in related multi-terminal interaction schemes. Compared with the prior art, the electronic device provided by the present application has the same beneficial effects as the multi-terminal interaction method provided by the above-mentioned embodiments, and other technical features in the electronic device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0114] Parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0116] In addition, the present application also provides a computer readable storage medium having computer readable program instructions (i.e. computer program) stored thereon, the computer readable program instructions being used to execute the multi-terminal interaction method in the above-mentioned embodiments.

[0117] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination thereof.

[0118] The above computer readable storage medium may be contained in an electronic device, or may exist separately without being assembled into an electronic device.

[0119] The above computer readable storage medium carries one or more programs, when the one or more programs are executed by an electronic device, the electronic device is caused to: acquire multi-modal data of a to-be-positioned device in an indoor environment, the multi-modal data at least including positioning data of the to-be-positioned device and environment data within a preset range around the to-be-positioned device; perform fusion processing on the multi-modal data to obtain fusion positioning data; and identify a target multi-terminal interaction result corresponding to the fusion positioning data based on a preset positioning model.

[0120] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0121] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0122] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0123] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the multi-terminal interaction method described above, and can solve the technical problems of slow multi-terminal interaction response and low interaction efficiency in related multi-terminal interaction schemes. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the multi-terminal interaction method provided by the above embodiments, and will not be described here.

[0124] The above merely describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or the like, which is made based on the technical concept of the present application and the content of the specification and drawings, is included in the patent protection scope of the present application.

Claims

1. A multi-terminal interaction method applied to a control terminal, wherein the control terminal is communicatively connected to a remote terminal, the multi-terminal interaction method comprising: After triggering an interaction task that interacts with the remote end, the interaction scenario of the interaction task is determined. When the type of the interaction scenario is data processing result interaction, an interaction instruction to control the remote processing of local data is generated based on the interaction task. The interaction command is sent to the remote end, and the processing result obtained by the remote end from processing local data based on the interaction command is received.

2. The multi-terminal interaction method as described in claim 1, wherein, The step of determining the interaction scenario of the interaction task includes: If the interactive content in the interactive task matches the interactive features of the preset scenario, the type of the interactive scenario is determined to be data processing result interaction. The preset scenario includes at least one of the following: remote local copy scenario, remote data thumbnail display scenario, and remote data search scenario.

3. The multi-terminal interaction method as described in claim 2, wherein, The method further includes: The interactive content of the interactive task includes data to be read, data to be written, the read address of the data to be read, and the write address of the data to be written. When the data to be read and the data to be written are the same, and the read address and the write address both belong to the remote end, it is determined that the interactive content in the interactive task matches the interactive features of the remote local copy scenario.

4. The multi-terminal interaction method as described in claim 3, wherein, The types of interactive commands include copy commands, and the step of generating interactive commands to control the remote processing of local data based on the interactive task includes: The data to be read in the interactive content is taken as the local data to be copied, and the write address in the interactive content is taken as the target copy address; Generate a copy instruction that controls the remote end to copy the local data to be copied to the target copy address.

5. The multi-terminal interaction method as described in claim 2, wherein, The method further includes: If the interactive content of the interactive task includes opening a folder, and the folder belongs to the remote end, monitor and execute the triggered operation within a preset time period after opening the folder; If the triggered operation includes opening a file in the folder, it is determined that the interactive content matches the interactive features of the remote data thumbnail display scene.

6. The multi-terminal interaction method as described in claim 2, wherein, The method further includes: When the interactive content of the interactive task includes opening a folder, and the folder belongs to the remote end, the current interface features are obtained, wherein the current interface features include at least one of interface operation features and interface element features. Based on a preset classification model, the current interface features are classified and identified to obtain classification results; If the classification result is in data preview mode, determine that the interactive content matches the interactive features of the remote data thumbnail display scene.

7. The multi-terminal interaction method as described in claim 5 or 6, wherein the types of interaction instructions further include abbreviated instructions, and the step of generating interaction instructions for controlling the remote processing of local data based on the interaction task further includes: The data in the folder mentioned in the interactive content will be used as the local data to be thumbnailed. Generate a thumbnail instruction that controls the remote end to thumbnail the local data to be thumbnailed to obtain the thumbnailed content.

8. The multi-terminal interaction method as described in claim 2, wherein, The method further includes: When the interactive content of the interactive task includes opening a folder and entering keywords, and the folder belongs to the remote end, the current interface features are obtained, wherein the current interface features include at least one of interface operation features and interface element features. Based on a preset classification model, the current interface features are classified and identified to obtain classification results; If the classification result is a data search mode, determine that the interactive content matches the interactive features of the remote data search scenario.

9. The multi-terminal interaction method as described in claim 8, wherein, The types of interactive commands also include search commands, and the step of generating interactive commands to control the remote processing of local data based on the interactive task further includes: The data in the folder mentioned in the interactive content will be used as local data to be searched. Generate a search instruction that controls the remote end to search the local data to be searched based on the keywords and obtain search results.

10. A multi-terminal interaction method applied to a remote end, wherein the remote end is communicatively connected to a control end, the multi-terminal interaction method comprising: Receive interactive commands sent by the control terminal, wherein, The interaction command is generated by the control terminal based on the interaction task when it is determined that the type of the interaction scenario triggered by the interaction task belongs to data processing result interaction; The local data pointed to by the interaction command is processed based on the interaction command to obtain the processing result; The processing result is fed back to the control terminal.

11. The multi-terminal interaction method as described in claim 7, wherein, The step of processing the local data pointed to by the interaction command based on the interaction command to obtain the processing result includes: When the interaction command is a copy command, the local data to be copied pointed to by the copy command is copied to the target copy address carried in the copy command, and a copy completion result is generated after the copy is completed.

12. The multi-terminal interaction method as described in claim 7, wherein, The step of processing the local data pointed to by the interaction command based on the interaction command to obtain the processing result further includes: When the interaction command is a shortening command, the local data to be shortened pointed to by the shortening command is shortened to obtain the shortened content of the local data to be shortened, and the shortened content is used as the processing result.

13. The multi-terminal interaction method as described in claim 7, wherein, The step of processing the local data pointed to by the interaction command based on the interaction command to obtain the processing result further includes: When the interaction command is a search command, the local data to be searched is searched based on the keywords carried in the search command to obtain search results, and the search results are used as the processing result.

14. An electronic device, comprising: A memory, a processor, and a multi-terminal interaction program stored in the memory and executable on the processor, wherein the multi-terminal interaction program, when executed by the processor, implements the multi-terminal interaction method as described in any one of claims 1 to 9, or 10 to 13.

15. A computer-readable storage medium, wherein, The computer-readable storage medium stores a multi-terminal interaction program, which, when executed by a processor, implements the multi-terminal interaction method as described in any one of claims 1 to 9, or 10 to 13.

Images