A method and device for multi-device collaborative work

Abstract

The present application discloses a method and apparatus for multi-device collaborative work, wherein the method includes: the collaborative control device of the collaborative group monitors the application operation and resource usage status of each member device in the group in real time; all devices in the group are in the same local area network; when the first member device needs to start the application operation, the collaborative control device determines whether the operation requires device collaboration, and if so, determines the hardware that needs collaboration based on the resource usage status of the first member device; the collaborative control device matches the hardware that needs collaboration based on the resource usage status of the member devices in the group with the second member device for providing corresponding collaboration; using the virtual bus simulated by the communication link of the local area network, the collaborative request instruction and the operation instruction and data set required for executing the corresponding collaboration are sent to the second member device to trigger the second member device to perform the corresponding collaborative operation. By adopting the present application, many-to-many device collaboration can be realized, and the efficiency of collaborative work can be effectively improved.

Description

Technical Field

[0001] The present invention relates to computer application technology, and in particular to a multi-device collaborative working method and device. Background Art

[0002] Existing device collaboration solutions usually achieve one-to-one collaboration between devices, but cannot achieve many-to-many device collaboration, that is, one device can collaborate with multiple devices at the same time, and one device can also be collaborated by multiple devices at the same time. In addition, the collaboration between devices is mostly surface-level business collaboration such as screen sharing and file delivery, and cannot achieve collaboration at the system hardware level such as memory and CPU. This makes the collaboration between devices insufficient, and thus cannot fully utilize the collaboration mechanism between devices to improve the operating efficiency of applications. Summary of the invention

[0003] In view of this, the main purpose of the present invention is to provide a multi-device collaborative working method and apparatus, which can realize many-to-many device collaboration and effectively improve the collaborative work efficiency.

[0004] In order to achieve the above object, the technical solution proposed in the embodiment of the present invention is:

[0005] A multi-device collaborative working method, comprising:

[0006] The collaborative control device of the collaborative group monitors the application operation and resource usage status of each member device in the group in real time; all member devices in the collaborative group are in the same local area network;

[0007] When the first member device needs to start the application running, the collaboration control device determines whether the running requires device collaboration, and if so, determines the hardware that needs collaboration for the running according to the resource usage status of the first member device; the first member device is any member device in the collaboration group;

[0008] The collaboration control device matches the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; utilizes the virtual bus of the collaboration group to send a collaboration request instruction and operation instructions and data sets required for executing the corresponding collaboration to each of the second member devices, so as to trigger the second member device to execute the corresponding collaborative operation based on the operation instructions and data sets; the virtual bus is simulated by the communication link of the local area network.

[0009] The embodiment of the present invention further provides a multi-device collaborative working device, which is arranged in a collaborative control device of a collaborative group, and includes:

[0010] A monitoring unit, used for monitoring the application running and resource usage status of each member device in the collaboration group in real time; all member devices in the collaboration group are in the same local area network;

[0011] A collaborative decision unit, configured to, when a first member device needs to start an application, determine whether the operation requires device collaboration by the collaborative control device, and if so, determine the hardware that requires collaboration for the operation according to the resource usage status of the first member device; the first member device is any member device in the collaborative group;

[0012] A collaboration control unit is used to match the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; using the virtual bus of the collaboration group, sending a collaboration request instruction and an operation instruction and a data set required for executing the corresponding collaboration to each of the second member devices, so as to trigger the second member device to execute the corresponding collaborative operation based on the operation instruction and the data set; the virtual bus is simulated by the communication link of the local area network.

[0013] The embodiment of the present invention also provides a multi-device collaborative working device, including a processor and a memory;

[0014] The memory stores an application program executable by the processor, which is used to enable the processor to execute the multi-device collaborative working method as described above.

[0015] An embodiment of the present invention further provides a computer-readable storage medium, in which computer-readable instructions are stored. The computer-readable instructions are used to execute the multi-device collaborative working method as described above.

[0016] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned multi-device collaborative working method when executed by a processor.

[0017] In summary, the multi-device collaborative work scheme proposed in the embodiment of the present invention uses a collaborative control device to monitor the application operation and resource usage status of each member device in the collaborative group in real time. When it is monitored that any member device needs to start the application operation, it is determined whether the application operation requires device collaboration, and the hardware that needs collaboration is matched with a device for providing corresponding collaboration, and the virtual bus simulated by the communication link of the local area network is used to send a collaboration request instruction and an operation instruction and data set required for executing the corresponding collaboration to each second member device, so as to trigger the second member device to perform the corresponding collaborative operation based on the operation instruction and data set. In this way, by using the local area network to simulate the virtual bus, connect each collaborative device, transmit hardware control signals such as CPU and GPU on the virtual bus, perform clock synchronization and timing signal transmission, so that the collaboration between devices can go deep into the system level, that is, the memory, CPU and other levels of cooperation, and is no longer limited to screen sharing, file delivery and other aspects, so as to improve the cross-system multi-device collaboration capability and effectively improve the efficiency of collaborative work. In addition, in the above embodiment, the local area network is used to connect the collaborative devices in the collaborative group, and the devices in the collaborative group can collaborate at will, so many-to-many collaboration can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of an application of an embodiment of the present invention in scenario one;

[0020] Figure 3 This is a schematic diagram of the application of the embodiment of the present invention in scenario 2;

[0021] Figure 4 This is a schematic diagram of the application of the embodiment of the present invention in scenario three;

[0022] Figure 5 This is a schematic diagram of the application of the embodiment of the present invention in scenario 5;

[0023] Figure 6 Schematic diagram of the device structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Figure 1 FIG. 1 is a flow chart of a multi-device collaborative working method according to an embodiment of the present invention. Figure 1 As shown, this embodiment mainly includes:

[0026] Step 101: A collaborative control device of a collaborative group monitors in real time the application running and resource usage status of each member device in the group; all member devices in the collaborative group are in the same local area network.

[0027] In this step, it is necessary to use the collaborative control device to monitor the running status of the application and the usage status of each hardware resource on each member device in the collaborative group in real time, so as to collaboratively control the running of the application on each device in the group in a timely manner.

[0028] Here, the collaborative control device is responsible for making decisions and controlling the collaborative work within the collaborative group. Specifically, a member device in the collaborative group can be selected in advance as the collaborative control device. For example, a device with strong hardware processing capabilities can be selected as the collaborative control device, but it is not limited to this. In practical applications, a dedicated device can also be used as the collaborative control device.

[0029] All member devices in the collaboration group are in the same local area network, so that the local area network simulation bus can be used to transmit operation instructions and data between devices, realize clock synchronization and timing signal transmission between collaborative devices, and enable collaboration between devices to extend to memory, CPU and other levels, rather than being limited to screen sharing, file delivery and other aspects.

[0030] In actual applications, member devices in the collaboration group may be mobile phones, computers, televisions, tablet computers, etc., but are not limited thereto.

[0031] Step 102: When the first member device needs to start application operation, the collaborative control device determines whether the operation requires device collaboration. If so, the hardware that requires collaboration is determined based on the resource usage status of the first member device; the first member device is any member device in the collaborative group.

[0032] In this step, when the collaborative control device monitors that an application needs to be started on a member device (i.e., the first member device) in the group, it is necessary to determine whether the application needs to be coordinated by other devices in the group. The specific judgment can be made according to the preset collaboration strategy. For example, if the collaboration switch is turned on on the first member device, it is determined that device collaboration is required.

[0033] When the collaborative control device determines that the application operation requires device collaboration, it needs to determine one by one whether each hardware required for the application operation needs to be collaborated by the corresponding hardware of other devices. Specifically, the hardware includes external devices (such as speakers, headphones, screens, etc.) and core hardware devices (such as CPU, GPU, memory, etc.).

[0034] In one implementation, the following method may be specifically used to determine the hardware that needs to cooperate with the operation according to the resource usage status of the first member device:

[0035] For each type of hardware required for the operation, it is determined whether the first member device can currently provide corresponding hardware resources to complete the operation according to the current resource usage status of the first member device. If not, it is determined that the hardware is hardware that requires collaboration.

[0036] In the above method, for each type of hardware required for the application to run, it is necessary to determine whether the idle resources of this type of hardware in the first member device can meet the resource requirements of the application for this type of hardware. If not, it is determined that other devices need to provide corresponding hardware cooperation, that is, the hardware that requires cooperation.

[0037] Step 103: The collaboration control device matches the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; utilizes the virtual bus of the collaboration group to send a collaboration request instruction and operation instructions and data sets required for executing the corresponding collaboration to each of the second member devices, so as to trigger the second member device to execute the corresponding collaborative operation based on the operation instructions and data sets; the virtual bus is simulated by the communication link of the local area network.

[0038] In this step, for each type of hardware that has been determined to require collaboration, it is necessary to match it with a member device (i.e., the second member device) from the collaboration group to provide the corresponding collaboration. Afterwards, the virtual bus simulated by the local area network is used to send collaboration request instructions and operation instructions and data sets required to perform the corresponding collaboration to each second member device, so as to trigger the second member device to perform the corresponding collaborative operation.

[0039] Here, the virtual bus simulated by the local area network has the same bus structure as the existing computer system, specifically including a virtual control bus and a virtual data bus. Among them, the virtual control bus is used to transmit control signals and timing signals, and is bidirectional. The control bus specifically includes a graphics card dedicated bus (AGP bus) for graphics card data transmission and video memory information transmission, a CPU bus for sending control signals and synchronizing memory information, and other buses for transmitting control signals and timing signals other than the CPU and graphics card. The virtual data bus is used to transmit data, including cache information, App software and other operating information.

[0040] By using the local area network to simulate a virtual bus, transmitting hardware control signals such as the CPU and GPU on the virtual bus, and performing clock synchronization and timing signal transmission, the collaboration between devices can go deeper into the system level, that is, cooperation at the memory, CPU and other levels, and is no longer limited to screen sharing, file delivery and other aspects, thereby improving the cross-system multi-device collaboration capability and effectively improving the efficiency of collaborative work.

[0041] In practical applications, the data transmission method on the virtual bus is the same as the prior art. Before data transmission, it is necessary to convert the corresponding bus transmission format so as to transmit on the bus.

[0042] In one implementation, the following method may be specifically used to match the second member device for providing corresponding cooperation to the hardware requiring cooperation in the cooperation group:

[0043] For each type of hardware that needs to collaborate, based on the resource usage status of each member device in the collaboration group, search for a member device that can provide corresponding hardware resources from other member devices in the collaboration group except the first member device, and select one member device from the found member devices as the second member device corresponding to the type of hardware.

[0044] In actual applications, among the member devices that can provide corresponding hardware resources, the device with the largest free resources can be selected as the second member device according to the occupancy of the corresponding hardware. Alternatively, the user can select a device from the found member devices as the second member device.

[0045] In one implementation, the following method may be further used to add a new member device to the collaboration group:

[0046] When a new device is connected to the local area network and the user agrees that the new device joins the collaboration group, the collaboration control device adds the new device to the collaboration group, and obtains and saves the hardware configuration and usage status information of the new device.

[0047] The above method can be used to add new devices joining the local area network to the collaboration group. The collaborative control device records all its available hardware (including core hardware and external devices) and its occupancy status. After that, the collaborative control device will also monitor the application operation and resource usage status on it in real time.

[0048] Furthermore, considering that in actual applications, the collaborative operation of the second member device may be interrupted due to some reasons (such as the second member device actively interrupting the collaboration or a higher priority request occupying the device, etc.), at this time, the following method can be used to consider continuing the interrupted application operation through collaborative transfer. The specific method is as follows:

[0049] In one embodiment, the method further comprises:

[0050] Step a1: When the collaborative operation of the second member device is interrupted, suspend the task of the application running and back up the corresponding breakpoint data.

[0051] In this step, when the collaborative operation is interrupted, the breakpoint data needs to be backed up to ensure that the application can continue to be executed in a device in the collaborative group when the interruption is continued, and to ensure that the application running data will not be lost due to the interruption.

[0052] Step a2: If the interruption is caused by the preemption of the second member device by a new task, the collaborative control device determines whether the execution time of the new task is greater than the preset short task duration threshold, and if so, executes a collaborative transfer process; the collaborative transfer process includes: determining whether the first member device needs the collaboration of corresponding hardware to continue running the application, and if so, matching the hardware corresponding to the collaborative operation within the collaborative group with a third member device for continuing to provide the corresponding collaboration based on the resource usage status of each member device in the collaborative group, and using the virtual bus of the collaborative group to send a collaborative request instruction, the breakpoint data, and the operation instructions and data sets required to continue to execute the corresponding collaboration to the third member device, so as to trigger the third member device to continue to provide the corresponding collaborative operation for the first member device, otherwise, sending the breakpoint data to the first member device to trigger the first member device to continue to execute the application at the interrupted breakpoint.

[0053] Here, when the interruption is caused by the preemption of the second member device by a new task, the execution time of the new task needs to be considered. If it is a short task (such as listening to a voice), then wait for the interruption to end and then continue to perform the collaborative operation in the second member device. If the duration is long, that is, greater than the short task duration threshold, then execute the collaborative transfer process to transfer to other devices to continue the collaboration, or continue to be executed by the first member device, so that the impact of the interruption on the collaborative operation of applications between devices can be reduced through collaborative transfer.

[0054] If the interruption is caused by the second member device actively giving up providing cooperation, the cooperation control device executes the cooperation transfer process.

[0055] Here, when the interruption is caused by the second member device actively giving up providing collaboration, it means that the second member device can no longer continue to provide corresponding collaboration. At this time, it is necessary to execute the collaboration transfer process to transfer to other devices to continue the collaboration, or continue to be executed by the first member device, so that the impact of the collaboration interruption on the application operation can be reduced through collaboration transfer.

[0056] In one implementation, considering that in actual applications, the first member device may no longer need the collaboration device to provide collaboration due to the release of hardware resources or other reasons (such as receiving a user's instruction to close collaboration, etc.), in this case, the following method can be used to transfer the collaboration operation to the first member device to continue to execute the corresponding application:

[0057] When the collaborative control device receives the termination collaboration instruction from the first member device, the collaborative control device triggers the task of suspending the application running and backs up the corresponding breakpoint data; the breakpoint data is sent to the first member device so that the first member device continues to execute the application running at the interrupted breakpoint.

[0058] It can be seen from the above technical solutions that the multi-device collaborative work solution proposed in the above embodiments uses a local area network to simulate a virtual bus, connect various collaborative devices, transmit hardware control signals such as CPU and GPU on the virtual bus, perform clock synchronization and timing signal transmission, so that the collaboration between devices can go deep into the system level, effectively improving the efficiency of collaborative work. In addition, the local area network is used to connect various collaborative devices in the collaborative group, so that the devices in the collaborative group can collaborate at will, thereby realizing many-to-many collaboration between devices.

[0059] The specific implementation of the above embodiment is described in detail below in conjunction with several multi-device scenarios in a home local area network.

[0060] Scenario 1: The user uses a computer, mobile phone, and tablet at the same time and all need Bluetooth headsets.

[0061] Specific scenarios:

[0062] 1. The user is watching a movie in full-screen mode on the computer (the Bluetooth headset is connected to the computer).

[0063] 2. Use WeChat on a tablet to chat with others via voice (one sentence at a time).

[0064] 3. The mobile phone has QQ voice access.

[0065] 4. Connect your mobile phone, tablet and computer to the local area network and join the collaboration.

[0066] like Figure 2 As shown, the above method embodiment is used to perform the following multi-device collaboration:

[0067] 1. Computers, mobile phones, and tablets are all connected to collaborative devices, and Bluetooth headsets are registered as audio playback devices of computers to form a collaborative group. Bluetooth headsets are paired with computers, and the computer multimedia channel sound (video sound) is output from the Bluetooth headset.

[0068] 2. When the tablet's WeChat voice is playing, the user chooses to use collaboration, and the matching module automatically matches the computer multimedia sound output device that the user is currently using. Establish short-term collaboration (if the user does not use the tablet to output audio for a period of time, the collaboration will be automatically disconnected). The audio is packaged as data and sent to the computer, and the computer's multi-channel and computer video sound are played to the Bluetooth headset at the same time.

[0069] 3. The mobile phone QQ call accesses and the user chooses to use collaboration. A short-term collaboration is established (if the user ends the QQ call, the collaboration is automatically disconnected). The mobile phone uses the audio as data and outputs it through the Bluetooth headset.

[0070] Scenario 2: A user plays a 3D game on a mobile phone. At the same time, a video app on the mobile phone is projected onto the TV to play a high-definition video of the game tutorial.

[0071] Specific scenarios:

[0072] 1. User A is working on his computer in his bedroom.

[0073] 2. User B plays a 3D game on his mobile phone in the living room with headphones (the headphones are plugged into the phone).

[0074] 3. User B hopes to cast the game guide in the mobile video app on the TV in the living room.

[0075] 4. Mobile phones, TVs, and computers are all connected to the local area network and join the collaboration.

[0076] like Figure 3 As shown, the above method embodiment is used to perform the following multi-device collaboration:

[0077] 1. The mobile phone, TV, and computer are all connected to the collaborative device, and the headphones are plugged into the mobile phone.

[0078] 2. The user opens the video software, selects the video to be played, places it in the background of the mobile phone, and specifies the TV screen to be cast.

[0079] 3. The mobile phone separates the video and audio, and the video data is sent to the TV through the virtual bus, and the audio is output by the headphones through the mobile phone channel.

[0080] 4. The TV uses its own processor to process and display it on the TV screen.

[0081] 5. The synchronization of the video sound on the mobile phone and the video on the TV is controlled by the clock signal.

[0082] 6. The user plays a 3D game on his mobile phone and pauses the video with the TV remote control.

[0083] 7. The TV receives the pause signal and synchronizes it to the mobile phone through the virtual bus, and the mobile phone pauses the audio playback

[0084] 8. When the user turns on the game with the highest picture quality, the collaborative control device matches the idle GPU of the computer. The mobile phone sends the geometric processing related data of the picture to the computer through the virtual bus. After the computer GPU processes it, the data is sent back to the mobile phone for raster processing and displayed on the screen, which improves the processing speed.

[0085] Scenario 3: The collaboration device is occupied, and the collaboration is interrupted and transferred.

[0086] Specific scenarios:

[0087] 1. User A plays a 3D game on his mobile phone in the living room with headphones (the headphones are plugged into the phone).

[0088] 2. User A projects the game guide on the mobile phone video app onto the TV in the living room.

[0089] 3. User B brings up the TV menu.

[0090] 4. Ask user A whether to transfer the screen projection to the tablet (there is no corresponding video software on the tablet)

[0091] like Figure 4 As shown, the above method embodiment is used to perform the following multi-device collaboration:

[0092] 1. User B brings up the TV menu.

[0093] 2. TV collaboration is interrupted and the interruption data is packaged.

[0094] 3. The collaborative control device considers that the mobile phone needs to collaborate and scans for collaborative group devices.

[0095] 4. The new device is matched to a tablet and the user is asked whether to continue playing on the tablet.

[0096] 5. User A selects to continue playing on the tablet and turns on the tablet (the user can also specify a new collaboration device).

[0097] 6. The interruption data is sent to the tablet computer after format conversion and standardization to establish a new collaborative relationship.

[0098] 7. Interrupted video continues playing on tablet.

[0099] Scenario 4: A user is watching a Blu-ray high-definition video on a mobile phone, while the phone is updating a large number of software and video chatting with friends at the same time.

[0100] Specific scenarios:

[0101] 1. The user opens the video software to watch Blu-ray HD video.

[0102] 2. The user opens the App Store to update all updatable software (the user has not updated for a long time).

[0103] 3. The user starts WeChat video chat with the highest image quality for members (the updated software does not include the video software and WeChat).

[0104] 4. The video and chat experience will be temporarily stuck and then return to normal.

[0105] The above method embodiment is used to perform the following multi-device collaboration:

[0106] 1. Users update the software to watch high-definition videos and video chat with friends at the same time.

[0107] 2. The CPU usage is detected to be extremely high, affecting the user's video decoding speed and video chat transcoding speed.

[0108] 3. Automatically match to unused tablets in the collaboration group.

[0109] 4. Establish collaboration from mobile phone to tablet computer and decode the video source code in the tablet computer.

[0110] 5. The decoded data is transmitted back to the mobile phone through the virtual bus and played in real time on the mobile phone.

[0111] 6. After the user software is updated, it is detected that the CPU usage of the mobile phone has decreased, and the collaboration is interrupted.

[0112] 7. Send the decoded data and breakpoints back to the mobile phone, which will be processed by the mobile phone CPU.

[0113] Scenario 5: Metaverse Collaboration Capabilities,In the future, such as the Metaverse, the boundaries of devices are destined to be broken. Therefore, in-depth and thorough collaboration is inevitable. Applying the above embodiments can maximize the use of the functions and computing power of devices.

[0114] Specific scenarios:

[0115] The Metaverse Has Powerful Functions

[0116] TVs and mobile phones use features from the Metaverse

[0117] like Figure 5 As shown, the above method embodiment is used to perform the following multi-device collaboration:

[0118] 1. All devices join the collaboration.

[0119] 2. The capabilities of the Metaverse (computing power, AR, etc.) are shared with TVs and mobile phones.

[0120] 3.TV can use mobile phone functions, such as face recognition.

[0121] Based on the above method embodiment, the embodiment of the present invention further proposes a multi-device collaborative working device, which is arranged in a collaborative control device of a collaborative group, such as Figure 6 As shown, including:

[0122] The monitoring unit 601 is used to monitor the application running and resource usage status of each member device in the collaboration group in real time; wherein all member devices in the group are in the same local area network.

[0123] The collaborative decision unit 602 is used to determine whether the operation requires device collaboration when the first member device needs to start the application. If so, the collaborative control device determines the hardware that requires collaboration for the operation based on the resource usage status of the first member device; the first member device is any member device in the collaborative group.

[0124] The collaboration control unit 603 is used to match the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; utilize the virtual bus of the collaboration group to send a collaboration request instruction and operation instructions and data sets required for executing corresponding collaboration to each of the second member devices, so as to trigger the second member device to perform corresponding collaborative operations based on the operation instructions and data sets; the virtual bus is simulated by the communication link of the local area network.

[0125] It should be noted that the above-mentioned method and device are based on the same inventive concept. Since the principles of solving problems by the method and device are similar, the implementation of the device and method can refer to each other, and the repeated parts will not be repeated.

[0126] Based on the above method embodiment, the embodiment of the present invention also proposes a multi-device collaborative working device, including a processor and a memory; the memory stores an application program that can be executed by the processor, which is used to enable the processor to execute the multi-device collaborative working method as described above. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium. In addition, the operating system operating on the computer can also be used to complete part or all of the actual operations based on instructions based on the program code. The program code read from the storage medium can also be written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is made to perform part and all of the actual operations, thereby realizing the functions of any of the above multi-device collaborative working method implementations.

[0127] The memory may be implemented as various storage media such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), and a programmable program read-only memory (PROM). The processor may be implemented as including one or more central processing units or one or more field programmable gate arrays, wherein the field programmable gate array integrates one or more central processing unit cores. Specifically, the central processing unit or the central processing unit core may be implemented as a CPU or an MCU.

[0128] The embodiment of the present application implements a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the multi-device collaborative working method as described above.

[0129] It should be noted that not all steps and modules in the above processes and structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The division of each module is only for the convenience of describing the functional division adopted. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can also be implemented by the same module. These modules can be located in the same device or in different devices.

[0130] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (such as a dedicated processor, such as an FPGA or ASIC) for performing a specific operation. The hardware module may also include a programmable logic device or circuit (such as a general-purpose processor or other programmable processor) temporarily configured by software to perform a specific operation. As for whether to implement the hardware module mechanically, or using a dedicated permanent circuit, or using a temporarily configured circuit (such as configured by software), it can be decided based on cost and time considerations.

[0131] In this article, "schematic" means "serving as an example, instance or explanation", and any diagram or implementation method described as "schematic" in this article should not be interpreted as a more preferred or more advantageous technical solution. In order to make the drawings concise, only the parts related to the present invention are schematically shown in each figure, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and easy to understand, in some figures, only one of the parts with the same structure or function is schematically drawn, or only one of them is marked. In this article, "one" does not mean that the number of the relevant parts of the present invention is limited to "only one", and "one" does not mean that the number of the relevant parts of the present invention is "more than one". In this article, "upper", "lower", "front", "back", "left", "right", "inside", "outside", etc. are only used to indicate the relative position relationship between the relevant parts, rather than to limit the absolute position of these relevant parts.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-device collaborative working method, characterized in that: include: The collaborative control device of the collaborative group monitors the application operation and resource usage status of each member device in the group in real time; All member devices in the collaboration group are in the same local area network; When the first member device needs to start the application running, the collaboration control device determines whether the running requires device collaboration, and if so, determines the hardware that needs collaboration for the running according to the resource usage status of the first member device; the first member device is any member device in the collaboration group; The collaboration control device matches the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; sends a collaboration request instruction and an operation instruction and a data set required for performing corresponding collaboration to each second member device by using a virtual bus of the collaboration group, so as to trigger the second member device to perform corresponding collaborative operations based on the operation instruction and the data set; the virtual bus is simulated by a communication link of the local area network; When the collaborative operation of the second member device is interrupted, suspending the task of the application running and backing up the corresponding breakpoint data; If the interruption is caused by the preemption of the second member device by the new task, the cooperative control device determines whether the execution time of the new task is greater than a preset short task duration threshold, and if so, executes a cooperative transfer process; The collaborative transfer process includes: determining whether the first member device needs to continue to cooperate with the corresponding hardware to run the application; if necessary, matching the hardware corresponding to the collaborative operation within the collaborative group with a third member device for continuing to provide the corresponding collaboration according to the resource usage status of each member device in the collaborative group, and using the virtual bus of the collaborative group to send a collaboration request instruction, the breakpoint data, and an operation instruction and data set required to continue to perform the corresponding collaboration to the third member device, so as to trigger the third member device to continue to provide the corresponding collaborative operation for the first member device; otherwise, sending the breakpoint data to the first member device, so as to trigger the first member device to continue to execute the application at the interrupted breakpoint; If the interruption is caused by the second member device actively giving up providing cooperation, the cooperation control device executes the cooperation transfer process.

2. The method according to claim 1, characterized in that The method further comprises: When a new device is connected to the local area network and the user agrees that the new device joins the collaboration group, the collaboration control device adds the new device to the collaboration group, and obtains and saves the hardware configuration and usage status information of the new device.

3. The method according to claim 1, characterized in that The determining, according to the resource usage status of the first member device, the hardware that needs to cooperate in the operation includes: For each type of hardware required for the operation, it is determined whether the first member device can currently provide corresponding hardware resources to complete the operation according to the current resource usage status of the first member device. If not, it is determined that the hardware is hardware that requires collaboration.

4. The method according to claim 1, characterized in that The step of matching the hardware that needs to cooperate with a second member device for providing corresponding cooperation in the cooperation group includes: For each type of hardware that needs to collaborate, based on the resource usage status of each member device in the collaboration group, search for a member device that can provide corresponding hardware resources from other member devices in the collaboration group except the first member device, and select one member device from the found member devices as the second member device corresponding to the type of hardware.

5. The method according to claim 1, characterized in that The method further comprises: When the collaborative control device receives the termination collaboration instruction from the first member device, the collaborative control device triggers the task of suspending the application running and backs up the corresponding breakpoint data; the breakpoint data is sent to the first member device so that the first member device continues to execute the application running at the interrupted breakpoint.

6. A multi-device collaborative working device, characterized in that: The collaborative control device provided in the collaborative group includes: A monitoring unit, used for monitoring the application running and resource usage status of each member device in the collaboration group in real time; all member devices in the collaboration group are in the same local area network; A collaborative decision unit, configured to, when a first member device needs to start an application, determine whether the operation requires device collaboration by the collaborative control device, and if so, determine the hardware that requires collaboration for the operation according to the resource usage status of the first member device; the first member device is any member device in the collaborative group; A collaboration control unit, used to match the hardware that needs collaboration with a second member device for providing corresponding collaboration within the collaboration group according to the resource usage status of each member device in the collaboration group; using the virtual bus of the collaboration group to send a collaboration request instruction and the operation instructions and data sets required for executing the corresponding collaboration to each second member device, so as to trigger the second member device to perform the corresponding collaboration operation based on the operation instructions and data sets; the virtual bus is simulated by the communication link of the local area network; when the collaboration operation of the second member device is interrupted, the task running by the application is suspended and the corresponding breakpoint data is backed up; if the interruption is caused by the preemption of the second member device by a new task, the collaboration control device determines whether the execution time of the new task is greater than the preset short task duration threshold, and if so, executes the collaboration transfer process; the collaboration The transfer process includes: determining whether the first member device needs the cooperation of corresponding hardware to continue to run the application; if so, matching the hardware corresponding to the collaborative operation within the collaborative group with a third member device for continuing to provide the corresponding collaboration based on the resource usage status of each member device in the collaborative group; and using the virtual bus of the collaborative group to send a collaboration request instruction, the breakpoint data, and the operation instructions and data sets required to continue to perform the corresponding collaboration to the third member device, so as to trigger the third member device to continue to provide the corresponding collaborative operation for the first member device; otherwise, sending the breakpoint data to the first member device, so as to trigger the first member device to continue to execute the application at the interrupted breakpoint; if the interruption is caused by the second member device actively giving up providing collaboration, the collaboration control device executes the collaboration transfer process.

7. A multi-device collaborative working device, characterized in that: including a processor and a memory; The memory stores an application program executable by the processor, which is used to enable the processor to execute the multi-device collaborative working method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored therein, and the computer-readable instructions are used to execute the multi-device collaborative working method as described in any one of claims 1 to 5.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the multi-device collaborative working method described in any one of claims 1 to 5 are implemented.

Images