Control method, device, medium and equipment for unmanned equipment

Abstract

The present disclosure relates to a control method, device, medium and equipment for unmanned equipment, the method comprising: displaying a first virtual device corresponding to a first unmanned equipment in a virtual twin environment corresponding to the environment where the first unmanned equipment is currently located; determining at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment based on the first virtual device; in response to receiving a first collaboration instruction triggered by a user, determining the collaborative virtual device corresponding to the first collaboration instruction from the candidate virtual devices; sending a task collaboration instruction to the control platform of the collaborative unmanned equipment corresponding to the collaborative virtual device, the task collaboration instruction is used to request task collaboration with the collaborative unmanned equipment. In this way, communication and collaboration between multiple unmanned equipment are realized based on the virtual twin environment, the sociability between unmanned equipment is improved, thereby improving the diversity and fun of unmanned equipment control, and reducing the resource idleness of unmanned equipment to a certain extent.

Description

Technical Field

[0001] The present disclosure relates to the field of control of unmanned equipment, and in particular, to a control method, device, medium and equipment for unmanned equipment. Background Art

[0002] With the development of computer technology, the application of unmanned equipment is becoming more and more extensive. At present, it is becoming more and more common for users to remotely control unmanned equipment to perform corresponding tasks.

[0003] In the related art, a communication connection is usually established between a terminal device controlled by a user and an unmanned device, so that the user can control the corresponding operation of the unmanned device based on the terminal device. However, in the above scheme, the user can only control the unmanned device to which he is connected to perform a single task, and it is difficult to perform task collaboration between multiple unmanned devices. Summary of the invention

[0004] The purpose of the present disclosure is to provide a control method, device, medium and equipment for unmanned equipment that can realize task collaboration among multiple unmanned equipment.

[0005] In order to achieve the above object, according to a first aspect of the present disclosure, a control method for unmanned equipment is provided, the method comprising:

[0006] Displaying a first virtual device corresponding to the first unmanned device in a virtual twin environment corresponding to an environment where the first unmanned device is currently located;

[0007] Based on the first virtual device, determining at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment;

[0008] In response to receiving a first collaboration instruction triggered by a user, determining a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices;

[0009] A task collaboration instruction is sent to a control platform of a collaborative unmanned device corresponding to the collaborative virtual device, wherein the task collaboration instruction is used to request task collaboration with the collaborative unmanned device.

[0010] Optionally, in response to receiving a first collaboration instruction triggered by a user, determining a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices includes:

[0011] If the first collaboration instruction is an invitation instruction, determining the candidate virtual device corresponding to the identifier indicated by the invitation instruction as the collaboration virtual device;

[0012] If the first cooperation instruction is a task instruction, each of the candidate virtual devices is determined as the cooperation virtual device.

[0013] Optionally, determining, based on the first virtual device, at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment includes:

[0014] Determine a target position of the first virtual device in the virtual twin environment, and determine a virtual device in the virtual twin environment whose distance from the target position is less than a distance threshold as the candidate virtual device; and / or

[0015] A virtual device in the virtual twin environment that has a friend association identifier with the first virtual device is determined as the candidate virtual device.

[0016] Optionally, the method further comprises:

[0017] Displaying a nearby device list and / or a friend list corresponding to the first virtual device on a display page, wherein the nearby device list includes virtual devices in the virtual twin environment whose distances to the target locations corresponding to the first virtual device are less than a distance threshold, and the friend list includes virtual devices in the virtual twin environment that have friend association identifiers with the first virtual device;

[0018] In response to a selection operation of the user in the displayed page, the first collaboration instruction is generated according to an identifier of the virtual device indicated by the selection operation.

[0019] Optionally, the method further comprises:

[0020] In response to receiving a collaboration confirmation instruction sent by a target unmanned device, obtaining operation information of the target unmanned device in a process of executing a first collaboration task corresponding to the task collaboration instruction, wherein the target unmanned device is the collaboration unmanned device that sent the collaboration confirmation instruction;

[0021] Based on the operation information, a virtual device corresponding to the target unmanned device is displayed in the virtual twin environment.

[0022] Optionally, the method further comprises:

[0023] After the first collaborative task corresponding to the task collaboration instruction is completed, obtaining the task data of each of the target unmanned equipment;

[0024] Determine reward information corresponding to each of the target unmanned equipment according to the task data;

[0025] Based on the reward information, a reward instruction is sent to the control platform of each target unmanned device to reward the target unmanned device.

[0026] Optionally, the method further comprises:

[0027] In response to receiving a second collaboration instruction sent by a control platform of a second unmanned device, determining whether to execute a second collaboration task corresponding to the second collaboration instruction, wherein the second unmanned device is different from the first unmanned device;

[0028] When it is determined to execute the second collaborative task, a collaboration confirmation instruction is sent to the control platform of the second unmanned device, and a control instruction is sent to the first unmanned device to perform task collaboration with the second unmanned device, and the control instruction is generated based on the operation of the first virtual device.

[0029] Optionally, the determining whether to execute the second collaboration task corresponding to the second collaboration instruction includes:

[0030] displaying the task information corresponding to the second collaboration instruction, and determining to execute the second collaboration task in response to receiving a confirmation instruction for the task information; or

[0031] Determine whether the task information corresponding to the second collaborative task satisfies an automatic execution condition, and if the task information satisfies the automatic execution condition, determine to execute the second collaborative task.

[0032] According to a second aspect of the present disclosure, a control device for unmanned equipment is provided, the device comprising:

[0033] A first display module, configured to display a first virtual device corresponding to the first unmanned device in a virtual twin environment corresponding to an environment where the first unmanned device is currently located;

[0034] A first determination module, configured to determine, based on the first virtual device, at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment;

[0035] A second determining module, configured to determine, in response to receiving a first collaboration instruction triggered by a user, a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices;

[0036] The first sending module is used to send a task collaboration instruction to the control platform of the collaborative unmanned device corresponding to the collaborative virtual device, and the task collaboration instruction is used to request task collaboration with the collaborative unmanned device.

[0037] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any method described in the first aspect are implemented.

[0038] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0039] a memory having a computer program stored thereon;

[0040] A processor is used to execute the computer program in the memory to implement the steps of any method described in the first aspect.

[0041] In the above technical solution, communication and collaboration between multiple unmanned devices can be realized based on the virtual twin environment. When controlling the unmanned device to perform a task, a task collaboration request can be initiated to the virtual device in the virtual twin environment, so as to facilitate the collaboration between multiple unmanned devices. Through the above technical solution, communication and collaboration between multiple unmanned devices can be realized based on the virtual twin environment, and the sociality between unmanned devices can be improved, thereby increasing the diversity and fun of unmanned device control, reducing the idle resources of unmanned devices to a certain extent, and improving the efficiency of collaborative task execution and enhancing the user experience.

[0042] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0044] Figure 1 is a flow chart of a control method for unmanned equipment provided according to an embodiment of the present disclosure;

[0045] Figure 2 is a schematic diagram of an interactive system of a control method for unmanned equipment provided in an embodiment of the present disclosure;

[0046] Figure 3 is a block diagram of a control device for unmanned equipment provided according to an embodiment of the present disclosure;

[0047] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0048] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0049] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0050] Figure 1As shown, it is a flow chart of a control method of an unmanned device provided according to an embodiment of the present disclosure, such as Figure 1 As shown, the method may include:

[0051] In step 11, a first virtual device corresponding to the first unmanned device is displayed in a virtual twin environment corresponding to an environment in which the first unmanned device is currently located.

[0052] The first unmanned device may be a robot or an unmanned delivery device, and the method may be applied to a control platform corresponding to the first unmanned device, which may be a terminal device such as a mobile phone, or a wearable device, etc. In this embodiment, a virtual twin environment corresponding to the current environment may be generated based on the digital twin technology, that is, for each object in the actual environment, such as items, obstacles, unmanned devices, etc. in the environment, a corresponding virtual object is generated in the virtual twin environment through three-dimensional modeling, so that the user can understand the corresponding operations in the current actual environment in real time through the virtual twin environment.

[0053] In step 12, based on the first virtual device, at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment is determined.

[0054] There may be multiple virtual devices in the virtual twin environment, so when controlling the first unmanned device to perform a corresponding task, it can collaborate with the unmanned devices corresponding to the virtual twin environment and other virtual devices. Figure 2 As shown, the virtual twin environment can be realized by a digital twin server. The unmanned devices A1 and A2 in the real environment can be connected to the digital twin server after startup, that is, connected to the virtual twin environment, and the control platforms B1 and B2 corresponding to each unmanned device are also connected to the digital twin server after startup to connect to the virtual twin environment.

[0055] Each unmanned device in the actual environment will have its corresponding virtual device in the virtual twin environment, such as Figure 2 As shown in examples a1 and a2, each unmanned device will establish a connection with the virtual twin environment. In this step, a candidate virtual device that can perform task collaboration with the first virtual device can be determined in the virtual twin environment in order to initiate a collaborative task.

[0056] In step 13, in response to receiving a first collaboration instruction triggered by a user, a collaboration virtual device corresponding to the first collaboration instruction is determined from candidate virtual devices.

[0057] For example, a user can perform operations in the control platform to trigger a first collaboration instruction, which may include relevant information about the task that requires collaboration. For example, a user can set the task that requires collaboration as a picking task in the control platform, and enter the quantity picked, the type of picked, etc. to trigger the first collaboration instruction. Then, in response to the first collaboration instruction, the collaborative virtual devices that can participate in the task collaboration can be further determined from the candidate virtual devices.

[0058] In step 14, a task collaboration instruction is sent to the control platform of the collaborative unmanned device corresponding to the collaborative virtual device, wherein the task collaboration instruction is used to request task collaboration with the collaborative unmanned device.

[0059] Accordingly, the user can control the virtual device in the virtual twin environment based on the control platform, thereby realizing the operation control of the unmanned device corresponding to the virtual device. Therefore, in this embodiment, after the collaborative virtual device is determined, the first collaboration instruction can be synchronized to the digital twin server, and then the corresponding task collaboration instruction can be sent to the control platform of each collaborative unmanned device connected to it through the digital twin server, so as to display the information corresponding to the task collaboration instruction in the control platform of the collaborative unmanned device, and then the task collaboration can be realized based on the operation of the user of the collaborative unmanned device.

[0060] In the above technical solution, communication and collaboration between multiple unmanned devices can be realized based on the virtual twin environment. When controlling the unmanned device to perform a task, a task collaboration request can be initiated to the virtual device in the virtual twin environment, so as to facilitate the collaboration between multiple unmanned devices. Through the above technical solution, communication and collaboration between multiple unmanned devices can be realized based on the virtual twin environment, and the sociality between unmanned devices can be improved, thereby increasing the diversity and fun of unmanned device control, reducing the idle resources of unmanned devices to a certain extent, and improving the efficiency of collaborative task execution and enhancing the user experience.

[0061] In a possible embodiment, an exemplary implementation of determining at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment based on the first virtual device is as follows. This step may include at least one of the following:

[0062] The first method is to determine the target position of the first virtual device in the virtual twin environment, and determine the virtual device in the virtual twin environment whose distance to the target position is less than a distance threshold as the candidate virtual device.

[0063] Among them, the first virtual device can be positioned based on the map corresponding to the virtual twin environment, so as to obtain the target position corresponding to the first virtual device. In actual application scenarios, the task collaboration of multiple unmanned devices usually requires multiple unmanned devices to move to the same target location. Therefore, in this embodiment, virtual devices near the first virtual device can be used as candidate virtual devices, thereby reducing the energy consumption of the unmanned device during movement.

[0064] For example, the distance between the position of each virtual device and the target position can be calculated. If the distance is less than the distance threshold, the virtual device is considered to be near the first virtual device, and its corresponding unmanned device is considered to be able to perform the corresponding task together with the first unmanned device. That is, the virtual device can be used as a candidate virtual device so that it can be requested to perform task collaboration, thereby reducing the resource consumption of each unmanned device during the execution of the collaborative task and ensuring the effectiveness and reliability of the unmanned device task execution.

[0065] The second method is to determine a virtual device that has a friend association identifier with the first virtual device in the virtual twin environment as a candidate virtual device.

[0066] In actual application scenarios, users can usually invite their friends to complete tasks together when collaborating on tasks. Accordingly, virtual devices in a virtual twin environment can socialize with each other, thereby establishing a friendship relationship. A friend association identifier can be set between two virtual devices that have established a friendship relationship. In this embodiment, a friend device associated with the first virtual device can be obtained, and the virtual device with the friend association identifier can be determined as a candidate virtual device, so as to invite friends to collaborate on tasks, thereby improving the efficiency and completion of task execution while improving the fun and sociality of task execution.

[0067] In a possible embodiment, the method further includes:

[0068] A nearby device list and / or a friend list corresponding to the first virtual device is displayed on a display page, wherein the nearby device list includes virtual devices in the virtual twin environment whose distance to the target location corresponding to the first virtual device is less than a distance threshold, and the friend list includes virtual devices in the virtual twin environment that have a friend association identifier with the first virtual device.

[0069] As an example, the display page can be a page in the control platform for displaying each virtual device, such as a page of a virtual twin environment, and multiple virtual devices can be displayed on the page. For example, the nearby devices corresponding to the first virtual device can be determined in the manner described above, and then displayed in the form of a list in the display page, so that the user can understand the virtual devices that can be invited to collaborate. The display method for the friend list is similar to that for the nearby device list, and will not be repeated here. Thus, each determined candidate virtual device can be displayed for user viewing and selection.

[0070] In response to a selection operation of the user in the displayed page, the first collaboration instruction is generated according to an identifier of the virtual device indicated by the selection operation.

[0071] Continuing with the above example, when a user wants to invite a specific device to collaborate, this can be achieved by sending an invitation instruction to the specific device. In this embodiment, the selection operation can be in the form of checking and submitting. Taking the nearby device list as an example, after the nearby device list is displayed on the display page, the user can select one or more virtual devices from the list and check the candidate virtual devices that he wants to invite to collaborate, and then select submit after confirmation. Furthermore, in response to the submission operation, a first collaboration instruction can be generated, and the first collaboration instruction can include identifiers corresponding to each virtual device checked by the user, and the first collaboration instruction is used to indicate an invitation to the devices corresponding to each identifier to collaborate on a task.

[0072] Therefore, through the above technical solution, nearby devices, friend devices, etc. corresponding to the first virtual device can be displayed in the display page, so that users can choose the devices that need to be invited to task collaboration according to their own needs, which meets the user's usage needs, enhances the sociality and personalization of task collaboration, and further improves the user experience.

[0073] In a possible embodiment, in response to receiving a first collaboration instruction triggered by a user, an exemplary implementation of determining a collaboration virtual device corresponding to the first collaboration instruction from candidate virtual devices is as follows, and the step may include:

[0074] If the first collaboration instruction is an invitation instruction, the candidate virtual device corresponding to the identifier indicated by the invitation instruction is determined as the collaboration virtual device.

[0075] As an example, a user can selectively invite unmanned devices for task collaboration. The invitation instruction is an instruction for an unmanned device specified by the user, and the invitation instruction may include the identifier of at least one virtual device specified by the user. For example, the determined candidate virtual devices can be displayed to the user through an operating platform, and the user can select at least one virtual device as a candidate virtual device, thereby triggering the first collaboration instruction. The device identifier in the instruction may include the identifiers of the candidate virtual devices selected by the user, and further, the candidate virtual devices corresponding to the identifier are used as collaborative virtual devices. Thus, the user can specify the unmanned device for the execution of the parameter collaborative task, so that a task collaboration request can be initiated for a specific unmanned device, which meets the user's usage needs and increases the diversity of collaborative task execution.

[0076] If the first cooperation instruction is a task instruction, each of the candidate virtual devices is determined as the cooperation virtual device.

[0077] As another example, a user can publish a task to have other unmanned devices confirm whether to accept the collaborative task to achieve task collaboration. For example, the task instruction may include but is not limited to the workload of the task, the working method, etc. If the device identifier in the first collaborative instruction is empty, it means that the user has not specified the unmanned device to invite. In this case, each of the candidate virtual devices determined can be determined as the collaborative virtual device, so that a request for task collaboration can be sent to the unmanned device corresponding to each collaborative virtual device, so as to determine the devices participating in the task collaboration through the responses of each unmanned device. Therefore, by inviting other unmanned devices to perform task collaboration by publishing tasks, the scope of unmanned devices participating in task collaboration can be further expanded, providing a basis for establishing friendships between subsequent unmanned devices, further increasing the fun of task collaboration, and improving the user experience.

[0078] In a possible embodiment, the method may further include:

[0079] In response to receiving a collaboration confirmation instruction sent by a target unmanned device, obtaining operation information of the target unmanned device in a process of executing a first collaboration task corresponding to the task collaboration instruction, wherein the target unmanned device is the collaboration unmanned device that sent the collaboration confirmation instruction;

[0080] The target unmanned equipment is displayed in the virtual twin environment based on the operation information.

[0081] Among them, after sending a task collaboration instruction to each collaborative unmanned device, if the user operating the collaborative unmanned device confirms to accept the first collaborative task corresponding to the task collaboration instruction, he can send a collaboration confirmation instruction to the control platform of the first unmanned device through the control platform that controls the collaborative unmanned device to determine that the collaborative unmanned device (i.e., the target unmanned device) that sends the collaboration confirmation instruction can perform task collaboration with the first unmanned device.

[0082] Furthermore, during the execution of the first collaborative task, the users corresponding to the various unmanned devices participating in the task collaboration can control the operation of the corresponding virtual devices through the control platform and the virtual twin environment to complete the first collaborative task. In this process, the operation information of the target unmanned device during the task execution can be collected in real time. The operation information can be the action data and status data of the unmanned device, thereby realizing real-time monitoring of the operation of the target unmanned device.

[0083] Furthermore, based on the operation information, the virtual device corresponding to the target unmanned device can be displayed in the virtual twin environment, so that the user can observe the operation of each unmanned device participating in the task collaboration, so that the user can timely and comprehensively understand the progress of task execution, in line with the user's usage needs, and at the same time, the operation of each unmanned device during the task execution process can also be monitored and stored, providing effective and comprehensive data support for subsequent task analysis.

[0084] In a possible embodiment, the method may further include:

[0085] After the first cooperative task corresponding to the task cooperative instruction is completed, the task data of each target unmanned device is obtained, wherein the task data may include but is not limited to the workload, working time, work consumption, etc. performed by the target unmanned device.

[0086] Determine reward information corresponding to each target unmanned equipment according to the task data.

[0087] Among them, the working conditions that each target unmanned device can provide during the execution of the task are not necessarily the same, so it is not appropriate to distribute the reward equally when distributing the reward. Therefore, in this embodiment, the reward can be distributed according to the task data corresponding to each target unmanned device.

[0088] For example, rewards can be distributed based on the proportion of workloads corresponding to each target unmanned device and the total reward, so that target unmanned devices with high workloads can get more rewards, which is in line with actual application scenarios. The total reward can be the reward value set by the user when creating a collaborative task.

[0089] As another example, allocation can be made in combination with workload and working hours. For example, the workload and working hours can be normalized separately, and then the allocation parameters of the target unmanned equipment are obtained by taking the weighted sum of the normalized workload and working time of each target unmanned equipment. Thereafter, the allocation ratio is determined based on the allocation parameters corresponding to each target unmanned equipment, and then the reward information corresponding to each target unmanned equipment is obtained based on the total reward.

[0090] Based on the reward information, a reward instruction is sent to the control platform of each target unmanned device to reward the target unmanned device.

[0091] Therefore, through the above-mentioned technical scheme, data from the process of each target unmanned equipment performing collaborative tasks can be collected and stored, so that rewards can be distributed based on the working conditions of each target unmanned equipment. On the one hand, it can effectively improve the participation and efficiency of task execution of each target unmanned equipment. On the other hand, it can reasonably distribute rewards to the target unmanned equipment, further improve the fun of unmanned equipment control, and enhance user participation and user experience.

[0092] In a possible embodiment, after rewarding each target unmanned device, the users corresponding to the target unmanned devices can also evaluate and make suggestions to each other. The evaluation and suggestion data can be associated and stored through the operation information of each target unmanned device during the task execution process. The users corresponding to the target unmanned devices can browse, review and share historical collaboration data through the control platform, further enhancing the social nature of controlling the unmanned devices for task collaboration.

[0093] In actual application scenarios, the first unmanned device can send task cooperation instructions to other unmanned devices, and it can also receive task cooperation instructions sent by other unmanned devices to determine whether it wants to participate in the tasks initiated by other unmanned devices. Based on this, the present disclosure also provides the following embodiments.

[0094] In a possible embodiment, the method further includes:

[0095] In response to receiving a second collaboration instruction sent by a control platform of a second unmanned device, determining whether to execute a second collaboration task corresponding to the second collaboration instruction, wherein the second unmanned device is different from the first unmanned device.

[0096] For example, the second unmanned device can be any unmanned device except the first unmanned device. The user controlling the second unmanned device can initiate a second collaborative task through the control platform of the second unmanned device in the same way as the first unmanned device initiated the first collaborative task mentioned above, which will not be repeated here.

[0097] When the operating platform of the first unmanned device receives the second collaboration instruction sent by the control platform of the second unmanned device, that is, when the second unmanned device requests the first unmanned device to jointly perform a task, it can first determine whether the first unmanned device wants to perform the second collaboration task.

[0098] For example, the exemplary implementation of determining whether to execute the second collaboration task corresponding to the second collaboration instruction is as follows. This step may include:

[0099] In a first manner, task information corresponding to the second collaboration instruction is displayed, and in response to receiving a confirmation instruction corresponding to the task information, execution of the second collaboration task is determined.

[0100] In this step, the task information corresponding to the second collaborative instruction can be displayed in the operating platform of the first unmanned device, so that the user can be prompted to understand the specific content of the task. Afterwards, the user can choose whether to accept the task according to his or her own needs or interests. Furthermore, the historical tasks issued by the second unmanned device can be displayed, so that the user can determine whether to accept the second collaborative task based on the execution status of the historical tasks. If you are sure to accept, you can click Confirm in the operating platform to trigger the confirmation instruction. In response to the confirmation instruction, it can be determined that the first unmanned device confirms the execution of the second collaborative task initiated by the second unmanned device.

[0101] In this embodiment, collaborative tasks initiated by other unmanned devices can be displayed to the user so that the user can select the collaborative tasks he wants to perform and control the corresponding unmanned devices to execute them. This can improve the flexibility and diversity of initiating collaborative tasks, improve the personalization of unmanned device control, meet user usage needs, and enhance user experience.

[0102] Or a second way is to determine whether the task information corresponding to the second collaborative task satisfies the automatic execution condition, and if the task information satisfies the automatic execution condition, determine to execute the second collaborative task.

[0103] Among them, the automatic execution condition can be set based on the actual application scenario. For example, the automatic execution condition can be that the workload is less than a preset workload threshold, that is, the unmanned device can automatically execute tasks with a smaller workload. Accordingly, after receiving the second collaborative task, the task workload in the task information can be obtained. If the task workload is less than the workload threshold, it is determined that the automatic execution condition is met, and then the first unmanned device is determined to execute the second collaborative task. In this way, it is possible to automatically accept collaborative tasks that meet the conditions without user judgment, thereby improving the efficiency of collaborative task execution and reducing user operations to a certain extent.

[0104] Afterwards, when it is determined to execute the second collaborative task, a collaboration confirmation instruction is sent to the control platform of the second unmanned device, and a control instruction is sent to the first unmanned device to perform task collaboration with the second unmanned device, and the control instruction is generated based on the operation of the first virtual device.

[0105] Accordingly, after the first unmanned device confirms that it has accepted the second collaborative task, it can send a confirmation instruction to the digital twin server, and then send a collaborative confirmation instruction to the control platform of the second unmanned device to notify the second unmanned device. Furthermore, after the first unmanned device accepts the second collaborative task, it needs to perform corresponding operations to complete the second collaborative task. In this process, the user can control the first virtual device in the virtual twin environment to generate a control instruction, or the first virtual device in the virtual twin environment can be operated based on an automatic control method in response to the second collaborative task to generate the control instruction, and then the control instruction is sent to the first unmanned device, so that the first unmanned device can operate and perform the second collaborative task.

[0106] Therefore, through the above technical solution, each unmanned device can participate in collaborative tasks initiated by other unmanned devices in the virtual twin environment, simplify the process of task collaboration between multiple unmanned devices, thereby improving the efficiency of task collaboration, providing support for multi-user control of unmanned devices for task collaboration, and further enhancing the user experience.

[0107] The present invention provides a control device for unmanned equipment, such as Figure 3 As shown, the device 10 comprises:

[0108] A first display module 100, configured to display a first virtual device corresponding to the first unmanned device in a virtual twin environment corresponding to an environment where the first unmanned device is currently located;

[0109] A first determination module 200, configured to determine, based on the first virtual device, at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment;

[0110] A second determination module 300 is configured to determine, in response to receiving a first collaboration instruction triggered by a user, a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices;

[0111] The first sending module 400 is used to send a task collaboration instruction to the control platform of the collaborative unmanned device corresponding to the collaborative virtual device, and the task collaboration instruction is used to request task collaboration with the collaborative unmanned device.

[0112] Optionally, the second determining module includes:

[0113] A first determining submodule, configured to determine, if the first collaboration instruction is an invitation instruction, a candidate virtual device corresponding to an identifier indicated by the invitation instruction as the collaboration virtual device;

[0114] The second determining submodule is configured to determine each of the candidate virtual devices as the cooperative virtual device if the first cooperative instruction is a task instruction.

[0115] Optionally, the first determining module includes:

[0116] a third determination submodule, configured to determine a target position of the first virtual device in the virtual twin environment, and determine a virtual device in the virtual twin environment whose distance from the target position is less than a distance threshold as the candidate virtual device; and / or

[0117] The fourth determination submodule is used to determine a virtual device in the virtual twin environment that has a friend association identifier with the first virtual device as the candidate virtual device.

[0118] Optionally, the device further comprises:

[0119] A second display module is used to display a nearby device list and / or a friend list corresponding to the first virtual device in a display page, wherein the nearby device list includes virtual devices in the virtual twin environment whose distance to the target location corresponding to the first virtual device is less than a distance threshold, and the friend list includes virtual devices in the virtual twin environment that have a friend association identifier with the first virtual device;

[0120] A generating module is used to generate the first collaboration instruction in response to a selection operation of the user in the display page and according to the identifier of the virtual device indicated by the selection operation.

[0121] Optionally, the device further comprises:

[0122] A first acquisition module is configured to, in response to receiving a collaboration confirmation instruction sent by a target unmanned device, acquire operation information of the target unmanned device in a process of executing a first collaborative task corresponding to the task collaboration instruction, wherein the target unmanned device is the collaborative unmanned device that sends the collaboration confirmation instruction;

[0123] The third display module is used to display the virtual device corresponding to the target unmanned device in the virtual twin environment based on the operation information.

[0124] Optionally, the device further comprises:

[0125] A second acquisition module is used to acquire the task data of each of the target unmanned equipment after the first collaborative task corresponding to the task collaboration instruction is completed;

[0126] A third determination module is used to determine reward information corresponding to each of the target unmanned equipment according to the task data;

[0127] The second sending module is used to send a reward instruction to the control platform of each target unmanned device based on the reward information to reward the target unmanned device.

[0128] Optionally, the device further comprises:

[0129] A fourth determination module, configured to determine whether to execute a second collaboration task corresponding to a second collaboration instruction received from a control platform of a second unmanned device, wherein the second unmanned device is different from the first unmanned device;

[0130] The third sending module is used to send a collaboration confirmation instruction to the control platform of the second unmanned device and send a control instruction to the first unmanned device to perform task collaboration with the second unmanned device when it is determined to execute the second collaborative task, and the control instruction is generated based on the operation of the first virtual device.

[0131] Optionally, the fourth determining module includes:

[0132] a fifth determination submodule, configured to display task information corresponding to the second collaboration instruction, and determine to execute the second collaboration task in response to receiving a confirmation instruction for the task information; or

[0133] The sixth determination submodule is used to determine whether the task information corresponding to the second collaborative task meets the automatic execution condition, and if the task information meets the automatic execution condition, determine to execute the second collaborative task.

[0134] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0135] Figure 4 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 4 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0136] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the control method of the unmanned device described above. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0137] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned control method of unmanned equipment.

[0138] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the control method of the unmanned device described above are implemented. For example, the computer-readable storage medium may be the memory 702 including the program instructions, and the program instructions may be executed by the processor 701 of the electronic device 700 to complete the control method of the unmanned device described above.

[0139] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned control method of the unmanned equipment when executed by the programmable device.

[0140] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0141] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0142] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A control method for unmanned equipment, characterized in that: The method comprises: Displaying a first virtual device corresponding to the first unmanned device in a virtual twin environment corresponding to an environment where the first unmanned device is currently located; Based on the first virtual device, determining at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment; In response to receiving a first collaboration instruction triggered by a user, determining a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices; Sending a task collaboration instruction to a control platform of a collaborative unmanned device corresponding to the collaborative virtual device, wherein the task collaboration instruction is used to request task collaboration with the collaborative unmanned device; The step of, in response to receiving a first collaboration instruction triggered by a user, determining a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices includes: If the first collaboration instruction is an invitation instruction, determining the candidate virtual device corresponding to the identifier indicated by the invitation instruction as the collaboration virtual device; If the first collaboration instruction is a task instruction, determining each of the candidate virtual devices as the collaboration virtual device; Wherein, determining at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment based on the first virtual device includes: Determine a target position of the first virtual device in the virtual twin environment, and determine a virtual device in the virtual twin environment whose distance from the target position is less than a distance threshold as the candidate virtual device; and / or Determine a virtual device in the virtual twin environment that has a friend association identifier with the first virtual device as the candidate virtual device; Displaying a nearby device list and / or a friend list corresponding to the first virtual device on a display page, wherein the nearby device list includes virtual devices in the virtual twin environment whose distances to the target locations corresponding to the first virtual device are less than a distance threshold, and the friend list includes virtual devices in the virtual twin environment that have friend association identifiers with the first virtual device; In response to a selection operation of the user in the displayed page, the first collaboration instruction is generated according to an identifier of the virtual device indicated by the selection operation.

2. The method according to claim 1, characterized in that The method further comprises: In response to receiving a collaboration confirmation instruction sent by a target unmanned device, obtaining operation information of the target unmanned device in a process of executing a first collaboration task corresponding to the task collaboration instruction, wherein the target unmanned device is the collaboration unmanned device that sent the collaboration confirmation instruction; Based on the operation information, a virtual device corresponding to the target unmanned device is displayed in the virtual twin environment.

3. The method according to claim 2, characterized in that The method further comprises: After the first collaborative task corresponding to the task collaboration instruction is completed, obtaining the task data of each of the target unmanned equipment; Determine reward information corresponding to each of the target unmanned equipment according to the task data; Based on the reward information, a reward instruction is sent to the control platform of each target unmanned device to reward the target unmanned device.

4. The method according to claim 1, characterized in that: The method further comprises: In response to receiving a second collaboration instruction sent by a control platform of a second unmanned device, determining whether to execute a second collaboration task corresponding to the second collaboration instruction, wherein the second unmanned device is different from the first unmanned device; When it is determined to execute the second collaborative task, a collaboration confirmation instruction is sent to the control platform of the second unmanned device, and a control instruction is sent to the first unmanned device to perform task collaboration with the second unmanned device, and the control instruction is generated based on the operation of the first virtual device.

5. The method according to claim 4, characterized in that The determining whether to execute the second collaboration task corresponding to the second collaboration instruction includes: displaying the task information corresponding to the second collaboration instruction, and determining to execute the second collaboration task in response to receiving a confirmation instruction for the task information; or Determine whether the task information corresponding to the second collaborative task satisfies an automatic execution condition, and if the task information satisfies the automatic execution condition, determine to execute the second collaborative task.

6. A control device for unmanned equipment, characterized in that: The device comprises: A first display module, configured to display a first virtual device corresponding to the first unmanned device in a virtual twin environment corresponding to an environment where the first unmanned device is currently located; A first determination module, configured to determine, based on the first virtual device, at least one candidate virtual device corresponding to the first virtual device in the virtual twin environment; A second determining module, configured to determine, in response to receiving a first collaboration instruction triggered by a user, a collaboration virtual device corresponding to the first collaboration instruction from the candidate virtual devices; A first sending module, used for sending a task collaboration instruction to a control platform of a collaborative unmanned device corresponding to the collaborative virtual device, wherein the task collaboration instruction is used for requesting task collaboration with the collaborative unmanned device; Wherein, the second determining module includes: A first determining submodule, configured to determine, if the first collaboration instruction is an invitation instruction, a candidate virtual device corresponding to an identifier indicated by the invitation instruction as the collaboration virtual device; A second determination submodule, configured to determine each of the candidate virtual devices as the collaborative virtual device if the first collaboration instruction is a task instruction; Wherein, the first determining module includes: a third determination submodule, configured to determine a target position of the first virtual device in the virtual twin environment, and determine a virtual device in the virtual twin environment whose distance from the target position is less than a distance threshold as the candidate virtual device; and / or a fourth determination submodule, configured to determine a virtual device having a friend association identifier with the first virtual device in the virtual twin environment as the candidate virtual device; The device also includes: A second display module is used to display a nearby device list and / or a friend list corresponding to the first virtual device in a display page, wherein the nearby device list includes virtual devices in the virtual twin environment whose distance to the target location corresponding to the first virtual device is less than a distance threshold, and the friend list includes virtual devices in the virtual twin environment that have a friend association identifier with the first virtual device; A generating module is used to generate the first collaboration instruction in response to a selection operation of the user in the display page and according to the identifier of the virtual device indicated by the selection operation.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

Images