Multi-object interaction assistance method and device, equipment and storage medium

Abstract

The invention discloses a multi-object interaction assistance method and device, equipment and a storage medium, and the method comprises the steps: receiving assistance information, the assistance information comprises basic information of an assisted object, and the basic information at least comprises position information and identity recognition information; determining an assistance mode required by the assisted object based on the assistance information; and performing corresponding assistance operation on the assisted object based on the assistance mode. The assistance mode required by the assisted object is determined based on the assistance information, and the corresponding assistance operation is performed on the assisted object, so that the assistance timeliness and efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of electronic assistance technology, and in particular to a multi-object interaction assistance method, apparatus, device and storage medium. Background Technology

[0002] With the development of smart devices, they can replace human labor to perform a variety of complex tasks. However, with large-scale use, smart devices may encounter problems such as falls, external impacts, or various hardware and software issues, preventing them from completing tasks successfully and leading to task failure. Therefore, how to enable smart devices to continue obstacle avoidance and rapid repair after suspected damage has become an important issue in smart device research.

[0003] Currently, when smart devices show signs of suspected damage, it's difficult to determine whether they can continue operating. This typically requires manual inspection or transport to a repair area for rapid repair. However, in scenarios with high usage of smart devices, or in unmanned or fully automated environments, waiting for manual intervention to inspect and transport the devices to the repair area is time-consuming and labor-intensive, thus reducing the efficiency of assistance. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for assisting multi-object interaction.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a multi-object interaction assistance method, comprising: receiving assistance information, the assistance information including basic information of the assisted object, the basic information including at least location information and identity recognition information; determining the required assistance mode of the assisted object based on the assistance information; and performing corresponding assistance operations on the assisted object based on the assistance mode.

[0007] Understandably, on the one hand, after receiving assistance requests from the recipient, assistance operations are performed based on the requests, avoiding the time spent waiting for repair personnel and improving the timeliness of assistance. On the other hand, the system determines the required assistance mode and provides different assistance modes. Multiple assistance modes ensure assistance in various scenarios, improving the flexibility and accuracy of assistance, and thus enhancing the timeliness and efficiency of assistance.

[0008] In some embodiments, the method further includes: moving to an assistance location based on assistance information; and at the assistance location, determining the required assistance mode for the assisted object.

[0009] Understandably, based on assistance information, the system moves to the assistance location, and at that location, determines the required assistance mode for the assisted individual. This provides the feasibility of determining the assistance mode when the assisted individual needs to rely on other devices for assistance, thereby improving the feasibility and flexibility of assistance in multiple scenarios.

[0010] In some embodiments, receiving assistance information includes one or more of the following: receiving a first message sent from the cloud, the first message carrying assistance information; receiving a broadcast from the cloud; the broadcast being used to instruct the assisted object to request assistance and the conditions for assistance; receiving assistance information if the assisted object's own capabilities meet the conditions for assistance; receiving assistance information sent by the assisted object if the assisted object is discovered; and receiving basic information sent by the assisted object if the assisted object is discovered, and generating assistance information based on the basic information.

[0011] Understandably, there are various ways to receive assistance information. Assistance information can be obtained by receiving the first message or broadcast from the cloud, or by receiving assistance information sent by the recipient or by receiving assistance information generated from basic information. Based on this, multiple methods of disseminating assistance information are provided, improving the effectiveness and flexibility of assistance information dissemination.

[0012] In some embodiments, determining the required assistance mode for the assisted object based on assistance information includes:

[0013] If the assistance information comes from the cloud or is sent directly by the assisted party, the required assistance mode for the assisted party is determined to be the first assistance mode, which instructs the assisted party to perform assistance operations based on the assistance information; or, if the assistance information is generated based on basic information, the required assistance mode for the assisted party is determined to be the second assistance mode, which instructs the assisted party to perform assistance operations under the guidance of the assisting party.

[0014] Understandably, depending on the different sources of assistance information, the assistance mode is determined to be either the first assistance mode or the second assistance mode, thereby providing targeted assistance to the recipient. If the recipient is capable of completing the assistance operation on their own, they will do so (first assistance mode), avoiding the waste of time caused by waiting for other recipients to provide assistance. If the recipient is unable to complete the assistance operation on their own, the assisting recipient will guide them to complete the assistance operation (second assistance mode). Based on this, the timeliness and efficiency of assistance are improved.

[0015] In some embodiments, performing corresponding assistance operations on the assisted object based on the assistance mode includes: when the assistance mode is a first assistance mode, sending correction information to the assisted object so that the assisted object can perform corresponding navigation correction or be guided back to the specified area according to the correction information, thereby realizing the assistance operation.

[0016] Understandably, in the first assistance mode, sending correction information enables the assisted party to make corresponding navigation corrections or be guided back to the designated area based on the correction information, thereby improving the accuracy and effectiveness of assistance in the first assistance mode.

[0017] In some embodiments, the corresponding assistance operation is performed on the assisted object based on the assistance mode, including: when the assistance mode is the second assistance mode, the assisted object is guided to follow by navigation planning or returning to a specified area to realize the assistance operation.

[0018] Understandably, in the second assistance mode, by planning navigation or returning to a designated area, the assisted object is guided to follow, providing a feasible way to assist in scenarios where the assisted object cannot perform assistance operations on its own.

[0019] In some embodiments, determining the required assistance mode for the assisted object based on assistance information includes: if the basic information of the assisted object includes fault information, determining the assistance mode of the assisted object based on the fault information; the fault information includes at least whether the navigation function of the assisted object is normal.

[0020] Understandably, different assistance modes are distinguished based on fault information, and different assistance modes are determined for the controlled objects in different states of the navigation and obstacle avoidance function, thereby improving the effectiveness of assistance.

[0021] In some embodiments, performing corresponding assistance operations on the assisted object based on the assistance mode includes: when the assistance mode is a first assistance mode, obtaining a first model of the assisted object and sending the first model to the assisted object to assist the assisted object in performing the assistance operation; the first model belongs to correction information; wherein, the first model is used to indicate the current component appearance and / or posture of the assisted object; the first assistance mode indicates that when the navigation function of the assisted object is normal, assisting the assisted object to navigate itself.

[0022] Understandably, different assistance modes are distinguished based on fault information, and different assistance modes are determined for the controlled objects in different states of the navigation and obstacle avoidance function, thereby improving the effectiveness of assistance.

[0023] In some embodiments, performing corresponding assistance operations on the assisted object based on the assistance mode includes: when the assistance mode is the second assistance mode, obtaining the first model of the assisted object, and providing navigation assistance guidance to the assisted object based on the assistance location and the first model, thereby assisting the assisted object in performing the assistance operation; wherein, when the second assistance mode indicates that the navigation function of the assisted object is abnormal, navigation is performed through the first model to guide the assisted object to follow.

[0024] Understandably, for the first assistance mode, the method of obtaining the first model and sending it to the assisted object allows the assisted object to use the first model to achieve navigation and obstacle avoidance, avoiding waiting for other assisted objects or maintenance personnel to come and assist, effectively saving assistance time, and thus improving assistance efficiency.

[0025] In some embodiments, assisting the assisted object to perform assisted operations by providing navigation guidance based on the assisted position and the first model includes: when in the assisted position, determining motion information and / or joint information based on the first model, and providing assisting the assisted object to perform navigation guidance based on the motion information and / or joint information, thereby assisting the assisted object to perform assisted operations.

[0026] Understandably, in the second assistance mode, when the navigation and obstacle avoidance function of the assisted object is abnormal, an assistance object is arranged to come and assist with navigation based on the assistance location. The assistance object enables the assisted object to receive assistance, avoiding waiting for maintenance personnel to come and effectively saving assistance time, thereby improving the efficiency of assistance.

[0027] In some embodiments, based on a first model, motion information and / or joint information of the assisted object are determined, and navigation assistance guidance is provided to the assisted object based on the motion information and / or joint information to assist the assisted object in performing assisted operations. This includes: fusing the first model with a second model of the assisted object to determine a third model, where the second model is the original model of the assisted object; performing navigation obstacle avoidance based on the third model; determining the motion information and / or joint information of the assisted object; sending the motion information and / or joint information to the assisted object; providing navigation assistance guidance to the assisted object to assist the assisted object in performing assisted operations; and using the motion information and / or joint information to guide the movement and following of the assisted object.

[0028] Understandably, by defining a third model whose outline is the largest outer outline resulting from the superposition of the projections of the assisted object and the object being assisted, the assisted object can navigate using the third model as a standard, ensuring that both the assisted object and the object being assisted can successfully pass through obstacles, thus ensuring navigation safety.

[0029] In some embodiments, navigation is performed based on a third model to determine the motion information and / or joint information of the assisted object, and the motion information and / or joint information is sent to the assisted object to provide navigation and obstacle avoidance assistance, thereby assisting the assisted object in performing the assisted operation. This includes: determining a first position based on the assisted position, wherein the first position is the starting point of navigation set by the assisted object; determining first information based on the first position and position information in the basic information, wherein the first information indicates the relative position between the first position and the current position of the assisted object; performing navigation based on the third model and the first position to determine the motion information and / or joint information at different times; and, when the assisted object reaches the first position based on the first information, sending the motion information and / or joint information generated by the navigation to the assisted object to provide navigation and obstacle avoidance assistance, thereby assisting the assisted object in performing the assisted operation.

[0030] It is understandable that setting a first position and sending the first information enables the assisted object to reach the first position, ensuring that the assisted object and the assisting object have the same starting point before navigation and obstacle avoidance. Then, based on the first position, the third model is used for navigation and obstacle avoidance to determine the motion information and / or joint information at different times, so that the assisted object can accurately follow the assisting object for navigation and obstacle avoidance. Based on this, the accuracy of navigation and obstacle avoidance is improved.

[0031] In some embodiments, when the assisted object reaches a first position based on the first information, the motion information and / or joint information generated by navigation are sent to the assisted object, including: if the identification information indicates that the assisted object and the assisting object have different models, then when the assisted object reaches a first position based on the first information, the motion information is sent to the assisted object; or, if the identification information indicates that the assisted object and the assisting object have the same model, then when the assisted object reaches a first position based on the first information, the joint information is sent to the assisted object.

[0032] Understandably, determining different assistance methods based on whether the assisted object and the assisted object are the same improves the flexibility of the assisted object in navigation and obstacle avoidance. When the assisted object and the assisted object are the same model, only the joint information that is easier to calculate and process needs to be sent to the assisted object. The assisted object can obtain the joint information required for navigation and obstacle avoidance more quickly, thereby improving the speed of the assisted object in navigation and obstacle avoidance.

[0033] In some embodiments, when the assisted object reaches a first position based on the first information, sending the motion information and / or joint information generated by navigation to the assisted object includes: determining the motion information and / or joint information of the assisted object at a first moment as the motion information and / or joint information of the assisted object at a second moment when the assisted object reaches a first position based on the first information; wherein, the first moment is any moment when the assisted object performs navigation after reaching the first position, the second moment is the difference between the first moment and the third moment, and the third moment is the time taken for the assisted object to move to the first position; and sending the motion information and / or joint information at the second moment to the assisted object.

[0034] Understandably, considering the time difference (third moment) between the movement of the assisted object and the object being assisted, the second moment is defined as the difference between the first moment and the third moment. The movement information and / or joint information of the first moment are determined as the movement information and / or joint information of the assisted object at the second moment. This ensures the consistency between the movement information of the assisted object at the first moment and the movement information of the assisted object at the second moment, effectively guaranteeing that the assisted object can navigate and avoid obstacles based on the movement information of the assisted object. In turn, this effectively improves the accuracy of navigation and obstacle avoidance using the assisted object.

[0035] In some embodiments, obtaining a first model of the assisted object includes: obtaining an original model of the assisted object, wherein the assisted object is an assisted object whose abnormal behavior is caused by external factors; based on the original model, checking the current component appearance and / or posture of the assisted object to obtain a check result; if the check result indicates that the component appearance and / or posture are abnormal, scanning the assisted object to obtain a first model; the first model is used to indicate the current component appearance and / or posture of the assisted object.

[0036] Understandably, on the one hand, upon receiving an assistance request from the assisted party, the system acquires the assisted party's original model, current component appearance, and posture. By comparison, it determines whether the component appearance and / or posture have changed compared to the appearance and posture in the original model. If the component appearance and / or posture have changed compared to the appearance and posture in the original model, the assisted party no longer uses the assisted party's original model for navigation and obstacle avoidance. Instead, it determines a first model based on the assisted party's current component appearance and / or posture, and performs navigation and obstacle avoidance based on the first model. This ensures that the assisted party can successfully pass through obstacles, avoids further damage, and improves the safety and accuracy of the assisted party's navigation and obstacle avoidance. On the other hand, navigation and obstacle avoidance based on the first model enables the assisted party to automatically reach the repair point or designated location through navigation and obstacle avoidance, improving the efficiency of the assisted party's navigation and obstacle avoidance.

[0037] In some embodiments, the assistance operation performed on the assisted object based on the assistance mode includes: when the appearance and / or posture of the characterizing component are abnormal, the assisted object is corrected or guided back to the specified area based on the assistance mode.

[0038] Understandably, when the appearance and / or posture of a component are abnormal, the assistance mode performs corresponding navigation corrections or guides the assisted object back to the specified area, thereby improving the effectiveness of the assistance.

[0039] Secondly, embodiments of this application provide a multi-object interaction assistance method applied to an assisted object, comprising: sending assistance information; the assistance information including basic information of the assisted object; the basic information including at least location information and identity recognition information; and, in the case of an assistance mode required by the assisted object determined based on the assistance information, performing corresponding assistance operations based on the assistance mode.

[0040] Understandably, based on assistance information, the system moves to the assistance location, and at that location, determines the required assistance mode for the assisted individual. This provides the feasibility of determining the assistance mode when the assisted individual needs to rely on other devices for assistance, thereby improving the feasibility and flexibility of assistance in multiple scenarios.

[0041] In some embodiments, when the required assistance mode of the assisted object is determined based on the assistance information, the corresponding assistance operation is performed based on the assistance mode, including: when the assistance mode determined based on the assistance information is a first assistance mode, receiving correction information and performing the assistance operation based on the correction information; when the assistance mode determined based on the assistance information is a second assistance mode, performing the assistance operation based on the assistance object dispatched under the second assistance mode and with the assistance of the following assistance object.

[0042] Understandably, determining the first or second assistance mode based on whether the navigation and obstacle avoidance functions of the assisted object are normal ensures the effectiveness and accuracy of the assistance provided to the assisted object.

[0043] In some embodiments, when the assistance mode determined based on the assistance information is a second assistance mode, the assistance operation is implemented with the assistance of the assistance object dispatched based on the second assistance mode, including: when the assistance mode determined based on the assistance information is the second assistance mode, if the identification information indicates that the assistance object and the assisted object are of the same model, then navigation and following are performed based on the joint information of the assistance object to implement the assistance operation; if the identification information indicates that the assistance object and the assisted object are of different models, then navigation and following are performed based on the motion information of the assistance object to implement the assistance operation.

[0044] Understandably, selecting the corresponding navigation tracking information (motion information and / or joint information) based on whether the model of the assisted object and the object being assisted are the same effectively improves the accuracy of the assistance.

[0045] Thirdly, embodiments of this application provide a multi-object interaction assistance device, the device comprising:

[0046] The receiving module is used to receive assistance information;

[0047] The determination module is used to determine the required assistance mode for the assisted object based on the assistance information.

[0048] The assistance module is used to perform corresponding assistance operations on the assisted object based on the assistance mode.

[0049] Fourthly, embodiments of this application provide a multi-object interaction assistance device, applied to the assisted object, the device comprising:

[0050] The first sending module is used to send assistance information;

[0051] The first mobile module is used to perform corresponding assistance operations based on the assistance mode required by the assisted object, which is determined based on the assistance information.

[0052] Fifthly, embodiments of this application provide a multi-object interaction assistance device, including a first memory and a first processor. The first memory is used to store executable data instructions; when the first processor executes the executable data instructions stored in the first memory, it implements the steps in a multi-object interaction assistance method applied to a multi-object interaction assistance device.

[0053] In a sixth aspect, embodiments of this application provide a assisted device for multi-object interaction, including a second memory and a second processor. The second memory is used to store executable data instructions; when the second processor executes the executable data instructions stored in the memory, it implements the steps in a multi-object interaction assistance method applied to a assisted device for multi-object interaction.

[0054] In a seventh aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0056] Figure 1 One of the flowcharts of a multi-object interaction assistance method provided in an embodiment of this application;

[0057] Figure 2 A second flowchart illustrating a multi-object interaction assistance method provided in an embodiment of this application;

[0058] Figure 3a A schematic diagram of a assisted object with a normal component appearance provided in an embodiment of this application;

[0059] Figure 3b A schematic diagram of an object with abnormal component appearance provided in an embodiment of this application;

[0060] Figure 4a A schematic diagram of a normally oriented assisted object provided in an embodiment of this application;

[0061] Figure 4b A schematic diagram of an assisted object with abnormal posture provided in an embodiment of this application;

[0062] Figure 5 A third flowchart illustrating a multi-object interaction assistance method provided in an embodiment of this application;

[0063] Figure 6 A fourth flowchart illustrating a multi-object interaction assistance method provided in an embodiment of this application;

[0064] Figure 7 This is a schematic diagram illustrating the time difference between the location of the assisting object and the assisting object when they have different models, as provided in an embodiment of this application.

[0065] Figure 8 This is a schematic diagram illustrating the position time difference between the assisting object and the assisting object when they have the same model, as provided in an embodiment of this application.

[0066] Figure 9a One of the flowcharts of an exemplary multi-object interaction assistance method provided in an embodiment of this application;

[0067] Figure 9b A second exemplary flowchart illustrating a multi-object interaction assistance method provided in this application embodiment;

[0068] Figure 10 One of the structural schematic diagrams of a multi-object interaction assistance device provided in an embodiment of this application;

[0069] Figure 11One of the structural schematic diagrams of a assisted device for multi-object interaction provided in an embodiment of this application;

[0070] Figure 12 A schematic diagram of the hardware entity of a multi-object interaction assistance device provided in an embodiment of this application;

[0071] Figure 13 This is a schematic diagram of the hardware entity of a assisted device for multi-object interaction, provided as an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0074] It should be noted that the terms "first, second, and third" used in the embodiments of this application are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0075] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0076] The following description, in conjunction with the accompanying drawings of the embodiments of this application, provides an exemplary description of the multi-object interaction assistance method, apparatus, device, and storage medium provided in the embodiments of this application.

[0077] In the embodiments of this application, Figure 1 This is a flowchart illustrating one of the assistance methods for multi-object interaction provided in the application embodiments. The method may include S101 to S103:

[0078] S101. Receive assistance information, which includes basic information about the person being assisted, including at least location information and identification information.

[0079] In related technologies, when electronic devices malfunction (are damaged or injured), they usually need to wait for repair personnel to perform on-site repairs or be sent back to the repair area for repairs. Therefore, malfunctioning electronic devices usually need to wait for rescue, which wastes time and reduces the efficiency of handling electronic device malfunctions.

[0080] In this embodiment of the application, the electronic device that is damaged or injured is regarded as the object to be assisted. Other functional electronic devices (assistance objects) are used to help the faulty electronic device (assistance object), so that the assisted object can be quickly rescued or returned to the repair point, avoiding the time that the assisted object spends waiting for rescue on the spot, thereby improving the fault handling efficiency of the assisted object.

[0081] In this application embodiment, the object being assisted is not limited and can be any electronic device that is damaged or injured, such as intelligent robots, drones, automated guided vehicles (AGVs), unmanned delivery vehicles, etc.

[0082] In this embodiment of the application, when the object being assisted is damaged or injured, the object being assisted actively sends an assistance message to request rescue. After receiving the assistance message, the assisting object provides assistance to the object being assisted so that the object being assisted can be restored to its original state or return to the repair shop for processing as soon as possible.

[0083] In the embodiments of this application, there are multiple ways for the recipient to receive assistance information, and no limitation is made on the way of receiving assistance information.

[0084] In some embodiments, the recipient receiving assistance information may be: the recipient receiving a first message sent by the cloud. That is, when the recipient needs assistance, the recipient actively requests assistance from the cloud, or the cloud actively identifies the recipient, the cloud will send a first message to the recipient. The first message carries assistance information. Therefore, after receiving the first message from the cloud, the recipient provides assistance to the recipient based on the assistance information in the first message.

[0085] In some embodiments, the assisted object may receive assistance information by: receiving a broadcast from the cloud, which instructs the assisted object to request assistance and specify the conditions for assistance; and receiving assistance information if its own capabilities meet the conditions for assistance. That is, when the cloud learns that the assisted object needs assistance, it can use broadcasting to search for suitable assistance objects. The broadcast content includes the assisted object's request for assistance and the conditions for assistance, such as the model of the assistance object and its fault status. After the cloud broadcasts, the assisting object determines whether it possesses the necessary conditions for assistance based on the broadcast's requirements. If the assisting object does possess the necessary conditions, it provides assistance to the assisted object after receiving the assistance information.

[0086] It should be noted that receiving assistance messages through the cloud can be divided into two situations. In one situation, the first message sent by the cloud carries assistance information. In this case, receiving the first message from the cloud will allow you to receive the assistance information. In the other situation, the cloud broadcasts requests for assistance and assistance conditions. In this case, if there is an object that meets the assistance conditions, the assistance information will be received.

[0087] In some embodiments, the recipient receiving assistance information may also be: upon discovering the recipient, the recipient receives assistance information sent by the recipient. That is, the recipient actively discovers that the recipient needs assistance and obtains assistance information. After receiving the assistance information sent by the recipient, the recipient provides assistance to the recipient.

[0088] In some embodiments, the recipient receiving assistance information may also involve: upon discovering the recipient, receiving basic information sent by the recipient and generating assistance information based on the basic information. In other words, after receiving the basic information of the recipient, the assisting object can use the basic information to generate the required assistance information and provide assistance to the recipient based on the assistance information.

[0089] In this embodiment of the application, the assistance information includes basic information of the assisted object, which includes at least location information and identity information.

[0090] In this application embodiment, the assistance information is not limited. In short, it is the information generated when the assisted object needs assistance. The assistance information may include the assisted object's request for help or the assisted object's basic information, so that the assisted object can find a way to get assistance through the assistance information.

[0091] In this embodiment of the application, the basic information is not limited. The basic information may include the location information and identification information of the assisted object, and may also include the fault information of the assisted object.

[0092] In some embodiments, location information may include the current location of the assisted object. The assisting object can locate the location of the assisted object based on the location information and reach that location to assist the assisted object.

[0093] In some embodiments, the identification information may include information such as the model of the assisted object. Based on the model of the assisted object, information such as the design parameters and original model of the assisted object can be obtained. In other words, the assisting object can obtain more information about the assisted object based on the identification information in order to assist the assisted object.

[0094] Understandably, there are various ways to receive assistance information. Assistance information can be obtained by receiving the first message or broadcast from the cloud, or by receiving assistance information sent by the recipient or by receiving assistance information generated from basic information. Based on this, multiple methods of disseminating assistance information are provided, improving the effectiveness and flexibility of assistance information dissemination.

[0095] S102. Based on the assistance information, determine the required assistance mode for the recipient.

[0096] In the embodiments of this application, the assistance mode is a method of assisting the assisted object. Under any assistance mode, the assisted object can achieve the assistance of the assisted object.

[0097] In the embodiments of this application, there is no limitation on the method of determining the required assistance mode for the assisted object; in short, the required assistance can be determined for the assisted object.

[0098] In some embodiments, the assistance mode can be determined as follows: the assistance information of the assisted object includes the fault information of the assisted object. The fault status of the assisted object can be classified according to the fault information. Then, different assistance modes are adopted for different fault statuses. Different fault statuses can indicate whether the assisted object can complete the assistance operation on its own or needs to be guided by other assisted objects to perform the assistance operation. Therefore, the assistance mode can be divided into the assistance object completing the assistance operation independently and the assistance object needing to perform the assistance operation under the guidance of other assisted objects.

[0099] In some embodiments, the assistance mode can also be determined as follows: confirm the way to obtain assistance information, and determine whether the assisted object can complete the assistance on its own or needs guidance from other assisted objects to perform the assistance operation based on different ways, and adopt different assistance modes accordingly.

[0100] It should be noted that there are various ways to determine the assistance mode of the assisted object based on the assistance information. However, different methods must ultimately determine whether the assisted object has the ability to complete the assistance operation independently, or whether the assisted object needs to complete the assistance operation under the guidance of other assisting objects. Based on this, the assistance mode is determined.

[0101] In some embodiments, determining the required assistance mode for the assisted object based on the assistance information may include S112 or S113:

[0102] S112. If the assistance information comes from the cloud or is sent directly by the assisted party, the required assistance mode for the assisted party is determined to be the first assistance mode. The first assistance mode indicates that the assisted party itself performs assistance operations based on the assistance information.

[0103] S113. If the assistance information is generated based on the basic information, then the required assistance mode for the assisted object is determined to be the second assistance mode. The second assistance mode indicates that the assisted object performs assistance operations under the guidance of the assisting object.

[0104] In this application embodiment, the assistance mode is divided into two types: the first assistance mode and the second assistance mode. The difference between the first assistance mode and the second assistance mode is whether the assisted object does not need guidance from other assisted objects and can perform assistance operations on its own.

[0105] In this application embodiment, when the received assistance information comes from the cloud or is sent directly by the assisted object, it is assumed that the assisted object itself can perform assistance operations based on the assistance information. That is, the assisted object does not need other assisting objects to guide it. At this time, the assistance mode required by the assisted object is determined to be the first assistance mode.

[0106] In the embodiments of this application, when the assistance information is generated from the basic information, that is, the assistance is obtained after processing such as calculation of the basic information, the assisted object cannot perform the assistance operation on its own and needs the guidance of other assisting objects. The assisting objects generate assistance information based on the basic information and guide the assisted object to perform the assistance operation.

[0107] Understandably, depending on the different sources of assistance information, the assistance mode is determined to be either the first assistance mode or the second assistance mode, thereby providing targeted assistance to the recipient. If the recipient is capable of completing the assistance operation on their own, they will do so (first assistance mode), avoiding the waste of time caused by waiting for other recipients to provide assistance. If the recipient is unable to complete the assistance operation on their own, the assisting recipient will guide them to complete the assistance operation (second assistance mode). Based on this, the timeliness and efficiency of assistance are improved.

[0108] S103. Perform corresponding assistance operations on the assisted object based on the assistance mode.

[0109] In this embodiment, the assisting object performs corresponding assistance operations on the assisted object based on an assistance mode. The assistance mode can be divided into a first assistance mode and a second assistance mode. That is, the assisted object performs corresponding assistance operations on the assisted object according to either the first assistance mode or the second assistance mode.

[0110] In some embodiments, S103 may include S106:

[0111] S106. When the assistance mode is the first assistance mode, by sending the correction information of the assisted object, the assisted object can make corresponding navigation corrections or be guided back to the designated area according to the correction information, thereby realizing the assistance operation.

[0112] In this embodiment of the application, the correction information is not limited; in short, it is information that can help the assisted object to make navigation corrections or guide it back to the designated area.

[0113] For example, when the object being assisted is damaged and its appearance is deformed, the correction information can be the current model information. The object being assisted can use this current model information (correction information) to navigate back to the designated area for repair.

[0114] Understandably, in the first assistance mode, sending correction information enables the assisted party to make corresponding navigation corrections or be guided back to the designated area based on the correction information, thereby improving the accuracy and effectiveness of assistance in the first assistance mode.

[0115] In some embodiments, S103 may include: when the assistance mode is the second assistance mode, guiding the assisted object to follow by performing navigation planning or returning to a designated area to achieve the assistance operation.

[0116] In the embodiments of this application, in the second assistance mode, the assisted object cannot perform the assistance operation on its own and needs to rely on the assistance object to perform the assistance operation. The assistance object first moves to the designated area by performing navigation planning, and the assisted object follows the assistance object to the designated area after the assistance object moves, thereby realizing the assistance operation.

[0117] Understandably, in the second assistance mode, by planning navigation or returning to a designated area, the assisted object is guided to follow, providing a feasible way to assist in scenarios where the assisted object cannot perform assistance operations on its own.

[0118] Understandably, on the one hand, after receiving assistance requests from the recipient, assistance operations are performed based on the requests, avoiding the time spent waiting for repair personnel and improving the timeliness of assistance. On the other hand, the system determines the required assistance mode and provides different assistance modes. Multiple assistance modes ensure assistance for the recipient in various scenarios, improving the flexibility and accuracy of assistance, and thus increasing the efficiency of assistance.

[0119] In some embodiments, the method further includes S201 and S202:

[0120] S201. Move to the assistance location based on the assistance information;

[0121] S202. At the assistance location, determine the required assistance mode for the person being assisted.

[0122] In some embodiments, the assisted object can obtain assistance by adjusting itself, or the assisted object needs to rely on other devices for assistance. In the case where the assisted object needs to rely on other devices for assistance, it is necessary to obtain the assistance location of the assisted object through assistance information, that is, to move to the assistance location based on the assistance information.

[0123] In this embodiment of the application, after moving to the assistance location based on the assistance information, the required assistance mode for the assisted object is determined. The method for determining the required assistance mode for the assisted object is not limited; it can be determined by scanning the assisted object.

[0124] Understandably, based on assistance information, the system moves to the assistance location, and at that location, determines the required assistance mode for the assisted individual. This provides the feasibility of determining the assistance mode when the assisted individual needs to rely on other devices for assistance, thereby improving the feasibility and flexibility of assistance in multiple scenarios.

[0125] In some embodiments, S102 may include: if the basic information of the assisted object includes fault information, determining the assistance mode of the assisted object based on the fault information; the fault information includes at least whether the navigation function of the assisted object is normal.

[0126] In this embodiment, the basic information includes fault information, which at least includes whether the navigation function of the assisted object is normal. The assisted object may have navigation function abnormalities due to damage.

[0127] In the embodiments of this application, different assistance modes are corresponding to the situation where the navigation function of the assisted object is abnormal and the situation where the navigation function of the assisted object is normal.

[0128] Understandably, different assistance modes are distinguished based on fault information, and different assistance modes are determined for the controlled objects in different states of the navigation and obstacle avoidance function, thereby improving the effectiveness of assistance.

[0129] In some embodiments, S103 may include: when the assistance mode is a first assistance mode, obtaining a first model of the assisted object and sending the first model to the assisted object to assist the assisted object in performing the assistance operation; assisting the assisted object in performing the assistance operation; the first model belongs to correction information; wherein, the first model is used to indicate the current component appearance and / or posture of the assisted object; the first assistance mode indicates that when the navigation function of the assisted object is normal, assisting the assisted object to perform navigation itself.

[0130] In the embodiments of this application, the first assistance mode indicates the assistance mode when the navigation function of the assisted object is normal, at which time the assisted object can use itself for navigation.

[0131] In the embodiments of this application, in the first assistance mode, a first model of the assisted object is obtained. The first model is used to indicate the current component appearance and / or posture of the assisted object. The first model is sent to the assisted object. The assisted object uses the first model as the model it needs to use for navigation and obstacle avoidance according to the current component appearance and / or posture indicated by the first model. The assisted object uses the first model for navigation or obstacle avoidance, thereby realizing the assistance operation for the controlled object.

[0132] In some embodiments, when the assistance mode is the first assistance mode, correction information is sent to the assisted object, causing the assisted object to make corresponding navigation corrections or be guided back to the designated area based on the correction information. Therefore, the first mode belongs to correction information.

[0133] In this application embodiment, the method of obtaining the first model of the assisted object is not limited. It can be obtained by scanning the controlled object with a camera or other device within the range of the controlled object, or by moving other assisted objects to the area where the controlled device is located and scanning them.

[0134] In some embodiments, such as Figure 2 As shown, the methods for obtaining the first model may include S301 to S304:

[0135] S301. Obtain the original model of the assisted object, which is the assisted object whose abnormal behavior is caused by external factors.

[0136] In the embodiments of this application, external factors may include falling, external force injury, etc. The external factors may cause changes in the appearance and / or posture of the assisted object.

[0137] In the embodiments of this application, the original model of the assisted object refers to the model established by the assisted object during the design process based on information such as the appearance and posture of the assisted object. In other words, the assisted object uses the original model to work normally when no abnormality occurs, for example, to navigate and avoid obstacles.

[0138] In some embodiments, the method of obtaining the original model of the assisted object is not limited, as long as the original model of the assisted object can be obtained.

[0139] For example, if the original model of the object being assisted is already stored in the cloud, the original model of the object being assisted can be obtained by downloading it from the cloud.

[0140] For example, the original model of the assisted object can be obtained by interacting with the assisted object through near-field communication.

[0141] S302. Based on the original model, check the current component appearance and / or posture of the assisted object to obtain the check results.

[0142] In the embodiments of this application, component appearance refers to the appearance of each component that makes up the assisted object, and posture refers to the posture of each joint of the assisted object.

[0143] In this application embodiment, the component appearance inspection of the assisted object mainly includes checking whether there are protrusions that exceed the range of the original model of the assisted object, and whether there are wobbly components that exceed the range of the original model.

[0144] For example, as shown in Figure 3, Figure 3a The appearance of the object being assisted under normal circumstances. Figure 3b The lower body structure of the object being assisted has detached, at which point the appearance of the components of the object being assisted appears abnormal.

[0145] In this embodiment of the application, the posture check of the assisted object mainly includes checking whether the current actual posture of the assisted object is consistent with the posture currently executed by the assisted object, that is, checking whether the assisted object has joint disconnection or joint damage problems.

[0146] For example, as shown in Figure 4, Figure 4a For the person being assisted, their perceived current posture. Figure 4b The current actual posture of the person being assisted shows that their right arm is bent, indicating an abnormal posture.

[0147] In the embodiments of this application, the method of checking the appearance and / or posture of the components of the assisted object is not limited; in short, the inspection results of the appearance and / or posture of the components of the assisted object can be obtained.

[0148] In some embodiments, the current component appearance and / or posture information of the assisted object can be obtained by describing the assisted object, and the current component appearance and / or posture of the assisted object can be compared with the component appearance and / or posture in the original model of the assisted object to determine the inspection result.

[0149] It should be noted that the purpose of checking the appearance and / or posture of the assisted object's components is to prevent secondary damage during subsequent movement. For example, if the assisted object has parts protruding from the original model, and the assisted object still navigates and avoids obstacles according to the original model, probabilistic collisions or even falls may occur. Alternatively, the clearance between the assisted object and the wall may be insufficient for the protruding part to pass through when turning a corner, leading to collisions with the protruding part when avoiding obstacles. Similarly, if the assisted object's actual posture does not match the posture it perceives it to be currently executing, probabilistic collisions or even falls may occur during navigation and obstacle avoidance. For example, the clearance between the assisted object and the wall may be insufficient for the current posture to pass through when turning a corner.

[0150] In this embodiment, the inspection results include the inspection results of the component's appearance and the inspection results of its posture. The inspection results of the component's appearance include whether the component's appearance is normal or abnormal. The inspection results of the posture include whether the posture is normal or abnormal.

[0151] S303. If the inspection results indicate that the appearance and / or posture of the component are abnormal, scan the assisted object to obtain a first model; the first model is used to indicate the current appearance and / or posture of the component of the assisted object.

[0152] S304. Based on the first model, perform assistance operations on the assisted object.

[0153] In the embodiments of this application, the first model is a model of the assisted object determined based on the current actual component appearance and / or posture of the assisted object.

[0154] In this embodiment of the application, when the inspection results indicate that the appearance and / or posture of the component are abnormal, the object to be assisted is scanned. The method of scanning the object to be assisted is not limited; it can be scanning by taking a picture with a camera or by other methods.

[0155] Understandably, on the one hand, upon receiving an assistance request from the assisted party, the system acquires the assisted party's original model, current component appearance, and posture. By comparison, it determines whether the component appearance and / or posture have changed compared to the appearance and posture in the original model. If the component appearance and / or posture have changed compared to the appearance and posture in the original model, the assisted party no longer uses the assisted party's original model for navigation and obstacle avoidance. Instead, it determines a first model based on the component appearance and / or posture, and performs navigation and obstacle avoidance based on the first model. This ensures that the assisted party can successfully pass through obstacles, avoids further damage, and improves the safety and accuracy of the assisted party's navigation and obstacle avoidance. On the other hand, navigation and obstacle avoidance based on the first model enables the assisted party to automatically reach the repair point or designated location through navigation and obstacle avoidance, improving the efficiency of the assisted party's navigation and obstacle avoidance.

[0156] Understandably, for the first assistance mode, the method of obtaining the first model and sending it to the assisted object allows the assisted object to use the first model to achieve navigation and obstacle avoidance, avoiding waiting for other assisted objects or maintenance personnel to come and assist, effectively saving assistance time, and thus improving assistance efficiency.

[0157] In some embodiments, S103 may include S104: when the assistance mode is the second assistance mode, obtain the first model of the assisted object, and provide navigation assistance guidance to the assisted object based on the assistance location and the first model, so as to assist the assisted object in performing the assistance operation; wherein, when the second assistance mode indicates that the navigation function of the assisted object is abnormal, navigation is performed through the first model to guide the assisted object to follow.

[0158] In this embodiment of the application, the second assistance mode is the assistance mode when the navigation function of the assisted object is abnormal. When the navigation function of the assisted object is abnormal, the assisted object needs to reach the area where the assisted object is located. According to the obtained first model, the assisted object first performs navigation and obstacle avoidance. After the assisted object starts navigation and obstacle avoidance, the assisted object follows and performs navigation and obstacle avoidance, thereby guiding the assisted object to follow.

[0159] In some embodiments, S104 may include S105: when in an assisted position, determining motion information and / or joint information based on a first model, and providing assisted navigation guidance to the assisted object based on the motion information and / or joint information, thereby assisting the assisted object in performing assisted operations.

[0160] In this embodiment, motion information and / or joint information are information generated by the assisted object during navigation and obstacle avoidance. Motion information may include the angle and angular velocity of the assisted object's movement; joint information may include the bending angle and movement speed of each joint of the assisted object.

[0161] In this embodiment of the application, when the assisting object is in the assisting position, it obtains the first model of the assisted object. The assisting object starts navigation and obstacle avoidance based on the first model. During the navigation and obstacle avoidance process, it determines motion information and / or joint information and feeds the motion information and / or joint information back to the assisted object. The assisted object performs navigation and obstacle avoidance based on the motion information and / or joint information. In other words, the assisted object follows the assisting object to perform navigation and obstacle avoidance.

[0162] Understandably, in the second assistance mode, when the navigation and obstacle avoidance function of the assisted object is abnormal, an assistance object is arranged to come and assist with navigation based on the assistance location. The assistance object enables the assisted object to receive assistance, avoiding waiting for maintenance personnel to come and effectively saving assistance time, thereby improving the efficiency of assistance.

[0163] In some embodiments, the assistive method for multi-object interaction is as follows: Figure 5 As shown, S105 may include S401 to S402:

[0164] S401. Merge the first model with the second model of the assisted object to determine the third model, where the second model is the original model of the assisted object;

[0165] S402. Navigate based on the third model, determine the motion information and / or joint information of the assisted object, and send the motion information and / or joint information to the assisted object to provide navigation assistance and guidance to the assisted object to achieve the assisted operation.

[0166] In this embodiment, the second model is the original model of the assisted object. The assisted object superimposes and merges the first model of the assisted object with its own second model, that is, the largest outer contour is obtained after the two models are projected and superimposed to obtain the third model. In other words, the assisted object uses the third model as a standard for navigation and obstacle avoidance to ensure that both the assisted object and the assisted object can pass smoothly.

[0167] In this application embodiment, the method of fusion is not limited. If the assisted object is a non-cloud architecture, the fusion can be completed on the assisted object itself. If the assisted object is a cloud-driven method similar to digital twin, the fusion can be completed more efficiently in the cloud.

[0168] In this embodiment of the application, the assisted object moves to avoid obstacles based on a third model. During the movement, motion information and / or joint information are generated. The motion information and / or joint information are sent to the assisted object so that the assisted object can move under the guidance of the motion information and / or joint information.

[0169] Understandably, by defining a third model whose outline is the largest outer outline resulting from the superposition of the projections of the assisted object and the object being assisted, the assisted object can navigate and avoid obstacles using the third model as a standard. This ensures that both the assisted object and the object being assisted can successfully pass through obstacles, thus guaranteeing the safety of navigation and obstacle avoidance.

[0170] In some embodiments, such as Figure 6 As shown, S402 may include S501 to S504:

[0171] S501. Based on the assisting location, determine the first location, where the first location is the starting point of the navigation set by the assisting object;

[0172] S502. Based on the first position and the position information in the basic information, determine the first information, which indicates the relative position between the first position and the current position of the assisted object.

[0173] In this embodiment of the application, the first position is the starting point set by the assisted object, and the first information includes information such as the relative distance and relative angle between the first position and the current position of the assisted object.

[0174] In some embodiments, the first position may be the same as the assisting position, or it may be moved from the assisting position to the first position.

[0175] In some embodiments, the purpose of determining the first position is to ensure that the assisted object and the assisted object have the same starting point so that navigation and obstacle avoidance can be performed synchronously in the future; determining the first information is to identify the relative position between the assisted object and the assisted object so that the assisted object can accurately reach the first position.

[0176] It should be noted that the starting point should be set so that the assisted object can reach it relatively quickly, for example, so that the assisted object can reach the starting point within 2 seconds.

[0177] S503. Navigation and obstacle avoidance are performed based on the third model and the first position, and motion information and / or joint information at different times are determined.

[0178] S504. When the assisted object reaches the first position based on the first information, the motion information and / or joint information generated by navigation are sent to the assisted object to assist and guide the assisted object in navigation and obstacle avoidance, and to assist the assisted object in performing the assisted operation.

[0179] In this embodiment of the application, the assisted object first goes to the first position, and from the first position, it navigates and avoids obstacles according to the third model. While navigating and avoiding obstacles, motion information and / or joint information at different times are determined.

[0180] In some embodiments, after the assisting object obtains the third model, it performs path planning based on the maintenance area or designated area to be reached. The assisting object can select any point as its starting point. The assisting object arrives at the starting point first and informs the assisted object of the first information.

[0181] For example, the person being assisted is told that they need to turn 10° clockwise, move forward 1 meter, and then turn 10° clockwise again to reach the starting point set by the person being assisted.

[0182] In this embodiment of the application, the assisted object performs navigation and obstacle avoidance based on a third model and a first position. That is, the third model used by the assisted object for navigation and obstacle avoidance starts at the first position.

[0183] In this embodiment of the application, the assisted object reaches the first position based on the first information. At this time, the assisted object has left the first position and is moving towards the maintenance area or the designated area (navigation and obstacle avoidance). During the movement, motion information and / or joint information are generated. The assisted object sends the motion information and / or joint information to the assisted object so that the assisted object can navigate and avoid obstacles based on the motion information and / or joint information.

[0184] In this embodiment of the application, after the assisted object leaves the starting point, it will determine in real time whether the assisted object has reached the starting point. Only after the assisted object has reached the starting point will the assisted object send motion information and / or joint information to the assisted object.

[0185] It is understandable that setting a first position and sending the first information enables the assisted object to reach the first position, ensuring that the assisted object and the assisting object have the same starting point before navigation and obstacle avoidance. Then, based on the first position, the third model is used for navigation and obstacle avoidance to determine the motion information and / or joint information at different times, so that the assisted object can accurately follow the assisting object for navigation and obstacle avoidance. Based on this, the accuracy of navigation and obstacle avoidance is improved.

[0186] In some embodiments, S503 may include S601 or S602:

[0187] S601. If the identification information indicates that the assisted object and the assisted object have different models, then when the assisted object reaches the first position based on the first information, the motion information is sent to the assisted object.

[0188] S602. If the identification information indicates that the assisted object and the assisted object have the same model, then when the assisted object reaches the first position based on the first information, the joint information is sent to the assisted object.

[0189] In the embodiments of this application, the model of the assisted object and the model of the assisted object can be the same or different. Regardless of whether the models are the same, the assisted object can assist in navigation and obstacle avoidance.

[0190] In this embodiment of the application, if the assisted object and the assisting object have different models, then when the assisted object reaches the first position based on the first information, the motion information is sent to the assisted object. That is to say, after the assisted object reaches the first position, it does not rely on its own navigation and obstacle avoidance function, but blindly walks according to the motion information. In other words, the assisted object directly controls its own motion module to move according to the received motion information.

[0191] In this embodiment of the application, if the assisted object and the assisting object have the same model, then when the assisted object reaches the first position based on the first information, the joint information is sent to the assisted object. That is to say, the assisting object can directly synchronize its own real-time information of the moving joints (joint information) to the assisted object. After the assisted object reaches the first position, it does not rely on its own navigation and obstacle avoidance function, but moves according to the joint information.

[0192] It should be noted that the amount of data in joint information is much smaller than that in motion information. Therefore, joint information is easier to calculate and process than motion information, and the transmission speed of sending only joint information is faster.

[0193] Understandably, determining different assistance methods based on whether the assisted object and the assisted object are the same improves the flexibility of the assisted object in navigation and obstacle avoidance. When the assisted object and the assisted object are the same model, only the joint information that is easier to calculate and process needs to be sent to the assisted object. The assisted object can obtain the joint information required for navigation and obstacle avoidance more quickly, thereby improving the speed of the assisted object in navigation and obstacle avoidance.

[0194] In some embodiments, S503 may further include S701 and S702:

[0195] S701. When the assisted object reaches the first position based on the first information, the motion information and / or joint information of the assisted object at the first moment are determined as the motion information and / or joint information of the assisted object at the second moment; wherein, the first moment is any moment when the assisted object starts navigation after reaching the first position, the second moment is the difference between the first moment and the third moment, and the third moment is the time taken for the assisted object to move to the first position.

[0196] S702, Send the motion information and / or joint information at the second moment to the assisted object.

[0197] In this embodiment, the first moment is any moment when the assisted object begins navigation and obstacle avoidance after reaching the first position. The second moment is any moment when the assisted object begins navigation and obstacle avoidance after reaching the first position. Because the initial positions of the assisted object and the assisted object are different, and the assisted object needs a certain amount of time to reach the first position first, there is a time difference between the first moment and the second moment (the third moment), and the third moment is the time taken for the assisted object to move to the first position.

[0198] For example, when the assisted object and the assisting object have different models, the diagram illustrating the time difference between their positions is as follows: Figure 7 As shown, assuming the time required for the assisted object to move to the first position is t, the motion information of the assisted object at any time T after reaching the first position, such as position, velocity, angle, and angular velocity, is completely consistent with the motion information of the assisted object at time Tt.

[0199] In some embodiments, considering the existence of errors, the assisted object can also rely on its own vision or other perception capabilities to determine the current position, speed, angle, angular velocity and other motion information of the assisted object in real time, so as to correct the deviation of the assisted object in real time.

[0200] For example, the assisting object uses visual inspection to correct the deviation of the assisted object. When a motion error is detected in the assisted object, the assisting object sends a correction command to the assisted object. The correction command may be, for example, rotating counterclockwise by 1° or increasing the speed by 0.05m / s.

[0201] For example, when the assisted object and the assisting object have the same model, the diagram illustrating the position-time difference between the assisting object and the assisted object is as follows: Figure 8 As shown, assuming the time required for the assisted object to move to the first position is t, then the joint information such as the position, action, and posture of the assisted object at any time T after reaching the starting point is completely consistent with the joint information of the assisted object at time Tt.

[0202] Understandably, considering the time difference (third moment) between the movement of the assisted object and the object being assisted, the second moment is defined as the difference between the first moment and the third moment. The movement information and / or joint information of the first moment are determined as the movement information and / or joint information of the assisted object at the second moment. This ensures the consistency between the movement information of the assisted object at the first moment and the movement information of the assisted object at the second moment, effectively guaranteeing that the assisted object can navigate and avoid obstacles based on the movement information of the assisted object. In turn, this effectively improves the accuracy of navigation and obstacle avoidance using the assisted object.

[0203] The application embodiment provides a multi-object interaction assistance method, applied to the assisted object, including S1101 to S1102:

[0204] S1101. Send assistance information; the assistance information includes basic information of the assisted object; the basic information includes at least location information and identification information.

[0205] In this embodiment of the application, after the object being assisted experiences abnormal behavior due to external factors, it seeks help by sending assistance information.

[0206] S1102. If the required assistance mode of the assisted object is determined based on the assistance information, perform the corresponding assistance operation based on the assistance mode.

[0207] In this embodiment, the assistance mode is determined based on whether the navigation and obstacle avoidance function is normal in the assistance information. If the navigation and obstacle avoidance function of the assisted object is normal, a first assistance mode is determined; if the navigation and obstacle avoidance function is abnormal, a second assistance mode is determined.

[0208] Understandably, on the one hand, by sending assistance requests, the recipient avoids wasting time waiting for help and improves the timeliness of their requests. On the other hand, performing corresponding assistance operations based on the assistance mode ensures the correctness of the assistance, thereby improving its accuracy.

[0209] In some embodiments, S1102 may include S1011 and S1012:

[0210] S1011. If the assistance mode determined based on the assistance information is the first assistance mode, receive correction information and perform assistance operation based on the correction information.

[0211] S1012. When the assistance mode determined based on the assistance information is the second assistance mode, the assistance object dispatched based on the second assistance mode performs the assistance operation with the assistance of the following assistance object.

[0212] In the embodiments of this application, if the navigation obstacle avoidance function of the assisted object is normal, the assisted object can achieve navigation obstacle avoidance simply by correcting the information. That is, after receiving the correction information, the assisted object uses the correction information as the model required for navigation obstacle avoidance.

[0213] In this embodiment of the application, if the navigation and obstacle avoidance function of the assisted object is abnormal, the assisted object cannot achieve navigation and obstacle avoidance simply by correcting the information. At this time, an assisting object, such as an assisting robot, is needed to jointly guide the assisted object. With the assistance of the assisting object, the assisted object can achieve navigation and obstacle avoidance.

[0214] Understandably, different assistance methods are determined based on whether the obstacle avoidance function of the assisted object is normal, thus ensuring the correctness of the assistance provided to the assisted object.

[0215] In some embodiments, S1012 may include S1021 and S1022:

[0216] S1021. When the assistance mode determined based on the assistance information is the second assistance mode, if the identification information indicates that the model of the assisted object is the same as that of the assisted object, then navigation and following are performed based on the joint information of the assisted object to realize the assistance operation.

[0217] S1022. If the identification information indicates that the model of the assisted object is different from that of the assisted object, then navigation and following are performed based on the motion information of the assisted object to realize the assisted operation.

[0218] In this embodiment, when the navigation obstacle avoidance function of the assisted object malfunctions, the assisting object needs to first perform navigation obstacle avoidance, and then guide the assisted object to follow the navigation. The assisting object sends joint information and / or motion information to the assisted object to achieve navigation following.

[0219] In this embodiment, if the model of the assisted object is the same as that of the assisted object, navigation and obstacle avoidance are performed based on the correction information and the joint information of the assisted object; if the model of the assisted object is different from that of the assisted object, navigation and following are performed based on the correction information and the motion information of the assisted object.

[0220] Understandably, selecting the corresponding navigation tracking information (motion information and / or joint information) based on whether the model of the assisted object and the object being assisted are the same effectively improves the accuracy of the assistance.

[0221] The following description uses an application example to illustrate the above-mentioned multi-object interaction assistance method. However, it is worth noting that this example is only for better illustration of this application and does not constitute an improper limitation of this application.

[0222] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0223] In this embodiment, a working robot experiences suspected damage for various reasons. If the robot is standing and can walk, and its navigation and obstacle avoidance modules are functioning normally, then this embodiment provides a multi-object interaction assistance method assuming the robot's navigation function is normal. Figure 9a As shown, including S2011 to S2013:

[0224] S2011. Assist the robot in receiving the robot's request information (assistance information);

[0225] In this embodiment of the application, the robot sends a request message to other nearby assisting robots via the cloud. The request message includes the robot's own coordinates and information about the suspected problem, enabling other assisting robots to reach the robot's location to provide assistance.

[0226] S2012, Assist the robot in inspecting the robot's appearance (component appearance) and posture to determine the first model;

[0227] In this embodiment of the application, after the robot arrives at the scene based on the received coordinate information, it obtains the fixed 3D model (original model) of the target damaged robot through the cloud and checks whether the robot's appearance has any protrusions that exceed the original model and whether there are any abnormalities in its posture, such as the arm being raised but unable to return to its original position, or the body being bent but unable to return to its original position, etc., which are inconsistent with the real-time state of the model.

[0228] In this embodiment of the application, when it is found that the appearance of the robot has a protrusion that exceeds the original model, regardless of whether the robot's posture is normal, the robot is scanned to obtain a temporary fixed 3D model of the robot (the model of the robot in its current state), and the temporary fixed 3D model is determined as the first model of the robot.

[0229] In this embodiment of the application, when it is found that the appearance of the robot does not exceed the protrusion of the original model, but the posture is abnormal, the robot is assisted in obtaining the robot's posture, and the posture in the robot's original model is modified to a first posture to form an original model with the first posture, and this model is determined as the first model.

[0230] In this embodiment of the application, if it is found that the appearance of the robot does not exceed the protrusion of the original model and the posture is normal, the original model of the robot is identified as the first model and sent to the robot. The robot determines whether to continue working normally or return to the maintenance area or the designated area based on the first model and its own internal inspection results.

[0231] S2013. Send the first model to the robot so that the robot can use the first model for navigation and obstacle avoidance.

[0232] In this embodiment, a first model is sent to the robot. The robot navigates and avoids obstacles based on the first model, moving to the maintenance area or a designated area, or continuing normal operation.

[0233] The above scenario describes a robot that appears to be damaged, is standing, can walk, and has a normal navigation and obstacle avoidance module. Only an assistance check of the robot's appearance and posture is needed to determine a primary model; the robot can then navigate and avoid obstacles based on this model. Alternatively, another scenario may exist where the robot appears to be damaged, is standing, can walk, but its navigation and obstacle avoidance module is malfunctioning, requiring assisted navigation.

[0234] In response to abnormal situations in robot navigation and obstacle avoidance functions, such as Figure 9b As shown, this application embodiment provides a multi-object interaction assistance method, including S2001 to S2007:

[0235] S2001, Assist the robot in receiving assistance information from the robot;

[0236] S2002, Assist the robot in inspecting the robot's appearance (component appearance) and posture to determine the first model;

[0237] S2003. Assist the robot in superimposing and fusing the original model and the first model to determine the 3D model (third model) for combined navigation.

[0238] In this embodiment, the assisting robot superimposes and fuses the robot's first model and posture with the assisting robot's original model and posture. The outermost contour after fusion is used as the third model for navigation. The assisting robot uses the third model fused by the two as the standard for navigation and obstacle avoidance, so as to ensure that both the assisting robot and the robot can pass smoothly.

[0239] S2004. Determine the starting point;

[0240] In this embodiment of the application, after the robot path planning is completed, any point can be selected as its own starting point, and information such as its relative position and relative angle with the robot can be sent to the robot.

[0241] S2005. When the assisting robot and the robot are different models, obstacle avoidance movement is performed according to the guide assist mode;

[0242] In this embodiment of the application, when the assisting robot and the robot are different models, after the assisting robot starts walking from the starting point, it synchronizes motion information such as speed, angle, and angular velocity to the robot in real time. After the robot reaches the starting point of the assisting robot, it does not rely on navigation and obstacle avoidance, but walks blindly according to the motion information.

[0243] S2006. When the assisting robot and the robot are the same model, obstacle avoidance movement is performed according to the trailing mode.

[0244] In this embodiment of the application, after the assisting robot starts walking from the starting point, the assisting robot can directly synchronize the real-time information of each joint to the robot. After the robot reaches the starting point of the assisting robot, it directly controls the movement of each joint according to the joint movement information without relying on navigation and obstacle avoidance.

[0245] It should be noted that because the assisting robot and the robot start at different points, the robot needs a certain amount of time to reach the assisting robot's starting point first. Therefore, there is a time difference in their travel positions. Assuming that it takes time t for the robot to move from its own starting point to the assisting robot's starting point, then the robot's position, actions, and posture at any time T after reaching the assisting robot's starting point should be completely consistent with the assisting robot's position at time Tt.

[0246] S2007. After the robot arrives at the maintenance area or designated area and stops, assist the robot in stopping the transmission of motion or joint information.

[0247] In this embodiment of the application, after the robot arrives at the maintenance area or designated area and stops, the robot stops sending motion or joint information and leaves the location on its own, thus completing the robot's task and ending the process.

[0248] In this embodiment, on the one hand, after receiving the robot's request information, the robot's original model, current component appearance, and posture are obtained. A first model is determined based on the component appearance and / or posture. Navigation and obstacle avoidance are performed based on the first model to ensure that the robot can smoothly pass through obstacles, avoid further damage, and improve the safety and accuracy of robot navigation and obstacle avoidance. On the other hand, navigation and obstacle avoidance based on the first model enables the robot to reach the repair point or designated location on its own, avoiding manual rescue and handling of the robot, saving labor costs and time, and thus improving the timeliness and efficiency of assistance.

[0249] This application provides an assistance device 1000 for multi-object interaction, such as... Figure 10 As shown, the multi-object interaction assistance device 1000 includes:

[0250] Receiver module 1001 is used to receive assistance information;

[0251] The determination module 1002 is used to determine the required assistance mode for the assisted object based on the assistance information;

[0252] The assistance module 1003 is used to perform corresponding assistance operations on the assisted object based on the assistance mode.

[0253] In some embodiments, the assistive device for multi-object interaction further includes a mobility module 1004.

[0254] The moving module 1004 is used to move to the assisting location based on the assistance information;

[0255] The determining module 1002 is also used to determine the required assistance mode for the assisted object based on assistance information when the object moves to the assistance location.

[0256] In some embodiments, the determining module 1002 is further configured to determine the required assistance mode of the assisted object as a first assistance mode if the assistance information comes from the cloud or is sent directly by the assisted object, wherein the first assistance mode indicates that the assisted object performs assistance operations based on the assistance information; or, if the assistance information comes from a source generated based on basic information, determine the required assistance mode of the assisted object as a second assistance mode, wherein the second assistance mode indicates that the assisted object performs assistance operations under the guidance of the assisting object.

[0257] In some embodiments, the multi-object interaction assistance device 1000 further includes a sending module 1005.

[0258] The sending module 1005 is used to send correction information to the assisted object when the assistance mode is the first assistance mode, so that the assisted object can make corresponding navigation corrections or be guided back to the designated area according to the correction information, thereby realizing the assistance operation.

[0259] In some embodiments, the assistance module 1003 is further configured to perform assistance operations by guiding the assisted object to follow through navigation planning or returning to a designated area when the assistance mode is the second assistance mode.

[0260] In some embodiments, the determining module 1002 is further configured to determine the assistance mode of the assisted object based on the fault information when the basic information of the assisted object includes fault information; the fault information includes at least whether the navigation function of the assisted object is normal.

[0261] In some embodiments, the sending module 1005 is further configured to obtain a first model of the assisted object and send the first model to the assisted object when the assistance mode is a first assistance mode, so as to assist the assisted object in performing the assistance operation; the first model belongs to the correction information.

[0262] In some embodiments, the assistance module 1003 is further configured to, when the assistance mode is a second assistance mode, obtain a first model of the assisted object, and provide navigation assistance guidance to the assisted object based on the assistance location and the first model, so as to assist the assisted object in performing assistance operations.

[0263] In some embodiments, the determining module 1002 is further configured to, when in an assisted position, determine motion information and / or joint information based on a first model, and provide assisted navigation guidance to the assisted object based on the motion information and / or joint information, thereby assisting the assisted object in performing assisted operations.

[0264] In some embodiments, the determining module 1002 is further configured to fuse the first model with the second model of the assisted object to determine a third model, wherein the second model is the original model of the assisted object; and to perform navigation based on the third model, determine the motion information and / or joint information of the assisted object, and send the motion information and / or joint information to the assisted object to provide navigation assistance guidance to the assisted object and assist the assisted object in performing the assisted operation.

[0265] In some embodiments, the determining module 1002 is further configured to determine a first position based on the assisting position, wherein the first position is the starting point of navigation set by the assisting object; and,

[0266] Based on the first location and the location information in the basic information, the first information is determined; and,

[0267] Navigation is performed based on the third model and the first position to determine motion information and / or joint information at different times;

[0268] The assistance module 1003 is also used to send motion information and / or joint information generated by navigation to the assisted object when the assisted object reaches the first position based on the first information, to assist and guide the assisted object in navigation and obstacle avoidance, and to assist the assisted object in performing the assisted operation.

[0269] In some embodiments, the sending module 1005 is further configured to, if the identification information indicates that the assisted object and the assisted object have different models, then, if the assisted object reaches a first position based on the first information, send motion information to the assisted object; or,

[0270] If the identification information indicates that the assisted object and the assisted object have the same model, then when the assisted object reaches the first position based on the first information, the joint information will be sent to the assisted object.

[0271] In some embodiments, the determining module 1002 is further configured to determine the motion information and / or joint information of the assisted object at a first moment as the motion information and / or joint information of the assisted object at a second moment when the assisted object reaches a first position based on the first information.

[0272] The sending module 1005 is also used to send motion information and / or joint information at the second moment to the assisted object.

[0273] In some embodiments, the determining module 1002 is further configured to obtain the original model of the assisted object, wherein the assisted object is an assisted object whose abnormal behavior is caused by external factors; and,

[0274] Based on the original model, the current component appearance and / or posture of the assisted object are checked to obtain the check results; and,

[0275] If the inspection results indicate that the component appearance and / or posture are abnormal, the assisted object is scanned to obtain a first model; the first model is used to indicate the current component appearance and / or posture of the assisted object.

[0276] In some embodiments, the assistance module 1003 is further configured to perform corresponding navigation corrections or guide the assisted object back to a specified area based on the assistance mode when the appearance and / or posture of the characterizing component are abnormal.

[0277] This application provides an assisted device 1100 for multi-object interaction, such as... Figure 11 As shown, the assisted device 1100 for multi-object interaction includes:

[0278] The first sending module 1101 is used to send assistance information;

[0279] The first mobile module 1102 is used to perform corresponding assistance operations based on the assistance mode required by the assisted object, which is determined based on the assistance information.

[0280] In some embodiments, the assisted device 1100 for multi-object interaction further includes a first assistance module 1103.

[0281] The first assistance module 1103 is configured to receive correction information and perform assistance operations based on the correction information when the assistance mode determined based on the assistance information is a first assistance mode; and,

[0282] When the assistance mode determined based on the assistance information is the second assistance mode, the assistance object dispatched based on the second assistance mode performs the assistance operation under the assistance guidance of the assistance object.

[0283] In some embodiments, the first assistance module 1103 is further configured to, when the assistance mode determined based on the assistance information is a second assistance mode, if the identification information indicates that the assisted object and the assisted object have the same model, then perform navigation following based on the joint information of the assisted object to realize the assistance operation; and,

[0284] If the identification information indicates that the model of the assisted object is different from that of the assisted object, then navigation and following are performed based on the motion information of the assisted object to achieve the assisted operation.

[0285] Understandably, on the one hand, after receiving assistance requests from the recipient, assistance operations are performed based on the requests, avoiding the time spent waiting for repair personnel and improving the timeliness of assistance. On the other hand, the system determines the required assistance mode and provides different assistance modes. Multiple assistance modes ensure assistance for the recipient in various scenarios, improving the flexibility and accuracy of assistance, and thus increasing the efficiency of assistance.

[0286] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0287] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of software products. These software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0288] This application provides an assistance device for multi-object interaction, such as... Figure 12 As shown, the hardware entity of the multi-object interaction assistance device 1200 includes: a first processor 1201 and a first memory 1203. The first memory stores a computer program that can run on the processor. When the first processor executes the program, it implements the steps in the multi-object interaction assistance method applied to the multi-object interaction assistance device.

[0289] This application provides an assisted device for multi-object interaction, such as... Figure 13 As shown, the hardware entity of the assisted device 1300 for multi-object interaction includes: a second processor 1301 and a second memory 1303. The second memory stores a computer program that can run on the processor. When the second processor executes the program, it implements the steps in the assistance method for multi-object interaction.

[0290] It should be noted that the first processor 1201 typically controls the overall operation of the multi-object interaction assistance device 1200. The first memory 1203 is configured to store instructions and applications executable by the first processor 1201, and can also cache data to be processed or already processed by various modules in the first processor 1201 and the multi-object interaction assistance device 1200 (e.g., image data, audio data, voice communication data, and video communication data). This can be implemented using flash memory or random access memory (RAM). The multi-object interaction assistance device may also include a first communication interface 1202; the first communication interface 1202 enables the multi-object interaction assistance device to communicate with other terminals or servers via a network. Data transmission between the first processor 1201, the first communication interface 1202, and the first memory 1203 can be performed via a bus 1204.

[0291] It should be noted that the second processor 1301 typically controls the overall operation of the assisted device 1300 in multi-object interaction. The second memory 1303 is configured to store instructions and applications executable by the second processor 1301, and can also cache data to be processed or already processed by various modules in the second processor 1301 and the assisted device 1300 in multi-object interaction (e.g., image data, audio data, voice communication data, and video communication data). This can be implemented using flash memory or random access memory (RAM). The assisted device in multi-object interaction may also include a first and second communication interface 1302; the first and second communication interfaces 1302 enable the assisted device in multi-object interaction to communicate with other terminals or servers via a network. Data transmission between the second processor 1301, the first and second communication interfaces 1302, and the second memory 1303 can be performed via a bus 1304.

[0292] Based on the foregoing embodiments, the multi-object interaction assistance device or the assisted device provided in this application, as well as the modules and units included in each module, can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0293] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements all steps of a collaborative method for multi-object interaction. The computer-readable storage medium can be transient or non-transient.

[0294] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the device and storage medium embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the device and storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0295] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0296] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0297] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. The above are only implementation methods of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A collaborative method for multi-object interaction, characterized in that, The method includes: Receive assistance information, which includes basic information about the object being assisted, including at least location information and identification information; Based on the assistance information, the required assistance mode for the assisted object is determined; Based on the assistance mode, perform corresponding assistance operations on the assisted object.

2. The method according to claim 1, characterized in that, The method further includes: Move to the assistance location based on the assistance information; Upon moving to the assistance location, the required assistance mode for the assisted object is determined based on the assistance information.

3. The method according to claim 1 or 2, characterized in that, The assistance information received includes one or more of the following: Receive a first message sent from the cloud, the first message carrying the assistance information; Receive broadcasts from the cloud; the broadcasts are used to instruct the recipient to request assistance and specify the conditions for assistance; if the recipient's own capabilities meet the conditions for assistance, receive the assistance information. If the object to be assisted is discovered, the assistance information sent by the object to be assisted is received; Upon discovering the object to be assisted, the system receives the basic information sent by the object to be assisted and generates the assistance information based on the basic information.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the required assistance mode for the assisted object based on the assistance information includes: If the assistance information comes from the cloud or is sent directly by the assisted party, then the required assistance mode for the assisted party is determined to be the first assistance mode, which indicates that the assisted party itself performs assistance operations based on the assistance information; or... If the assistance information is generated based on the basic information, then the required assistance mode for the assisted object is determined to be the second assistance mode, which indicates that the assisted object performs assistance operations under the guidance of the assisting object.

5. The method according to any one of claims 1 to 4, characterized in that, The step of performing corresponding assistance operations on the assisted object based on the assistance mode includes: When the assistance mode is the first assistance mode, by sending correction information to the assisted object, the assisted object can make corresponding navigation corrections or be guided back to the designated area according to the correction information, thereby realizing the assistance operation.

6. The method according to any one of claims 1 to 4, characterized in that, The step of performing corresponding assistance operations on the assisted object based on the assistance mode includes: When the assistance mode is the second assistance mode, the assisted object is guided to follow by performing navigation planning or returning to a designated area, thereby realizing the assistance operation.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the required assistance mode for the assisted object based on the assistance information includes: If the basic information of the assisted object includes fault information, the assistance mode of the assisted object is determined based on the fault information; the fault information includes at least whether the navigation function of the assisted object is normal.

8. The method according to any one of claims 1 to 7, characterized in that, The step of performing corresponding assistance operations on the assisted object based on the assistance mode includes: When the assistance mode is the first assistance mode, the first model of the assisted object is obtained and sent to the assisted object to assist the assisted object in performing the assistance operation; the first model belongs to the correction information. Wherein, the first model is used to indicate the current component appearance and / or posture of the assisted object; the first assistance mode indicates that, when the navigation function of the assisted object is normal, it assists the assisted object in navigating itself.

9. The method according to any one of claims 1 to 7, characterized in that, The step of performing corresponding assistance operations on the assisted object based on the assistance mode includes: When the assistance mode is the second assistance mode, the first model of the assisted object is obtained, and based on the assistance location and the first model, navigation assistance guidance is provided to the assisted object to assist the assisted object in performing the assistance operation; In the second assistance mode, if the navigation function of the assisted object is abnormal, navigation is performed through the first model to guide the assisted object to follow.

10. The method according to claim 9, characterized in that, The assistance guidance, based on the assisted location and the first model, for navigating the assisted object to assist it in performing assisted operations includes: When in the assisted position, motion information and / or joint information are determined based on the first model, and navigation assistance is provided to the assisted object based on the motion information and / or the joint information to assist the assisted object in performing assisted operations.

11. The method according to claim 10, characterized in that, The step of determining motion information and / or joint information based on the first model, and providing navigation guidance to the assisted object based on the motion information and / or joint information to assist the assisted object in performing assisted operations, includes: The first model is fused with the second model of the assisted object to determine the third model, where the second model is the original model of the assisted object; Navigation is performed based on the third model, determining the motion information and / or joint information of the assisted object, and sending the motion information and / or joint information to the assisted object to provide navigation assistance and guidance, assisting the assisted object in performing assisted operations; the motion information and / or joint information are used to guide the movement and following of the assisted object.

12. The method according to claim 11, characterized in that, The navigation based on the third model, determining the motion information and / or joint information of the assisted object, and sending the motion information and / or joint information to the assisted object to provide navigational assistance and guidance, assisting the assisted object in performing assisted operations, includes: Based on the assisted location, a first location is determined, which is the starting point of the navigation set by the assisted object; Based on the first location and the location information in the basic information, first information is determined, which indicates the relative position between the first location and the current location of the assisted object. Navigation is performed based on the third model and the first position to determine the motion information and / or joint information at different times; When the assisted object reaches the first position based on the first information, the motion information and / or joint information generated by navigation are sent to the assisted object to assist the assisted object in navigation and obstacle avoidance, and to assist the assisted object in performing the assisted operation.

13. The method according to claim 12, characterized in that, When the assisted object reaches the first position based on the first information, sending the motion information and / or the joint information generated by navigation to the assisted object includes: If the identification information indicates that the assisted object and the assisted object have different models, then when the assisted object reaches the first position based on the first information, the motion information is sent to the assisted object; or, If the identification information indicates that the assisted object and the assisted object have the same model, then when the assisted object reaches the first position based on the first information, the joint information is sent to the assisted object.

14. The method according to claim 12 or 13, characterized in that, When the assisted object reaches the first position based on the first information, sending the motion information and / or the joint information generated by navigation to the assisted object includes: When the assisted object reaches the first position based on the first information, the motion information and / or joint information of the assisted object at the first moment are determined as the motion information and / or joint information of the assisted object at the second moment. Wherein, the first moment is any moment when the assisted object begins navigation after arriving at the first location, the second moment is the difference between the first moment and the third moment, and the third moment is the time taken for the assisted object to move to the first location; The motion information and / or joint information at the second moment are sent to the assisted object.

15. The method according to any one of claims 8 to 14, characterized in that, The step of obtaining the first model of the assisted object includes: Obtain the original model of the assisted object, which is the assisted object whose abnormal behavior is caused by external factors; Based on the original model, the current component appearance and / or posture of the assisted object are checked to obtain the check results; If the inspection results indicate that the appearance and / or posture of the component are abnormal, the assisted object is scanned to obtain a first model; the first model is used to indicate the current appearance and / or posture of the component of the assisted object.

16. A collaborative method for multi-object interaction, characterized in that, Applied to the recipient of assistance, including: Send assistance information; the assistance information includes basic information about the assisted object; the basic information includes at least location information and identification information; If the required assistance mode for the assisted object is determined based on the assistance information, the corresponding assistance operation is performed based on the assistance mode.

17. The method according to claim 16, characterized in that, In the case of determining the required assistance mode for the assisted object based on the assistance information, the step of performing corresponding assistance operations based on the assistance mode includes: If the assistance mode determined based on the assistance information is the first assistance mode, correction information is received, and assistance operation is performed based on the correction information. If the assistance mode determined based on the assistance information is the second assistance mode, the assistance operation is performed with the assistance of the assistance object dispatched under the second assistance mode.

18. The method according to claim 17, characterized in that, When the assistance mode determined based on the assistance information is a second assistance mode, the assistance operation is performed by the assistance object dispatched under the second assistance mode, with the assistance of the assistance object, including: If the assistance mode determined based on the assistance information is the second assistance mode, and if the identity recognition information indicates that the assistance object and the assisted object have the same model, then navigation and following are performed based on the joint information of the assistance object to realize the assistance operation; If the identification information indicates that the model of the assisted object is different from that of the assisted object, then navigation and following are performed based on the motion information of the assisted object to achieve the assistance operation.

19. A multi-object interaction assistance device, characterized in that, The device includes: The receiving module is used to receive assistance information; The determining module is used to determine the required assistance mode for the assisted object based on the assistance information; The assistance module is used to perform corresponding assistance operations on the assisted object based on the assistance mode.

20. A assisted device for multi-object interaction, characterized in that, The device, applied to the object being assisted, includes: The first sending module is used to send assistance information; The first mobile module is used to perform corresponding assistance operations based on the assistance mode required by the assisted object, as determined based on the assistance information.

21. A multi-object interaction assistance device, characterized in that, include: The first memory is used to store executable data instructions; A first processor, when executing executable data instructions stored in the first memory, implements the steps of the multi-object interaction assistance method according to any one of claims 1 to 15.

22. A assisted device for multi-object interaction, characterized in that, include: The second memory is used to store executable data instructions; The second processor, when executing executable data instructions stored in the second memory, implements the steps of the multi-object interaction assistance method according to any one of claims 16 to 18.

23. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the multi-object interaction assistance method according to any one of claims 1 to 15 or 16 to 18.

Images