A method, system, storage medium and electronic device for optimizing the motion trajectory of a robotic arm

Abstract

The present invention provides a method, a system, a storage medium and an electronic device for optimizing the motion trajectory of a robotic arm, relating to the technical field of robotic arm control. The method includes: identifying a first latest key area where interaction occurs between the robotic arm and the user; respectively determining a first local trajectory before entering the first latest key area and a second local trajectory within the first latest key area from the first current future motion trajectory of the robotic arm; and performing an optimization and replacement process on the second local trajectory based on a first interaction situation when the robotic arm moves along the first local trajectory and interacts with the user. This enables the robotic arm to adapt to this sudden situation when moving along the optimized and replaced second local trajectory in the future, achieving flexible and intelligent responses when the user's requirements suddenly change, avoiding interfering with the user's normal activities, ensuring a smooth interaction process, and greatly improving the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arm control, and particularly to a method, system, storage medium, and electronic device for optimizing the motion trajectory of a robotic arm. Background Art

[0002] In the context of the rapid development of robotic arm technology, its application scenarios have covered multiple fields such as industrial production, home service, and medical health. In these applications, the robotic arm needs to interact with users in real time, such as carrying items, delivering items, and fixing items.

[0003] However, the motion trajectory planning of the robotic arm often needs to be dynamically adjusted according to the actual environment and user requirements. Existing technologies generally rely on preset motion paths and fixed trajectory planning, lacking a flexible response mechanism to the sudden changes in user real-time needs. Although in some scenarios, the robotic arm already has a certain ability of automatic trajectory planning, currently most robotic arms still have difficulty in making sufficiently flexible and intelligent responses when interacting with users, especially when the user's needs change suddenly (such as when the user suddenly turns on the TV or engages in other activities), which may cause the motion of the robotic arm to interfere with the normal activities of the user (such as blocking the TV screen, affecting the line of sight, or preventing the user from performing other operations), resulting in an unsmooth interaction process and thus greatly affecting the user experience.

[0004] Therefore, there is an urgent need for a more flexible trajectory optimization method to optimize the motion trajectory of the robotic arm in an intelligent way, enabling it to automatically adapt to sudden changes in user needs and avoid unnecessary conflicts during the interaction process. Summary of the Invention

[0005] One of the purposes of the present invention is to provide a method for optimizing the motion trajectory of a robotic arm. When a sudden situation occurs during the interaction between the robotic arm and the user, the first latest key area that needs to be locally trajectory-optimized is identified. Based on the first interaction situation between the robotic arm and the user when moving along the first local trajectory before entering the first latest key area, the second local trajectory in the first latest key area is optimized and replaced, so that when the robotic arm moves along the optimized and replaced second local trajectory in the future, it can adapt to this sudden situation, realizing flexible and intelligent responses when the user's needs change suddenly, avoiding interfering with the normal activities of the user, ensuring a smooth interaction process, and greatly improving the user experience.

[0006] A method for optimizing the motion trajectory of a robotic arm provided by an embodiment of the present invention includes:

[0007] Identifying the first latest key area where the robotic arm interacts with the user;

[0008] Determine a first local trajectory before entering the first latest key area and a second local trajectory within the first latest key area from the first current future motion trajectory of the robotic arm respectively;

[0009] Optimize and replace the second local trajectory based on a first interaction situation during the movement of the robotic arm along the first local trajectory with the user.

[0010] Optionally, the recognition of the first latest key area where the robotic arm interacts with the user includes:

[0011] Obtain a sequence of changes in the user's field of view after the robotic arm's most recent interaction with the user;

[0012] Draw an appearance curve based on the appearance degrees of the robotic arm in each first field of view in the sequence of changes in the field of view;

[0013] When more than N target peaks higher than the first peak appear continuously immediately after the first peak in the appearance curve, intercept the local field of view sequence between the first fields of view corresponding to the head and tail peaks of the target peaks from the sequence of changes in the field of view; where N is a positive integer greater than or equal to 2;

[0014] Obtain a set of target objects that appear in each second field of view in the local field of view sequence;

[0015] Match the set of target objects with multiple sets of standard objects respectively;

[0016] When the match is successful, obtain the key object indication rule of the successfully matched set of standard objects;

[0017] Based on the key object indication rule, determine the key object and the corresponding optimization category from the set of target objects;

[0018] When the optimization category is active optimization, use the area of the first field of view where the key object appears last in the first field of view corresponding to the trough in the appearance curve as the first latest key area;

[0019] When the optimization category is passive optimization, use the area of the overlapping field of view between the first fields of view corresponding to the target peaks as the first latest key area.

[0020] Optionally, the optimizing and replacing the second local trajectory based on the first interaction situation during the movement of the robotic arm along the first local trajectory with the user includes:

[0021] Perform feature description processing on the first interaction situation to obtain a first feature description vector;

[0022] When the optimization category is active optimization, determine the active behavior corresponding to the first feature description vector from the active behavior library;

[0023] Perform an optimization replacement process on the second local trajectory so that the robotic arm can exhibit active behavior in the first latest key area when moving along the optimized and replaced second local trajectory;

[0024] When the optimization category is passive optimization, determine the passive behavior corresponding to the first feature description vector from the passive behavior library;

[0025] Perform an optimization replacement process on the second local trajectory so that the robotic arm can exhibit passive behavior in any area outside the first latest key area when moving along the optimized and replaced second local trajectory.

[0026] Optionally, the step of plotting the appearance degree curve based on the appearance degrees that successively appear in each first field of view in the field of view change sequence of the robotic arm includes:

[0027] Plot the appearance degree curve based on the curve plotting template and the appearance degrees that successively appear in each first field of view in the field of view change sequence of the robotic arm;

[0028] Wherein, the horizontal axis of the template curve coordinate system in the curve plotting template is the order of the field of view change, and the vertical axis of the template curve coordinate system is the appearance degree size.

[0029] Optionally, after performing the optimization replacement process on the second local trajectory based on the first interaction situation between the robotic arm and the user when the robotic arm moves along the first local trajectory, it further includes:

[0030] After the robotic arm finishes moving along the optimized and replaced second local trajectory, if another user stays in the first latest key area for more than the first preset duration, identify the second latest key area where the robotic arm interacts with the user and the other user;

[0031] Respectively determine the third local trajectory before entering the second latest key area and the fourth local trajectory entering the second latest key area from the second current future motion trajectory of the robotic arm;

[0032] Perform an optimization replacement process on the fourth local trajectory based on the second interaction situation between the user and the other user when the robotic arm moves along the third local trajectory.

[0033] Optionally, the step of identifying the second latest key area where the robotic arm interacts with the user and the other user includes:

[0034] When the key object has not appeared in the third field of view of the user and the other user for more than the second preset duration, take the area of the overlapping field of view that overlaps and persists for more than the third preset duration in the third field of view of the user and the other user as the second latest key area.

[0035] Optionally, optimizing and replacing the fourth partial trajectory based on the second interaction situation where users interact with each other when the robotic arm moves along the third partial trajectory includes:

[0036] Performing feature description processing on the second interaction situation to obtain a second feature description vector;

[0037] Determining the interaction trigger degree corresponding to the second feature description vector from the interaction trigger degree library;

[0038] When the interaction trigger degree exceeds the trigger degree threshold, planning the docking behavior of the robotic arm to continuously dock with the key object;

[0039] Performing optimization and replacement processing on the fourth partial trajectory so that the robotic arm can exhibit docking behavior in the second latest key area when moving along the optimized and replaced fourth partial trajectory.

[0040] A robotic arm motion trajectory optimization system provided by an embodiment of the present invention includes:

[0041] A latest key area recognition module, configured to recognize a first latest key area where the robotic arm interacts with the user;

[0042] A partial trajectory determination module, configured to respectively determine a first partial trajectory before entering the first latest key area and a second partial trajectory entering the first latest key area from the first current future motion trajectory of the robotic arm;

[0043] A trajectory optimization and replacement processing module, configured to perform optimization and replacement processing on the second partial trajectory based on the first interaction situation where the robotic arm interacts with the user when moving along the first partial trajectory.

[0044] A computer-readable storage medium provided by an embodiment of the present invention, on which a computer program is stored, and when a processor executes the computer program, the method described in any one of the above is implemented.

[0045] An electronic device provided by an embodiment of the present invention, the electronic device includes a memory and a processor, a computer program is stored in the memory, and when the processor executes the computer program, the method described in any one of the above is implemented.

[0046] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or be understood by implementing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the written description and the drawings.

[0047] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Brief Description of the Drawings

[0048] The drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0049] Figure 1 is a schematic diagram of a method for optimizing the motion trajectory of a robotic arm in an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a system for optimizing the motion trajectory of a robotic arm in an embodiment of the present invention. Detailed Embodiments

[0051] The preferred embodiments of the present invention are described below with reference to the drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0052] An embodiment of the present invention provides a method for optimizing the motion trajectory of a robotic arm, as Figure 1 shown, including:

[0053] S1. Identify the first latest key area where the robotic arm interacts with the user;

[0054] S2. Determine, from the first current future motion trajectory of the robotic arm, a first local trajectory before entering the first latest key area and a second local trajectory within the first latest key area respectively;

[0055] S3. Optimize and replace the second local trajectory based on the first interaction situation between the robotic arm and the user when the robotic arm moves along the first local trajectory.

[0056] The working principle and beneficial effects of the above technical solution are as follows:

[0057] The interaction between the robotic arm and the user refers to the process in which the robotic arm provides services to the user, interacts with the user, etc. For example, the robotic arm transports the items required by the user and delivers them to the user, or the robotic arm holds an item for the user temporarily. When the robotic arm is working, it will adaptively plan the future motion trajectory according to its own working conditions in real time. The first current future motion trajectory is the trajectory of the robotic arm's future movement after the moment when the first latest key area is identified.

[0058] When an unexpected situation occurs during the interaction between the robotic arm and the user, a first latest critical area will be generated. It is necessary to optimize and replace the second local trajectory that enters the first latest critical area for the unexpected situation. When the robotic arm moves along the first local trajectory, the unexpected situation will further develop. Therefore, the first interaction situation can be used to optimize and replace the second local trajectory. Specifically, for example, when an unexpected situation occurs where the user temporarily turns on the TV to watch a movie, to avoid blocking the user's line of sight, the first latest critical area is the field of view area when the user is watching TV, and the first interaction situation is that there is no interaction between the robotic arm and the user. Further explaining that the user wants to watch TV comfortably at this time, then the second local trajectory is optimized and replaced, and the goal of the optimization and replacement is to make the robotic arm not move within the first latest critical area in the future.

[0059] In this application, when an unexpected situation occurs during the interaction between the robotic arm and the user, the first latest critical area that needs to be optimized for the local trajectory is identified. Based on the first interaction situation between the robotic arm and the user when moving along the first local trajectory before entering the first latest critical area, the second local trajectory that enters the first latest critical area is optimized and replaced, so that the robotic arm can adapt to this unexpected situation when moving along the optimized and replaced second local trajectory in the future, realizing flexible and intelligent responses when the user's needs change suddenly, avoiding interfering with the user's normal activities, ensuring the smoothness of the interaction process, and greatly improving the user experience.

[0060] In the application scenario of the above embodiment of this application, when an unexpected situation occurs during the interaction between the robotic arm and the user, the accuracy of identifying the first latest critical area directly determines the effect of optimizing and replacing the second local trajectory that enters it. If the identified first latest critical area is too large, it will cause excessive resource consumption for the system to optimize the trajectory. If the identified first latest critical area is too small, it will cause the trajectory optimization to not enable the interaction between the robotic arm and the user to flexibly adapt to the unexpected situation that appears.

[0061] Therefore, to solve the above problems, in one embodiment, the identification of the first latest critical area for the interaction between the robotic arm and the user includes:

[0062] Obtain the sequence of the user's visual field changes after the robotic arm's most recent interaction with the user;

[0063] Based on the appearance degrees that appear successively in each first visual field in the visual field change sequence of the robotic arm, draw an appearance degree curve;

[0064] When more than N target peaks higher than the first peak appear continuously immediately after the first peak in the appearance degree curve, intercept the local visual field sequence between the first visual fields corresponding to the first and last peaks of the target peaks from the visual field change sequence; where N is a positive integer greater than or equal to 2;

[0065] Obtain a set of target objects that appear in each second visual field in the local visual field sequence;

[0066] Match the set of target objects with multiple sets of standard objects respectively;

[0067] When the match is successful, obtain the key object indication rule of the set of standard objects that match successfully;

[0068] Based on the key object indication rule, determine the key objects and corresponding optimization categories from the set of target objects;

[0069] When the optimization category is active optimization, take the area of the first visual field where the last key object appears in the visual field corresponding to the trough in the appearance degree curve as the first latest key area;

[0070] When the optimization category is passive optimization, take the area of the overlapping visual fields between the first visual fields corresponding to the target peak as the first latest key area.

[0071] The visual field change sequence includes the first visual fields that the user has successively changed after the manipulator's most recent interaction with the user; this first visual field refers to the user's eye visual field in the real environment, which can be determined by the manipulator detecting the user's eye position, orientation, etc. and combining with the standard eye visual field range. The appearance degree of the manipulator in the first visual field refers to the ratio of the appearance area of the manipulator in the first visual field to the total area of the first visual field. The drawn appearance degree curve reflects the change trend of the appearance degrees of the manipulator that appear successively in the user's first visual field.

[0072] When an unexpected situation occurs during the interaction between the manipulator and the user, the user hopes that the manipulator can be aware of the unexpected situation and make a response, so the user will first take a preliminary look at the manipulator, and then look at the manipulator in depth multiple times. During this period, the user will also look at the objects that can help the manipulator be aware of the unexpected situation. For example: the user suddenly turns on the TV and wants to watch TV, but the manipulator beside it may affect its line of sight in future work, so the user will first take a preliminary look at the manipulator, hoping it to avoid, then look at the TV, and then look at the manipulator in depth multiple times, hoping it to confirm and make an avoidance response, and will also look at the TV. Another example: when the user is watching TV and wants the manipulator to accompany, the user will also first take a preliminary look at the manipulator, hoping it to approach, then look at the TV, and then look at the manipulator in depth multiple times, hoping it to confirm and make an approach response, and will also look at the TV. Thus, more than N target peaks higher than the first peak appear continuously immediately after the first peak in the appearance degree curve. At this time, the local visual field sequence corresponding to the first and last peaks of the target peaks can reflect whether the user hopes the manipulator to perform active optimization or passive optimization, for example: approach or avoid.

[0073] The target object set contains objects that appear in each second visual field, such as: TV sets, TV cabinets, snack carts, coffee tables, etc. A standard object set and corresponding key object indication rules are preset. The key object indication rules indicate how to determine the objects that need to be used as the basis for processing to enable the robotic arm to adapt to emergencies from the target object set, and also indicate the corresponding optimization categories. For example, if the standard object set contains a TV set, a TV cabinet, and a snack cart, it means that the user not only wants to watch TV but may also want to eat snacks midway, and the robotic arm can carry the snacks to the user. Then the key object indication rule is to use the TV as the key object, and the optimization category is active optimization. Another example is that if the standard object set contains a TV set and a TV cabinet, it means that the user only wants to watch TV. Then the key object indication rule is to use the TV as the key object, and the optimization category is passive optimization.

[0074] When the optimization category is active optimization, the user hopes that the robotic arm is as close as possible without affecting their view of the key object. Then, in the first visual field, the appearance degree of the robotic arm needs to be very low and the key object can be viewed. The user will also subjectively adjust the first visual field during the process of looking back and forth between the key object and the robotic arm to let the robotic arm know the position area where they hope the robotic arm is located. After continuous adjustment, the adjustment is finally successful. Therefore, the area of the first visual field corresponding to the trough in the appearance degree curve where the last key object appears in the first visual field is used as the first latest key area. When the optimization category is passive optimization, the user hopes that the robotic arm stays away as much as possible so as not to affect their view of the key object. The user will also subjectively adjust the first visual field during the process of looking back and forth between the key object and the robotic arm to let the robotic arm know the position area where they do not want the robotic arm to be located. Therefore, the area of the overlapping visual field between the target peak and the first visual field is used as the first latest key area.

[0075] The embodiment of the present invention accurately identifies the first latest key area, ensuring the effect brought by the optimization replacement process of the second local trajectory entering it, avoiding excessive consumption of system resources caused by over-identification, and also avoiding that the trajectory optimization cannot enable the flexible adaptation of the interaction between the robotic arm and the user to the emergencies due to under-identification, greatly improving the applicability of the system.

[0076] In addition, a visual field change sequence is introduced, an appearance curve is plotted, a trigger condition is set that after the first peak in the appearance curve, there are continuously more than N target peaks higher than the first peak, the timing of intercepting the head and tail peaks of the target peaks from the visual field change sequence to obtain the local visual field sequence between the corresponding first visual fields is determined, the target object set, which is the object to be used as the basis for processing to make the robotic arm adapt to sudden situations, is accurately determined, and using the standard object set and the key object indication rules, the key objects and the corresponding optimization categories are quickly determined. According to the different optimization categories, the first latest key area is respectively determined using the appearance curve in a progressive manner, greatly improving the recognition efficiency and recognition accuracy of the first latest key area. At the same time, it is also more intelligent.

[0077] In one embodiment, the optimizing and replacing process of the second local trajectory based on the first interaction situation during the movement of the robotic arm along the first local trajectory includes:

[0078] Performing feature description processing on the first interaction situation to obtain a first feature description vector;

[0079] When the optimization category is active optimization, determining the active behavior corresponding to the first feature description vector from the active behavior library;

[0080] Performing an optimizing and replacing process on the second local trajectory so that when the robotic arm moves along the optimized and replaced second local trajectory, it can exhibit an active behavior in the first latest key area;

[0081] When the optimization category is passive optimization, determining the passive behavior corresponding to the first feature description vector from the passive behavior library;

[0082] Performing an optimizing and replacing process on the second local trajectory so that when the robotic arm moves along the optimized and replaced second local trajectory, it can exhibit a passive behavior in any area outside the first latest key area.

[0083] The working principle and beneficial effects of the above technical solution are as follows:

[0084] When the optimization category is active optimization, for example: if the first interaction situation is that the user commands the robotic arm to tidy up the snacks on the snack cart, it means that the user wants the robotic arm to accompany them while watching TV and be ready to carry and deliver snacks at any time. Then the active behavior corresponding to the first feature description vector after feature description processing in the active behavior library is to carry and deliver snacks. That is, performing an optimizing and replacing process on the second local trajectory so that when the robotic arm moves along the optimized and replaced second local trajectory, it can exhibit an active behavior in the first latest key area. The optimizing and replacing process means first optimizing the trajectory and then replacing the original trajectory with the optimized trajectory. When optimizing, the trajectory planning ability of the robotic hand itself can be utilized.

[0085] When the optimization category is passive optimization, for example: the first interaction situation is that the user has been watching TV and not interacting with the robotic arm, which indicates that the user wants to watch TV at ease. Then the passive behavior corresponding to the first feature description vector after feature description processing in the passive behavior library is to avoid entering the first latest critical area. That is, the second local trajectory is optimized and replaced so that when the robotic arm moves along the optimized and replaced second local trajectory, it can exhibit passive behavior in any area outside the first latest critical area. At this time, generally, the second local trajectory can be directly replaced with the return trajectory of the first local trajectory.

[0086] The embodiment of the present invention performs feature description processing on the first interaction situation to obtain the first feature description vector, and respectively determines the active behavior or passive behavior corresponding to the first feature description vector according to different optimization categories, and accordingly quickly performs targeted optimization and replacement processing on the second local trajectory, greatly improving the efficiency of the optimization and replacement processing of the second local trajectory.

[0087] In one embodiment, drawing the appearance degree curve based on the appearance degrees that appear successively in each first visual field in the visual field change sequence of the robotic arm includes:

[0088] Based on the curve drawing template, draw the appearance degree curve according to the appearance degrees that appear successively in each first visual field in the visual field change sequence of the robotic arm;

[0089] Wherein, the horizontal axis of the template curve coordinate system in the curve drawing template is the order of visual field change, and the vertical axis is the appearance degree size.

[0090] The curve drawing template includes a template curve coordinate system. The template curve coordinate system is a two-dimensional coordinate system, whose horizontal axis is the order of visual field change and the vertical axis is the appearance degree size. When drawing the appearance degree curve, first find multiple coordinate points in the template curve coordinate system according to the change order of each first visual field and the corresponding appearance degree size, and connect the coordinate points in sequence to obtain the appearance degree curve. The curve drawing template is set in advance by technicians according to actual needs.

[0091] In the actual application of this application, there will also be a special problem scenario: other users will newly enter the first latest critical area. At this time, if their stay time is relatively long, it may also affect the user. However, as the user interacts with other users, the sudden situation generated by the original interaction between the robotic arm and the user may also mutate, and the robotic arm needs to make further adaptive actions to avoid interfering with both the user and other users.

[0092] Therefore, to overcome the above special problem scenarios, in one embodiment, after optimizing and replacing the second local trajectory based on the first interaction situation between the robotic arm and the user when the robotic arm moves along the first local trajectory, it further includes:

[0093] After the robotic arm finishes moving along the second local trajectory after the optimization and replacement process, if another user stays in the first latest key area for more than the first preset duration, identify the second latest key area where the robotic arm interacts with the user and the other user;

[0094] Respectively determine the third local trajectory before entering the second latest key area and the fourth local trajectory within the second latest key area from the second current future motion trajectory of the robotic arm;

[0095] Based on the second interaction situation between the user and the other user when the robotic arm moves along the third local trajectory, optimize and replace the fourth local trajectory.

[0096] The first preset duration is a duration threshold representing a relatively long stay, for example: taking a value of 20 seconds. The second latest key area is the interval where the robotic arm needs to make the above further adaptive actions and requires local trajectory optimization. Correspondingly, respectively determine the third local trajectory before entering the second latest key area and the fourth local trajectory within the second latest key area from the second current future motion trajectory of the robotic arm. To avoid jointly disturbing the user and the other user after the robotic arm makes the above further adaptive actions, it is necessary to optimize and replace the fourth local trajectory based on the second interaction situation between the user and the other user when the robotic arm moves along the third local trajectory.

[0097] In the embodiment of the present invention, when another user stays in the first latest key area for a long time, identify the second latest key area where the robotic arm interacts with the user and the other user, and based on the second interaction situation, optimize and replace the fourth local trajectory, so that when the robotic arm encounters the above special problem scenarios, it can make further adaptive actions and avoid jointly disturbing the user and the other user to overcome the above special problem scenarios.

[0098] In one embodiment, the identification of the second latest key area where the robotic arm interacts with the user and the other user includes:

[0099] When the key object has not appeared in the third field of view of the user and the other user for more than the second preset duration, take the area of the overlapping field of view that overlaps and remains for more than the third preset duration in the third field of view of the user and the other user as the second latest key area.

[0100] The second preset duration is a duration threshold representing that something has not appeared for a long time. For example, it takes a value of 100 seconds. The third preset duration is a duration threshold representing that the overlapping retention duration is relatively long. For example, it takes a value of 15 seconds. When the key object has not appeared in the third field of view between the user and other users for more than the second preset duration, the area of the overlapping field of view that overlaps and persists for more than the third preset duration in the third field of view between the user and other users is taken as the second latest key area. For example, since other users entered the first latest key area, if the user and other users have not watched TV for a long time, it indicates that they want to jointly create a new event, and this event will occur in the area of the overlapping field of view where they overlap and persist for a long time, so this area is taken as the second latest key area.

[0101] The embodiment of the present invention accurately identifies the second latest key area, and also ensures the effect brought by the optimized replacement processing of the fourth local trajectory entering it.

[0102] In one embodiment, the optimizing and replacing process of the fourth local trajectory based on the second interaction situation during the movement of the robotic arm along the third local trajectory between the user and other users includes:

[0103] Performing feature description processing on the second interaction situation to obtain a second feature description vector;

[0104] Determining the interaction trigger degree corresponding to the second feature description vector from the interaction trigger degree library;

[0105] When the interaction trigger degree exceeds the trigger degree threshold, planning the docking behavior of the robotic arm to continuously dock with the key object;

[0106] Performing an optimizing and replacing process on the fourth local trajectory so that when the robotic arm moves along the optimized and replaced fourth local trajectory, it can exhibit a docking behavior in the second latest key area.

[0107] The second interaction situation reflects whether the user and other users hope that the robotic arm participates in the interaction process between the two. Then, the second interaction situation feature description is processed into a second feature description vector, and the corresponding interaction trigger degree is determined from the interaction trigger degree library. The interaction trigger degree reflects the degree to which the user and other users hope that the robotic arm participates in the interaction process between the two. For example, the second interaction situation is that the user and other users jointly use a tablet computer, indicating that other users do not want the user to continue watching TV and hope that the user accompanies them to view the tablet computer. Then, they do not want the robotic arm to participate in the interaction process between the two, and the interaction trigger degree is 0. Another example is that the second interaction situation is that the user or other users send a request to the robotic arm to review and summarize the played content of the TV, so that other users can quickly understand the played content of the TV to continue watching the TV with the user. Then, the interaction trigger degree is 10. The trigger degree threshold is a threshold representing a relatively large degree to which the user and other users hope that the robotic arm participates in the interaction process between the two. For example, the value is 5. When the interaction trigger degree exceeds the trigger degree threshold, plan the docking behavior of the robotic arm to continuously dock with the key object. The docking behavior refers to the behavior of docking with the key object. For example, communicate with the TV, analyze its played content for review and summary, and perform an optimization and replacement process on the fourth local trajectory, so that when the robotic arm moves along the optimized and replaced fourth local trajectory, it can exhibit the docking behavior in the second latest key area.

[0108] In the embodiment of the present invention, the interaction trigger degree is introduced. When the interaction trigger degree exceeds the trigger degree threshold, plan the docking behavior of the robotic arm to continuously dock with the key object, and perform an optimization and replacement process on the fourth local trajectory, so that when the robotic arm moves along the optimized and replaced fourth local trajectory, it can exhibit the docking behavior in the second latest key area, greatly improving the accuracy and efficiency of the optimization and replacement process of the fourth local trajectory.

[0109] The embodiment of the present invention provides a robotic arm motion trajectory optimization system, as Figure 2 shown, including:

[0110] The latest key area recognition module 1 is used to recognize the first latest key area for the interaction between the robotic arm and the user;

[0111] The local trajectory determination module 2 is used to respectively determine the first local trajectory before entering the first latest key area and the second local trajectory in the first latest key area from the first current future motion trajectory of the robotic arm;

[0112] The trajectory optimization and replacement processing module 3 is used to perform an optimization and replacement process on the second local trajectory based on the first interaction situation between the robotic arm and the user when the robotic arm moves along the first local trajectory.

[0113] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the method described in any one of the above.

[0114] An embodiment of the present invention provides an electronic device, which includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the method described in any one of the above.

[0115] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A method for optimizing the motion trajectory of a robotic arm, characterized in that: include: Identifying a first and latest critical area for interaction between the robotic arm and the user; Determine respectively from a first current future motion trajectory of the robot arm a first local trajectory before entering a first latest critical area and a second local trajectory entering the first latest critical area; Based on a first interaction situation between the robot arm and the user when the robot arm moves along the first partial trajectory, performing an optimization replacement process on the second partial trajectory; The identifying the first latest key area for interaction between the robot arm and the user comprises: Obtain the user's field of view change sequence after the robot arm last interacted with the user; Based on the appearance of the robot arm in each first field of view in the field of view change sequence, an appearance curve is drawn; When more than N target peaks higher than the first peak appear successively after the first peak in the appearance curve, the first and last peaks of the target peak are intercepted from the field of view change sequence, and each of them corresponds to a local field of view sequence between the first fields of view; wherein N is a positive integer greater than or equal to 2; Acquire a set of target objects that appear in every second field of view in the local field of view sequence; Matching the target object set with multiple standard object sets respectively; When the match is met, the key object indication rule of the matching standard object set is obtained; Based on the key object indication rule, determine the key objects and corresponding optimization categories from the target object set; When the optimization category is active optimization, the area of ​​the first visual field corresponding to the last occurrence of the key object in the first visual field in the trough of the appearance curve is taken as the first latest key area; When the optimization category is passive optimization, the area of ​​the overlapping fields of view between the target peaks corresponding to the first fields of view is taken as the first latest key area.

2. The method for optimizing the motion trajectory of a robotic arm according to claim 1, characterized in that: The optimizing and replacing processing of the second partial trajectory based on the first interaction situation between the robot arm and the user when the robot arm moves along the first partial trajectory includes: Performing feature description processing on the first interaction situation to obtain a first feature description vector; When the optimization category is active optimization, determining the active behavior corresponding to the first feature description vector from the active behavior library; Performing an optimization replacement process on the second local trajectory, so that the robot arm can exhibit active behavior in the first latest key area when moving along the second local trajectory after the optimization replacement process; When the optimization category is passive optimization, determining the passive behavior corresponding to the first feature description vector from the passive behavior library; The second local trajectory is optimized and replaced, so that the robot arm can show passive behavior in any area outside the first latest key area when moving along the second local trajectory after the optimization and replacement processing.

3. The method for optimizing the motion trajectory of a robotic arm according to claim 1, characterized in that: The step of drawing an appearance curve based on the appearance of the robot arm in each first field of view in the field of view change sequence comprises: Based on the curve drawing template, an appearance curve is drawn according to the appearance of the robot arm in each first field of view in the field of view change sequence; The horizontal axis of the template curve coordinate system in the curve drawing template is the order of field of view change, and the vertical axis of the template curve coordinate system is the magnitude of the appearance.

4. The method for optimizing the motion trajectory of a robotic arm according to claim 1, characterized in that: After the second partial trajectory is optimized and replaced based on the first interaction situation between the robot arm and the user when the robot arm moves along the first partial trajectory, the method further includes: After the robot arm finishes moving along the second local trajectory after the optimization replacement process, if other users stay in the first latest key area for more than a first preset time, identifying the second latest key area for interaction between the robot arm and the user and other users; Determine respectively from the second current future motion trajectory of the robot arm a third local trajectory before entering the second latest key area and a fourth local trajectory entering the second latest key area; Based on a second interaction situation of the user interacting with other users when the robot arm moves along the third partial trajectory, the fourth partial trajectory is optimized and replaced.

5. The method for optimizing the motion trajectory of a robot arm according to claim 4, characterized in that: The second latest key area for identifying the interaction between the robot arm and the user or other users includes: When the key object does not appear in the third visual field of the user and other users for more than a second preset time period, an overlapping area of ​​the third visual field of the user and other users that overlaps for more than a third preset time period is taken as the second latest key area.

6. The method for optimizing the motion trajectory of a robot arm according to claim 4, characterized in that: The optimizing and replacing processing of the fourth partial trajectory based on the second interaction situation of the user interacting with other users when the robot arm moves along the third partial trajectory includes: Performing feature description processing on the second interaction situation to obtain a second feature description vector; Determining the interaction triggering degree corresponding to the second feature description vector from the interaction triggering degree library; When the interaction trigger degree exceeds the trigger degree threshold, the robot arm is planned to continuously dock with the key object; The fourth local trajectory is optimized and replaced so that the robot arm can exhibit docking behavior in the second latest key area when moving along the fourth local trajectory after the optimization and replacement processing.

7. A robot arm motion trajectory optimization system, characterized in that: include: A latest key area recognition module, used to recognize the first latest key area for interaction between the robot arm and the user; A local trajectory determination module, used to determine a first local trajectory before entering a first latest key area and a second local trajectory entering the first latest key area from a first current future motion trajectory of the robot arm; A trajectory optimization and replacement processing module, used for performing optimization and replacement processing on the second partial trajectory based on a first interaction situation between the robot arm and the user when the robot arm moves along the first partial trajectory; The latest key area recognition module recognizes the first latest key area for interaction between the robot arm and the user, including: Obtain the user's field of view change sequence after the robot arm last interacted with the user; Based on the appearance of the robot arm in each first field of view in the field of view change sequence, an appearance curve is drawn; When more than N target peaks higher than the first peak appear successively after the first peak in the appearance curve, the first and last peaks of the target peak are intercepted from the field of view change sequence, and each of them corresponds to a local field of view sequence between the first fields of view; wherein N is a positive integer greater than or equal to 2; Acquire a set of target objects that appear in every second field of view in the local field of view sequence; Matching the target object set with multiple standard object sets respectively; When the match is met, the key object indication rule of the matching standard object set is obtained; Based on the key object indication rule, determine the key objects and corresponding optimization categories from the target object set; When the optimization category is active optimization, the area of ​​the first visual field corresponding to the last occurrence of the key object in the first visual field in the trough of the appearance curve is taken as the first latest key area; When the optimization category is passive optimization, the area of ​​the overlapping fields of view between the target peaks corresponding to the first fields of view is taken as the first latest key area.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 6.

Images