Action generation method, robot control method and device and robot

Abstract

The invention provides an action generation method, a robot control method, a robot control device and a robot wherein the action generation method comprises: receiving an action input of a user, the action input comprising at least one group of actions to be executed, each group of actions to be executed comprising a background action and a body action cooperating with the background action, the starting time and the ending time of the background action and the limb action are acquired; and performing action generation based on the action input to obtain an action instruction, thereby realizing diversified action generation, overcoming the defect of limitation of a robot in action expression in a traditional scheme, and improving the user experience by supporting a mode of customizing the interaction action between the robot and a person by a user. According to the technical scheme, the action instruction meeting the expectation of the user and meeting the requirement of the user can be obtained, so that the robot has more action choices, the displayed action is more expressive, the interactivity and interestingness between the robot and people are improved, and richer, diversified and novel use experience is brought to the user.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an action generation method, a robot control method, a device and a robot. Background Art

[0002] Pet robots, as an emerging smart home product, have gradually become a reality and provide companionship and entertainment for people. Such robots are usually designed with biological appearance, such as robot dogs and robot cats, and provide users with an emotional experience similar to that of real pets by simulating the calls, movements, habits, etc. of pets.

[0003] Current pet robots mainly interact with users through images and voice. They can not only respond to users' encyclopedia questions and answers, but also perform specific actions according to instructions, such as squatting and waving hands. However, in terms of action execution, since the robot's action range is fixed at the factory, users can only choose within a limited range, which undoubtedly limits the diversity and flexibility of interaction, not only affecting the interactivity between users and robots, but also making it difficult to meet users' expectations for the diversity of robot actions. Summary of the invention

[0004] The present invention provides an action generation method, a robot control method, a device and a robot, which are used to solve the problem that robots in the prior art have obvious limitations in action performance, realize diversified action generation, and enable users to generate actions according to their own needs and personalized preferences, so that the generated actions are more expressive and can better meet user needs.

[0005] The present invention provides an action generation method, comprising: Receive an action input from a user, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and a start and end time of each of the background action and the body action; An action is generated based on the action input to obtain an action instruction.

[0006] According to an action generation method provided by the present invention, each group of actions to be executed further includes the execution times of the background action and the limb action, and the action execution parameters of the corresponding body parts; The step of generating an action based on the action input to obtain an action instruction includes: Performing action verification based on the action input, the action verification including action conflict verification and / or action validity verification; When the result of the action verification is that the verification is passed, an action is generated based on the action input to obtain an action instruction.

[0007] According to an action generation method provided by the present invention, the action input also corresponds to a companion input, the companion input is coordinated with the action to be executed in the action input, and the companion input includes at least one of expression input, sound effect input and light input; The expression input, the sound effect input and the light input respectively include the expression to be displayed, the sound effect to be displayed and the light to be displayed, as well as the start and end time of the expression to be displayed, the sound effect to be displayed and the light to be displayed; The background action and the body action are selected from a pre-constructed action library; the facial expression to be displayed, the sound effect to be displayed and the light to be displayed are selected from a pre-constructed facial expression library, sound effect library and light library respectively.

[0008] According to an action generation method provided by the present invention, the step of generating an action based on the action input to obtain an action instruction includes: receiving a preview instruction from the user, and applying the action input to perform an action preview based on the preview instruction; receiving feedback input from the user after the action preview, and generating an action adjustment output based on at least one of the start and end time, execution count, and action execution parameters of the background action and the body action in the action input if the feedback input indicates that the user is dissatisfied; The action adjustment output is displayed to enable the user to adjust the action input based on the action adjustment output.

[0009] According to an action generation method provided by the present invention, the action adjustment output is generated based on at least one of the start and end time, execution number and action execution parameter of the background action and the limb action in the action input, including: In the case where there is a concomitant input for the background action and the action to be performed corresponding to the body action, determining an input parameter of the corresponding concomitant input; Based on at least one of the start and end time, execution times and action execution parameters of the background action and the limb action in the action input, and the input parameters of the accompanying input corresponding to the background action and the limb action, an action adjustment output is generated.

[0010] According to an action generation method provided by the present invention, the action adjustment output includes at least one of an action sequence adjustment output, an action coordination adjustment output, and an action parameter adjustment output.

[0011] The present invention also provides a robot control method, comprising: Get action instructions; Based on the action instruction, control the robot to perform the action; The action instruction is determined based on any of the action generation methods described above.

[0012] The present invention also provides an action generating device, comprising: An input receiving unit, configured to receive an action input from a user, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and a start and end time of each of the background action and the body action; The action generation unit is used to generate an action based on the action input to obtain an action instruction.

[0013] The present invention also provides a robot control device, comprising: An instruction acquisition unit, used for acquiring action instructions; An action execution unit, used to control the robot to perform an action based on the action instruction; The action instruction is determined based on any of the action generation methods described above.

[0014] The present invention also provides a robot, the robot is communicatively connected with a user terminal; the robot comprises a processor; The processor is used to receive the action instruction sent by the user terminal, and control the robot to perform the action based on the action instruction; The action instruction is obtained by the user terminal generating an action based on the user's action input, wherein the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, physical movements coordinated with the background actions, and the start and end times of the background actions and the physical movements respectively.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements any one of the above-described action generation methods or robot control methods.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described action generation methods or robot control methods.

[0017] The action generation method, robot control method, device and robot provided by the present invention receive user action input, the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements, and generate actions based on the action input to obtain action instructions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting users to customize the interactive actions between the robot and humans, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the actions displayed are more expressive, thereby enhancing the interactivity and fun between the robot and humans, and bringing users a richer, more diverse and novel use experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flowchart of the action generation method provided by the present invention; Figure 2 is an example diagram of the action interface provided by the present invention; Figure 3 is an example diagram of the action library provided by the present invention; Figure 4 is an example diagram of the expression library provided by the present invention; Figure 5 is an example diagram of the sound effect library provided by the present invention; Figure 6 is an example diagram of the lighting library provided by the present invention; Figure 7 It is a flow chart of the robot control method provided by the present invention; Figure 8 It is a structural schematic diagram of the action generation device provided by the present invention; Fig. 9 It is a structural schematic diagram of the robot control device provided by the present invention; Fig.10 It is a structural schematic diagram of the robot provided by the present invention; Fig.11 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, pet robots interact with people mainly through voice, images, etc. They can not only respond to users' encyclopedia questions and answers, but also tell stories, interpret picture books, play music, etc., which greatly enriches users' daily lives. In addition, pet robots can also complete some specific actions according to users' instructions, such as squatting, wagging tails, waving hands, etc. These functions further enhance the interactivity and fun of robots.

[0022] However, existing pet robots still have some limitations in terms of motion performance. Specifically, since the robot's motion range is fixed when it leaves the factory, users can only choose within a limited range of motions and cannot customize the robot's motions according to actual needs or personalized preferences. This limitation not only affects the interaction between users and robots, but also makes it difficult to meet users' expectations for the diversity of robot motions.

[0023] In this regard, the present invention provides an action generation method, which aims to support users to customize the interactive actions between robots and humans, that is, actions can be generated according to the user's action input to obtain action instructions that meet the user's expectations. Through this process, users can flexibly set actions according to their own needs, thereby achieving more action performances. While meeting the personalized needs of users, it can also greatly enhance the interactivity and fun between robots and humans, bringing users a richer, more diverse and novel usage experience.

[0024] Figure 1 It is a flow chart of the action generation method provided by the present invention. Figure 1 As shown, it can be applied to robots, and can also be applied to user terminals that directly control robots, which are not specifically limited in the embodiments of the present invention. The robot here can be a pet robot, a child companion robot, a humanoid robot, etc.

[0025] Step 110, receiving a user's action input, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and a start and end time of the background action and the body action; Step 120: generate an action based on the action input to obtain an action instruction.

[0026] Specifically, in the process of action generation, considering that the current robot's action performance is limited to a pre-set action range and cannot jump out of this range to select and execute actions, resulting in the problem that the robot's performance lacks diversity and liveliness, in an embodiment of the present invention, it is proposed that interactive actions can be customized so that the robot can perform actions according to user needs and personalized preferences, thereby better meeting user expectations for the diversity of robot actions.

[0027] Based on this, in the embodiment of the present invention, when generating an action, the first thing that needs to be determined is the user's needs, specifically the embodiment of the user's needs in the generation of the action. In order to visualize this, the embodiment of the present invention also provides an action interface for users to edit and input actions. The action interface can be set on the robot or on the user terminal bound to the robot. The embodiment of the present invention does not make specific restrictions on this. As a preference, in order to facilitate user input, action editing can be performed anytime and anywhere. In the embodiment of the present invention, the action interface is set on the user terminal, and the user can input actions on this terminal. The user terminal here can be a smart phone, a tablet computer, etc.

[0028] Specifically, the user can input actions in the action interface. Figure 2 is an example diagram of the action interface provided by the present invention, such as Figure 2 As shown in the figure, the action interface is similar to a coordinate system, which includes a horizontal axis and a vertical axis, where the horizontal axis is the time axis and the vertical axis is the material axis. The material axis contains materials of multiple dimensions, such as action groups, expressions, sound effects, lighting, etc. Among them, the action group is the focus of attention. Users can input actions under this item, and after the input is completed, they can wait for the automatic action generation, and finally get the action instructions that match the user input, waiting for the robot to perform the action.

[0029] Among them, each action input by the user may include the action to be performed, as well as the start time and execution duration of the action. That is, the action input contains one or more groups of actions to be performed, and each group of actions to be performed contains the actions to be performed and the start and end time of the actions. It is worth noting that when the user sets the action in the action interface, the set actions to be performed are divided into background actions and non-background actions. Background actions act as "background" and exist throughout the execution of the entire group of actions, and are infinitely cyclic; non-background actions are normal limb movements, which can be divided into forelimb movements and rear wheel movements, and are executed according to the user-set / default parameters.

[0030] Moreover, the body movements and background movements in each group of actions to be executed may be one or more. For example, a group of background movements may include one background movement and two body movements, or two background movements and two body movements, or only one background movement and one body movement. The embodiments of the present invention do not make specific limitations on this.

[0031] It should be noted that since the background action is always executed, the body movements executed under this "background" must be coordinated with the background action and cannot be opposite or conflicting actions. For example, when the background action is "both ears swinging", the corresponding body movements can be "cross-pounding the ground", "turning in a circle on the spot", etc., but cannot be "both ears horizontal and motionless".

[0032] In addition, in determining the start and end time, the user sets the start time of each limb action and background action on the timeline in the action interface, and the end time of each action is determined by the execution duration. In addition, if there is only one background action in a group of actions to be executed, there is no need to set the start time and execution duration, and it will loop infinitely and execute continuously during the entire action process.

[0033] Further, after receiving the action input of the user, in the embodiment of the present invention, the action can be generated based on the action input to obtain an action instruction that meets the user's expectations and satisfies the user's needs. Here, specifically, after receiving the action input of the user and determining that the user input is completed, the action can be generated according to the action input to generate a set of completed action instructions, specifically, according to the data in the action input, the instruction is generated according to the rules, templates or other methods to generate action instructions that the robot can receive, understand and decode, so that when the robot performs the action, it can perform the action output in a complete and orderly manner according to the action input set by the user, and accurately display the action to be executed.

[0034] In an embodiment of the present invention, a simple action editing method based on an action interface can support users to customize the interactive actions between robots and humans, and each group of actions to be executed can contain multiple actions, and different actions can be arbitrarily combined on the timeline, thereby realizing the infinite combination possibilities of limited data, providing users with rich editing options, and the editing method is simple and easy to operate. After editing is completed, it can be directly generated or stored for subsequent use, which can not only make human-computer interaction more flexible and convenient, but also enhance the fun and vividness of robot interaction, and better meet the personalized needs of users.

[0035] The action generation method provided by the present invention receives the action input of the user, the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements, and generates actions based on the action input to obtain action instructions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting users to customize the interactive actions between the robot and humans, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the displayed actions are more expressive, thereby enhancing the interactivity and fun between the robot and humans, and bringing users a richer, more diverse and novel use experience.

[0036] Based on the above embodiment, each group of actions to be executed also includes the execution times of the background action and the limb action, as well as the action execution parameters of the corresponding body parts; Step 120 includes: Perform action verification based on action input, where the action verification includes action conflict verification and / or action validity verification; When the result of the action verification is that the verification is passed, the action is generated based on the action input to obtain the action instruction.

[0037] Specifically, in step 120, the process of generating an action according to the action input to obtain an action instruction specifically includes the following steps: See also Figure 2 It can be seen that when the user sets each group of actions to be executed in the action interface, for each action (background action and body action) in each group of actions to be executed, there is a parameter setting column / window, from which the parameters can be set. The specific parameters set can include the number of repetitions of the action, that is, the number of repetitions plus one is the number of executions of the action, and the motion parameters of the body parts of the robot involved in each action, that is, the action execution parameters of the body parts corresponding to the action. The action execution parameters correspond to the specific actions, and the parameters corresponding to different actions may be different. For example, the action execution parameters corresponding to the action "double ear swing" are "ear left angle", "ear left speed", "ear right angle", "ear right speed", etc.; the action execution parameters corresponding to the action "up and down cross thumping" are "forelimb left angle", "forelimb left speed", "forelimb right angle", "forelimb right speed", etc.; the action execution parameters corresponding to the action "turn in place" are "rear wheel left linear speed", "rear wheel right linear speed", "rear wheel left angular speed", etc.

[0038] Based on this, in an embodiment of the present invention, after receiving the user's action input and before actually generating the action, considering that due to the large number of actions and parameters to be set, it is difficult for the user to fully control all actions and ensure that all actions cooperate with each other and do not conflict with each other, and can be matched to achieve better results, therefore, in an embodiment of the present invention, the action input can be first checked to verify whether there is a problem, thereby obtaining the verification result.

[0039] Specifically, action conflict check and / or action validity check can be performed based on the action input to verify whether there are coordination conflicts and time conflicts within each group of actions to be executed input by the user, whether each action is valid, and whether there are time conflicts and validity between different groups of actions to be executed. Through multi-faceted and multi-angle verification, the success rate and smoothness of action generation can be better guaranteed, as well as the continuity, coordination and liveliness of the actions presented by the generated action instructions when the actions are executed.

[0040] After that, it can be determined whether to generate an action based on the result of the action verification. Specifically, when the verification passes, that is, the result of the action verification is verification passed, the action can be generated based on the user's action input to obtain an action instruction that meets the user's expectations; on the contrary, if the verification fails, that is, the result of the action verification is verification failed, the action input needs to be adjusted to remove invalid settings and avoid conflicts, and then the action verification is performed again until the verification passes, and then the action can be generated.

[0041] In the embodiment of the present invention, during the action generation process, an action conflict check and / or an action validity check is first performed, and then the action is generated based on the check result. This can not only improve the success rate and smoothness of the action generation, but also enhance the coordination and vividness of the generated action, and provide a basis for action adjustment, thereby making it more accurate and expressive, and better meeting the personalized needs of users.

[0042] Based on the above embodiment, the action input also corresponds to an accompanying input, which is coordinated with the action to be executed in the action input, and includes at least one of an expression input, a sound effect input, and a light input; the expression input, the sound effect input, and the light input respectively include an expression to be displayed, a sound effect to be displayed, and a light to be displayed, as well as the start and end times of the expression to be displayed, the sound effect to be displayed, and the light to be displayed; Background actions and body actions are selected from a pre-built action library; expressions to be displayed, sound effects to be displayed, and lights to be displayed are selected from a pre-built expression library, sound effect library, and light library, respectively.

[0043] Specifically, see Figure 2It can be seen that under the vertical axis in the action interface, there are not only action groups for users to input actions, but also data of other dimensions that match the action groups. Users can input in the corresponding dimensions, and this input can be output along with the action to be executed.

[0044] Specifically, in an embodiment of the present invention, the action input also corresponds to an accompanying input, which is coordinated / corresponds to the action to be executed in the action input, but it should be noted that each group of actions to be executed corresponds to an accompanying input. Here, the accompanying input includes at least one of expression input, sound effect input, and lighting input, which correspond to the expression, sound effect, and lighting under the vertical axis, respectively, wherein the expression input includes the expression to be displayed, and its corresponding start and end time; the sound effect input and the lighting input respectively include the expression to be displayed and its corresponding start and end time, and the lighting to be displayed and its corresponding start and end time; the method for determining the start and end time here is the same as the method for determining the start and end time in the action input, both of which are to set the start time and then determine the end time by the execution time.

[0045] It should be noted that the accompanying input corresponds to the action group to be executed, not the background action or body movement in the action group to be executed. The same body movement may correspond to multiple expressions to be displayed, multiple sound effects to be displayed, multiple lights to be displayed, etc., and the same sound effect to be displayed may also correspond to multiple expressions to be displayed and multiple lights to be displayed. All expressions, lights, sound effects, etc. can be freely set according to user needs.

[0046] It is worth noting that in order to further improve the convenience of user input, in the embodiment of the present invention, a material library can also be preset. The material library here includes at least one of an expression library, a sound effect library, and a lighting library, as well as an action library. When the user inputs an action, the user can directly select a material from the corresponding material library, specifically by clicking the insert button under the corresponding dimension ( Figure 2 ), and then jump to the material selection interface to directly select materials from the corresponding material library. Figure 3 is an example diagram of the action library provided by the present invention, Figure 4 is an example diagram of the expression library provided by the present invention, Figure 5 is an example diagram of the sound effect library provided by the present invention, Figure 6 is an example diagram of the lighting library provided by the present invention, such as Figures 3 to 6As shown in the figure, the action library contains a large number of actions suitable for robot output, the expression library contains a large number of expressions suitable for robot output, the sound effect library contains a large number of expression sound effect data suitable for robot output, and the lighting library contains a large number of lighting data suitable for robot output. When users input actions and accompanying inputs, they can directly jump through the buttons to directly select materials from the corresponding material library. After selection, they can further modify and adjust the parameters of the selected actions, expressions, sound effects, lights, etc. to make them more suitable for specific scenes and more coordinated with other materials.

[0047] In the embodiment of the present invention, materials are directly selected from the corresponding material library, so that action input and accompanying input can be more convenient and quick, and materials of multiple dimensions can be arbitrarily combined on the timeline, realizing the infinite combination possibility of limited data, providing users with rich editing options, and the editing method is simple and easy to operate. After editing is completed, it can be directly generated or stored for subsequent use, which can not only make human-computer interaction more flexible and convenient, but also enhance the fun and vividness of robot interaction, and better meet the personalized needs of users.

[0048] Based on the above embodiment, step 120 includes: Receive a preview instruction from the user, and based on the preview instruction, apply the action input to perform an action preview; receiving feedback input from the user after the action preview, and generating an action adjustment output based on at least one of the start and end time, execution count, and action execution parameter of the background action and the body action in the action input if the feedback input indicates that the user is dissatisfied; The action adjustment output is displayed to enable the user to adjust the action input based on the action adjustment output.

[0049] Specifically, the above process of generating an action according to the action input to obtain an action instruction may include: It is understandable that after receiving the user's action input, in order to ensure that the effect presented by the final generated action instruction when executed is consistent with the expectation, in an embodiment of the present invention, an action preview can be performed before the final action is generated to preview the action to see if there is anything wrong and whether it meets one's own expectations.

[0050] Specifically, the user's preview instruction may be received first, and the preview instruction may be the user directly clicking a preview button on the action interface ( Figure 2), or it can be content inputted by voice, text, etc. to indicate the need to preview the action. This can be used directly as a preview instruction to wait for a command response, or a preview instruction can be further generated based on this to wait for a command response. Then, the action can be previewed according to this preview instruction, that is, the action can be previewed according to the action input to preview the action effect presented by the action instruction corresponding to the action input after the action is executed to see whether it meets expectations, and feedback can be received after the preview, that is, feedback input from the user after the action preview.

[0051] Furthermore, if the user feedback indicates that the preview effect is not good, that is, the feedback input indicates that the user is not satisfied with the preview effect, and the action effect presented in the preview does not meet the user's expectations, then it can be determined that there are one or more problems with the user's action input and it needs to be adjusted. Therefore, in this case, an action adjustment output can be generated based on at least one of the start and end time, execution count and action execution parameters of the background action and body action in the action input to serve as a prompt, thereby assisting the user in making action adjustments.

[0052] Specifically, here, when it is confirmed that there is a problem with the current action input, the time, number of times, parameters, etc. of each action in the action input can be combined to analyze and determine which aspect has the problem, whether there is a conflict between actions, or the actions are incompatible, the effect presented by the action combination is weird, uncoordinated, abrupt, etc., and action adjustment suggestions are generated for the user based on the analyzed information and output, that is, action adjustment output is generated.

[0053] After that, the action adjustment output can be displayed so that the user can adjust the current action input according to the displayed action adjustment output. Here, the action adjustment output can be displayed in a pop-up window, in a blank space or a specified position, in a floating display, or directly highlighted at the location where the problem exists, which is not specifically limited in the embodiment of the present invention.

[0054] Based on the above embodiment, based on at least one of the start and end time, execution times and action execution parameters of the background action and the body action in the action input, generating an action adjustment output includes: In the case where the background action and the to-be-executed action corresponding to the body action have accompanying input, determining input parameters of the corresponding accompanying input; Generate an action adjustment output based on at least one of the start and end time, execution count and action execution parameter of each of the background action and the body action in the action input, and the input parameters of the accompanying input corresponding to the background action and the body action; The action adjustment output includes at least one of an action sequence adjustment output, an action coordination adjustment output, and an action parameter adjustment output.

[0055] Specifically, the process of generating the action adjustment output according to at least one of the start and end time, execution times and action execution parameters of the background action and the body action in the action input may specifically include: When it is determined that the current action input has a problem, the action to be executed with the problem can be first determined based on the feedback input, and whether this group or groups of actions to be executed have corresponding accompanying inputs. If the user's feedback input does not indicate which group of actions to be executed has a problem, it is determined whether each group of actions to be executed in the action input has accompanying inputs.

[0056] Next, if it is determined that there is an accompanying input, it is necessary to determine the input parameters of this accompanying input, that is, the input parameters of at least one of the corresponding expression input, sound effect input, and lighting input. The input parameters here can be the amplitude of the expression, the volume of the sound effect, the brightness of the light, the color temperature, etc.

[0057] After that, the action adjustment output can be generated according to the input parameters of this accompanying input, that is, based on the start and end time, execution times and action execution parameters of the background action and the limb action in the problematic action to be executed, combined with the input parameters of the corresponding accompanying input, the problem analysis can be performed to analyze and determine which aspect has the problem, whether there is a conflict between different actions in each group of actions to be executed, the actions are not compatible, or the actions to be executed and the corresponding accompanying inputs do not match, and the accompanying output causes the effect presented to be weird, uncoordinated, abrupt, etc., and the action adjustment output is generated according to the analyzed information. The action adjustment output here includes any one or more of the action sequence adjustment output, the action coordination adjustment output, and the action parameter adjustment output.

[0058] Among them, action sequence adjustment output refers to the analysis that the order setting between different actions in the action input is unreasonable, and the suggestions for adjusting the order of actions are given; action coordination adjustment output refers to the analysis that different actions in the action input do not match, and / or the action input and the accompanying input do not match, and the suggestions for adjusting the matching relationship between actions, and / or the matching relationship between actions and accompanying inputs are given; action parameter adjustment output refers to the analysis that the parameters of the action in the action input are improperly set, and the suggestions for adjusting the parameters of the action (number of executions, action execution parameters, start and end time, etc.) are given.

[0059] In an embodiment of the present invention, when there is a problem with the action input, an action adjustment output is generated and displayed based on the start and end time, the number of executions and at least one of the action execution parameters of the background action and the limb action in the action input, as well as the input parameters of the accompanying input corresponding to the background action and the limb action, so that the user can adjust the action input according to the action adjustment output, which can not only improve the success rate and smoothness of action generation, but also enhance the continuity, coordination and vividness of the actions presented by the generated action instructions when the action is executed.

[0060] The present invention also provides a robot control method. Figure 7 It is a flow chart of the robot control method provided by the present invention, such as Figure 7 As shown, the method can be applied to a robot, and the specific execution subject can be a processor in the robot. The method includes: Step 710, obtaining an action instruction; Step 720, based on the action instruction, controlling the robot to perform the action; The action instructions are determined based on any of the action generation methods described above.

[0061] Specifically, taking into account the limitations of current robots in action execution, namely, they can only select and execute according to a set action range, and are unable to execute actions according to user needs, personalized preferences, etc., resulting in a lack of liveliness and interest in the human-computer interaction process, which can easily lead to a poor user experience. In an embodiment of the present invention, it is proposed that actions can be generated in advance, that is, users can input actions in the action interface according to their own needs, and actions can be generated based on the action input to obtain action instructions that meet user expectations and meet personalized needs. The processor can obtain this action instruction and can control the robot to execute the action accordingly.

[0062] Among them, each action input by the user may include the action to be performed, as well as the start time and execution duration of the action. That is, the action input contains one or more groups of actions to be performed, and each group of actions to be performed contains the actions to be performed and the start and end time of the actions. It is worth noting that when the user sets the action in the action interface, the set actions to be performed are divided into background actions and non-background actions. Background actions act as "background" and exist throughout the execution of the entire group of actions, and are infinitely cyclic; non-background actions are normal limb movements, which can be divided into forelimb movements and rear wheel movements, and are executed according to the user-set / default parameters.

[0063] Moreover, the body movements and background movements in each group of actions to be executed may be one or more. For example, a group of background movements may include one background movement and two body movements, or two background movements and two body movements, or only one background movement and one body movement. The embodiments of the present invention do not make specific limitations on this.

[0064] It should be noted that since the background action is always executed, the body movements executed under this "background" must be coordinated with the background action and cannot be opposite or conflicting actions. For example, when the background action is "both ears swinging", the corresponding body movements can be "cross-pounding the ground", "turning in a circle on the spot", etc., but cannot be "both ears horizontal and motionless".

[0065] In addition, in determining the start and end time, the user sets the start time of each limb action and background action on the timeline in the action interface, and the end time of each action is determined by the execution duration. In addition, if there is only one background action in a group of actions to be executed, there is no need to set the start time and execution duration, and it will loop infinitely and execute continuously during the entire action process.

[0066] When generating actions, instructions can be generated according to the data in the action input, according to rules, templates or other methods, to generate action instructions that the robot can receive, understand and decode, so that when the robot performs actions, it can follow the action input set by the user, perform action outputs completely and orderly, and accurately display the actions to be executed.

[0067] In an embodiment of the present invention, a simple action editing method based on an action interface can support users to customize the interactive actions between robots and humans, and each group of actions to be executed can contain multiple actions, and different actions can be arbitrarily combined on the timeline, thereby realizing the infinite combination possibilities of limited data, providing users with rich editing options, and the editing method is simple and easy to operate. After editing is completed, it can be directly generated or stored for subsequent use, which can not only make human-computer interaction more flexible and convenient, but also enhance the fun and vividness of robot interaction, and better meet the personalized needs of users.

[0068] The robot control method provided by the present invention obtains action instructions, and controls the robot to perform actions based on the action instructions; the action instructions are generated by performing actions according to the action input of the user, and the action input includes at least one group of actions to be performed, and each group of actions to be performed includes background actions, body actions coordinated with the background actions, and the start and end times of the background actions and the body actions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting the user to customize the interactive actions between the robot and the human, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the actions displayed are more expressive, thereby enhancing the interactivity and fun between the robot and the human, and bringing the user a richer, more diverse and novel use experience.

[0069] The action generation device provided by the present invention is described below. The action generation device described below and the action generation method described above can be referenced to each other.

[0070] Figure 8 is a schematic diagram of the structure of the action generation device provided by the present invention, such as Figure 8 As shown, the device comprises: The input receiving unit 810 is used to receive the action input of the user, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and the start and end time of the background action and the body action; The action generation unit 820 is used to generate an action based on the action input to obtain an action instruction.

[0071] The action generation device provided by the present invention receives the action input of the user, the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements, and generates actions based on the action input to obtain action instructions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting users to customize the interactive actions between the robot and humans, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the actions displayed are more expressive, thereby enhancing the interactivity and fun between the robot and humans, and bringing users a richer, more diverse and novel use experience.

[0072] Based on the above embodiment, each group of actions to be executed further includes the execution times of the background action and the limb action, and the action execution parameters of the corresponding body parts; The action generation unit 820 is used to: Performing action verification based on the action input, the action verification including action conflict verification and / or action validity verification; When the result of the action verification is that the verification is passed, an action is generated based on the action input to obtain an action instruction.

[0073] Based on the above embodiment, the action input further corresponds to an accompanying input, the accompanying input is coordinated with the action to be executed in the action input, and the accompanying input includes at least one of an expression input, a sound effect input, and a light input; the expression input, the sound effect input, and the light input respectively include an expression to be displayed, a sound effect to be displayed, and a light to be displayed, as well as the start and end times of the expression to be displayed, the sound effect to be displayed, and the light to be displayed; The background action and the body action are selected from a pre-constructed action library; the facial expression to be displayed, the sound effect to be displayed and the light to be displayed are selected from a pre-constructed facial expression library, sound effect library and light library respectively.

[0074] Based on the above embodiment, the action generation unit 820 is used to: receiving a preview instruction from the user, and applying the action input to perform an action preview based on the preview instruction; receiving feedback input from the user after the action preview, and generating an action adjustment output based on at least one of the start and end time, execution count, and action execution parameters of the background action and the body action in the action input if the feedback input indicates that the user is dissatisfied; The action adjustment output is displayed to enable the user to adjust the action input based on the action adjustment output.

[0075] Based on the above embodiment, the action generation unit 820 is used to: In the case where there is a concomitant input for the background action and the action to be performed corresponding to the body action, determining an input parameter of the corresponding concomitant input; Based on at least one of the start and end time, execution times and action execution parameters of the background action and the limb action in the action input, and the input parameters of the accompanying input corresponding to the background action and the limb action, an action adjustment output is generated.

[0076] Based on the above embodiment, the action adjustment output includes at least one of an action sequence adjustment output, an action coordination adjustment output, and an action parameter adjustment output.

[0077] The robot control device provided by the present invention is described below. The robot control device described below and the robot control method described above can be referenced to each other.

[0078] Fig. 9 Schematic diagram of the structure of the robot control device provided by the present invention, such as Fig. 9 As shown, the device comprises: An instruction acquisition unit 910, used to acquire an action instruction; An action execution unit 920, used to control the robot to perform an action based on the action instruction; The action instruction is determined based on any of the action generation methods described above.

[0079] The robot control device provided by the present invention obtains action instructions, and controls the robot to perform actions based on the action instructions; the action instructions are generated by performing actions according to the action input of the user, and the action input includes at least one group of actions to be performed, and each group of actions to be performed includes background actions, body actions coordinated with the background actions, and the start and end times of the background actions and the body actions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting the user to customize the interactive actions between the robot and the human, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the actions displayed are more expressive, thereby enhancing the interactivity and fun between the robot and the human, and bringing the user a richer, more diverse and novel use experience.

[0080] Fig.10 : is a schematic diagram of the structure of the robot provided by the present invention, such as Fig.10 As shown, the robot is in communication connection with a user terminal; the robot includes a processor 1010; The processor 1010 is used to receive the action instruction sent by the user terminal, and control the robot to perform the action based on the action instruction; The action instruction is obtained by the user terminal generating an action based on the user's action input, wherein the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, physical movements coordinated with the background actions, and the start and end times of the background actions and the physical movements respectively.

[0081] The robot provided by the present invention includes a processor, which is used to receive action instructions sent by a user terminal that is communicatively connected to the robot, and control the robot to perform actions based on the action instructions; the action instructions are generated by the user terminal according to the user's action input, and the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body actions coordinated with the background actions, and the start and end times of the background actions and the body actions, thereby realizing diversified action generation and overcoming the defects of the robot's limitations in action performance in traditional solutions. By supporting users to customize the interactive actions between the robot and humans, action instructions that meet the user's expectations and adapt to their needs can be obtained, so that the robot has more action options and the actions displayed are more expressive, thereby enhancing the interactivity and fun between the robot and humans, and bringing users a richer, more diverse and novel use experience.

[0082] Fig.11 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.11 As shown, the electronic device may include: a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 may call the logic instructions in the memory 1130 to execute the action generation method or the robot control method, wherein the action generation method includes: receiving the action input of the user, wherein the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements; performing action generation based on the action input to obtain an action instruction. The robot control method includes: obtaining an action instruction; based on the action instruction, controlling the robot to perform an action; and the action instruction is determined based on the action generation method described in any one of the above.

[0083] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0084] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by the computer, the computer can execute the action generation method or robot control method provided by the above methods, wherein the action generation method includes: receiving the user's action input, the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements; based on the action input, action generation is performed to obtain action instructions. The robot control method includes: obtaining action instructions; based on the action instructions, controlling the robot to perform actions; the action instructions are determined based on the action generation method as described in any of the above.

[0085] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the action generation method or robot control method provided by the above methods, wherein the action generation method includes: receiving a user's action input, wherein the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, body movements coordinated with the background actions, and the start and end times of the background actions and the body movements; performing action generation based on the action input to obtain an action instruction. The robot control method includes: obtaining an action instruction; based on the action instruction, controlling the robot to perform an action; the action instruction is determined based on the action generation method as described in any one of the above.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An action generation method, characterized in that: include: Receive an action input from a user, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and a start and end time of each of the background action and the body action; An action is generated based on the action input to obtain an action instruction.

2. The action generation method according to claim 1, characterized in that: Each group of actions to be executed also includes the execution times of the background action and the limb action, and the action execution parameters of the corresponding body parts; The step of generating an action based on the action input to obtain an action instruction includes: Performing action verification based on the action input, the action verification including action conflict verification and / or action validity verification; When the result of the action verification is that the verification is passed, an action is generated based on the action input to obtain an action instruction.

3. The action generation method according to claim 1, characterized in that: The action input also corresponds to an accompanying input, which is coordinated with the action to be executed in the action input, and includes at least one of an expression input, a sound effect input, and a light input; the expression input, the sound effect input, and the light input respectively include an expression to be displayed, a sound effect to be displayed, and a light to be displayed, as well as the start and end times of the expression to be displayed, the sound effect to be displayed, and the light to be displayed; The background action and the body action are selected from a pre-constructed action library; the facial expression to be displayed, the sound effect to be displayed and the light to be displayed are selected from a pre-constructed facial expression library, sound effect library and light library respectively.

4. The action generation method according to claim 3, characterized in that: The step of generating an action based on the action input to obtain an action instruction includes: receiving a preview instruction from the user, and applying the action input to perform an action preview based on the preview instruction; receiving feedback input from the user after the action preview, and generating an action adjustment output based on at least one of the start and end time, execution count, and action execution parameters of the background action and the body action in the action input if the feedback input indicates that the user is dissatisfied; The action adjustment output is displayed to enable the user to adjust the action input based on the action adjustment output.

5. The action generation method according to claim 4, characterized in that: The generating of the action adjustment output based on at least one of the start and end time, execution times and action execution parameters of the background action and the body action in the action input comprises: In the case where there is a concomitant input for the background action and the action to be performed corresponding to the body action, determining an input parameter of the corresponding concomitant input; Based on at least one of the start and end time, execution times and action execution parameters of the background action and the limb action in the action input, and the input parameters of the accompanying input corresponding to the background action and the limb action, an action adjustment output is generated.

6. The action generation method according to claim 5, characterized in that: The action adjustment output includes at least one of an action sequence adjustment output, an action coordination adjustment output, and an action parameter adjustment output.

7. A robot control method, characterized in that: include: Get action instructions; Based on the action instruction, control the robot to perform the action; The motion instruction is determined based on the motion generation method according to any one of claims 1 to 6.

8. An action generating device, characterized in that: include: An input receiving unit, configured to receive an action input from a user, wherein the action input includes at least one group of actions to be performed, each group of actions to be performed includes a background action, a body action coordinated with the background action, and a start and end time of each of the background action and the body action; The action generation unit is used to generate an action based on the action input to obtain an action instruction.

9. A robot control device, characterized in that: include: An instruction acquisition unit, used for acquiring action instructions; An action execution unit, used to control the robot to perform an action based on the action instruction; The motion instruction is determined based on the motion generation method according to any one of claims 1 to 6.

10. A robot, characterized in that: The robot is communicatively connected with a user terminal; the robot comprises a processor; The processor is used to receive the action instruction sent by the user terminal, and control the robot to perform the action based on the action instruction; The action instruction is obtained by the user terminal generating an action based on the user's action input, wherein the action input includes at least one group of actions to be executed, each group of actions to be executed includes background actions, physical movements coordinated with the background actions, and the start and end times of the background actions and the physical movements respectively.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the action generation method according to any one of claims 1 to 6 or the robot control method according to claim 7 is implemented.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the action generation method according to any one of claims 1 to 6 or the robot control method according to claim 7 is implemented.

Images