Task generation method, storage medium, electronic equipment and program product

Abstract

The invention provides a task generation method, a storage medium, electronic equipment and a program product, and relates to the technical field of robots. The method comprises the following steps: receiving a first input for triggering video acquisition based on a task generation interface; obtaining a target video in response to the first input; receiving a second input for triggering task generation based on the task generation interface; and in response to the second input, simulating an object behavior in the target video, and generating an equipment performance task corresponding to the object behavior. According to the embodiment of the invention, the secondary development difficulty of intelligent agent equipment such as a robot can be reduced, and the secondary development efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a task generation method, a storage medium, an electronic device and a program product. BACKGROUND

[0002] With the penetration of robot technology from structured factories to diversified scenarios, the demand for customized secondary development of robot behavior is increasingly urgent. This process aims to endow general robots with exclusive skills suitable for specific tasks, becoming a key bridge connecting hardware platforms and complex applications.

[0003] At present, when the secondary development of robot behavior is carried out, a series of cumbersome processes such as offline motion capture and online model construction are required, which makes the secondary development difficult and low in efficiency. SUMMARY

[0004] Therefore, the embodiments of the present application provide a task generation method, a storage medium, an electronic device and a program product.

[0005] In a first aspect, an embodiment of the present application provides a task generation method, which comprises: receiving a first input for triggering video acquisition based on a task generation interface; in response to the first input, acquiring a target video; receiving a second input for triggering task generation based on the task generation interface; in response to the second input, simulating an object behavior in the target video, and generating a device performance task corresponding to the object behavior.

[0006] In combination with the first aspect, in some implementations of the first aspect, the method further comprises: receiving a third input for a task generation mode display area in the task generation interface; in response to the third input, displaying at least one task generation mode; receiving a fourth input for a target generation mode in the at least one task generation mode; in response to the fourth input, displaying the target generation mode in the task generation mode display area. Correspondingly, simulating the object behavior in the target video and generating the device performance task corresponding to the object behavior comprises: simulating the object behavior in the target video based on the target generation mode, and generating the device performance task corresponding to the object behavior.

[0007] In combination with the first aspect, in some implementations of the first aspect, the method further comprises: receiving a fifth input for adjusting a video configuration parameter; in response to the fifth input, determining a target configuration parameter, wherein the target configuration parameter matches the content of the target video. Correspondingly, simulating the object behavior in the target video and generating the device performance task corresponding to the object behavior comprises: simulating the object behavior in the target video based on the target configuration parameter, and generating the device performance task corresponding to the object behavior.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, after generating the device performance task corresponding to the object behavior, the method further includes: displaying task decomposition information corresponding to the device performance task in chronological order in the timeline editor.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, after displaying the task breakdown information corresponding to the device performance task in chronological order, the method further includes: receiving a sixth input for a target time frame in the timeline editor; determining the target time frame in response to the sixth input; displaying a preset material library, the preset material library including at least one preset behavior material; receiving a seventh input for a target behavior material in the preset material library; and inserting the target behavior material into the device performance task based on the target time frame in response to the seventh input.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the behavioral material includes at least one of audio material, motion material, and facial expression material.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, after generating the device performance task corresponding to the object behavior, the method further includes: displaying a device performance simulation screen corresponding to the device performance task based on the task generation interface.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, after generating the device performance task corresponding to the object behavior, the method further includes: receiving an eighth input for triggering task deployment; and in response to the eighth input, deploying the device performance task to a target device so that the target device performs the device performance task.

[0013] Secondly, one embodiment of this application provides a task generation apparatus, which includes: a first receiving module, configured to receive a first input for triggering video acquisition based on a task generation interface; a video acquisition module, configured to acquire a target video in response to the first input; a second receiving module, configured to receive a second input for triggering task generation based on the task generation interface; and a task generation module, configured to simulate object behavior in the target video in response to the second input, and generate a device performance task corresponding to the object behavior.

[0014] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a third receiving module, configured to receive a third input to a task generation mode display area in the task generation interface; a first display module, configured to display at least one task generation mode in response to the third input; a fourth receiving module, configured to receive a fourth input to a target generation mode among the at least one task generation modes; and a second display module, configured to display the target generation mode in the task generation mode display area in response to the fourth input. Correspondingly, the task generation module is also configured to simulate object behavior in the target video based on the target generation mode, and generate a device performance task corresponding to the object behavior.

[0015] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a fifth receiving module, configured to receive a fifth input for adjusting video configuration parameters; and a parameter determining module, configured to determine target configuration parameters in response to the fifth input, wherein the target configuration parameters match the content of the target video. Correspondingly, the task generation module is also configured to simulate object behavior in the target video based on the target configuration parameters, and generate a device performance task corresponding to the object behavior.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a third display module, used to display, in the timeline editor, task decomposition information corresponding to the device performance task in chronological order.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a sixth receiving module, configured to receive a sixth input for a target time frame in the timeline editor; a target determination module, configured to determine the target time frame in response to the sixth input; a fourth display module, configured to display a preset material library, the preset material library including at least one preset behavior material; a seventh receiving module, configured to receive a seventh input for a target behavior material in the preset material library; and a material insertion module, configured to insert the target behavior material into the device performance task based on the target time frame in response to the seventh input.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, behavioral material includes at least one of audio material, motion material, and facial expression material.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a fifth display module, used to display a device performance simulation screen corresponding to the device performance task based on the task generation interface.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: an eighth receiving module, configured to receive an eighth input for triggering task deployment; and a task deployment module, configured to deploy the device performance task to a target device in response to the eighth input, so that the target device performs the device performance task.

[0021] Thirdly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the task generation method described in the first aspect.

[0022] Fourthly, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute the task generation method described in the first aspect.

[0023] Fifthly, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the task generation method described in the first aspect.

[0024] In this application, the target video that the device needs to imitate is conveniently obtained by using the task generation interface, and the behavior of the object in the target video is quickly simulated to generate a device performance task that simulates the behavior of the object. This allows intelligent devices such as robots to perform the device performance task, making it convenient and quick to complete the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development. Attached Figure Description

[0025] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 The diagram shown is a flowchart of a task generation method provided in an embodiment of this application.

[0027] Figure 2 The diagram shown is a schematic of a task generation interface provided in an embodiment of this application.

[0028] Figure 3 The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0029] Figure 4 The diagram shown is a schematic of a task generation interface provided in another embodiment of this application.

[0030] Figure 5 The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0031] Figure 6 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0032] Figure 7 The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0033] Figure 8 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0034] Figure 9 The diagram shown is a structural schematic of a task generation device provided in an embodiment of this application.

[0035] Figure 10 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In some instances, methods and means well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0038] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0039] Furthermore, the terms “first,” “second,” “third,” and “fourth” are used only for distinguishing descriptions and should not be interpreted as indicating or implying relative importance.

[0040] With the continuous advancement and development of robotics technology, more and more industries and fields are beginning to utilize robots to replace humans in performing specific tasks. Against this backdrop, in order to enable robots to better adapt to various complex and diverse application scenarios, it is often necessary to conduct secondary development and optimization of their behavior patterns. However, currently, the entire process of secondary development for the behavior of robots and other intelligent devices is particularly cumbersome and complex. Specifically, this process typically involves a series of meticulous and time-consuming operations, such as offline motion capture and online model building. These complex processes not only significantly increase the technical difficulty of secondary development but also significantly reduce overall development efficiency, posing certain obstacles and challenges to practical applications.

[0041] This application aims to construct a comprehensive and efficient one-stop platform, enabling users to easily perform secondary development of the behavior of intelligent agents such as robots by leveraging the various functional modules provided by the task generation interface. In the traditional model, users often need to go through a series of complex and cumbersome procedures to perform such secondary development, which not only consumes a lot of time and energy but also places high demands on the user's skill level. However, with the platform provided by this application, users can smoothly complete the secondary development process of intelligent agent behavior through intuitive and simple interface interaction, greatly simplifying the operation steps, lowering the technical threshold for secondary development, and significantly improving the overall efficiency of secondary development.

[0042] Specifically, the solution provided in this application encompasses an innovative function: allowing users to upload video files and, through a simple and clear human-computer interaction method, enabling intelligent agent devices to mimic the behaviors and actions of the subjects filmed in the videos. This function allows intelligent agent devices to quickly learn and reproduce specific behaviors, effectively achieving the goal of secondary development of intelligent agent device behavior. This approach not only fully demonstrates the intelligence and convenience of the secondary development process but also provides users with more flexible and diverse secondary development methods, further expanding the application scenarios and potential of intelligent agent devices.

[0043] The following is combined Figures 1 to 8 The task generation method provided in this application is described in detail.

[0044] Figure 1 The diagram shown is a flowchart illustrating a task generation method according to an embodiment of this application. This method can be applied to electronic devices; exemplarily, electronic devices may include smart devices such as computers and mobile phones. Figure 1 As shown, the method may include the following steps.

[0045] S110, based on the task generation interface, receives the first input used to trigger video acquisition.

[0046] In some examples, the task generation interface can be an interface set up based on a pre-defined secondary development platform. This interface may include multiple functional controls, such as controls for triggering video acquisition and controls for triggering task generation. Users can easily perform secondary development on the device behavior of robots and other intelligent devices by manipulating these functional controls. In some specific examples, the task generation interface may be, for example, as follows: Figure 2 The interface shown includes a video acquisition control 21 for triggering video acquisition and a task generation control 22 for triggering task generation.

[0047] In some examples, the first input can be an input that triggers the acquisition of video, including but not limited to clicks, drags, etc. For example, Figure 2 As shown, the first input can be the user dragging the target video to the video acquisition control 21, or the user clicking the video acquisition control 21 to select the video file corresponding to the target video from the local file, or the user clicking the video acquisition control 21 to trigger the connected camera to record video in order to obtain the target video.

[0048] S120, in response to the first input, acquires the target video.

[0049] In some examples, the target video can be a video selected or recorded by the user, and the video content of the target video may contain object behaviors that the intelligent agent device needs to imitate.

[0050] like Figure 2 As shown, in some specific examples, users can drag and drop the target video file into the video acquisition control 21, and the platform can then obtain the target video through the task generation interface. In other specific examples, users can click on a specific area of ​​the video acquisition control 21 to trigger the display of a local file browsing window. Through this window, users can select the desired target video file from the corresponding video storage path and then obtain the target video from that path after confirming the selection. In still other specific examples, users can also click on other specific areas of the video acquisition control 21 to trigger the activation of a camera connected to the platform, and then record video through that camera to obtain the target video.

[0051] S130, based on the task generation interface, receives a second input to trigger task generation.

[0052] In some examples, the second input can be an input that triggers the generation of a task that the intelligent agent device can perform, including but not limited to actions such as clicking. For example, Figure 2As shown, the second input can be the user's action of clicking the task generation control 22.

[0053] S140, in response to the second input, simulates the behavior of an object in the target video and generates a device performance task corresponding to the object behavior.

[0054] In some examples, a device performance task can be a task that the platform simulates based on the behavior of an object in a target video, and generates a task that the intelligent agent device can perform based on the simulation results. The object's behavior includes, but is not limited to, actions, facial expressions, and sounds.

[0055] like Figure 2 As shown, in some specific examples, users can click on the task generation control 22 to trigger the platform to simulate the user's actions, expressions, and voices in the target video uploaded by the user, and generate corresponding device performance tasks in combination with specific algorithms, so as to deploy them to connected intelligent agent devices, such as robots, so that the intelligent agent devices can simulate the user's actions, expressions, and voices in the target video by executing the device performance tasks.

[0056] In this way, by using the task generation interface, the target video that the device needs to imitate can be easily obtained, and the behavior of the object in the target video can be quickly simulated to generate a device performance task that simulates the behavior of the object. This allows robots and other devices to perform the device performance task, making it convenient and quick to complete the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0057] Based on this, in some embodiments, after S140 above, the method may further include: displaying a device performance simulation screen corresponding to the device performance task based on the task generation interface.

[0058] In some examples, the task generation interface may also include a display area for the device performance simulation, used to display the device performance simulation, or to display the device performance simulation in a pop-up window on the task generation interface. For example, Figure 2 As shown, the task generation interface may also include a simulation display area 23, and / or display a pop-up window (not shown in the figure) on the task generation interface to display a device performance simulation screen in the simulation display area 23 and / or the pop-up window.

[0059] For example, to facilitate user viewing and adjustment of simulation results after a device performance task is generated, the platform can simulate the intelligent agent device performing the task and display the corresponding device performance simulation screen when the user selects a preview. For instance, it can... Figure 2 The simulated display area 23 and / or the pop-up interface show the device performance simulation screen obtained after the task is executed.

[0060] Based on this, in some other embodiments, after S140 above, the method may further include: receiving an eighth input for triggering task deployment; and in response to the eighth input, deploying a device performance task to a target device so that the target device performs the device performance task.

[0061] In some examples, the eighth input can be an input that triggers the task deployment process. For example, such as... Figure 2 As shown, the task generation interface may also include a task deployment control 24, and the eighth input can be the user's action of clicking the task deployment control 24. The target device can be an intelligent agent device such as a robot, and the target device can be connected to the intelligent device where the platform is located via wired or wireless means.

[0062] For example, such as Figure 3 As shown, after a device performance task is generated, the user can click the task deployment control 24 to trigger the platform to deploy the device performance task to the connected target device, or display at least one connected device, from which the user can select a target device for task deployment. In this way, after receiving the device performance task, the target device will execute the task according to the triggering conditions, thereby simulating the behavior of objects in the target video and enabling secondary development of the target device's functions.

[0063] Figure 1 The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 3 Extending from the illustrated embodiment Figure 2 The illustrated embodiment will be discussed below in conjunction with... Figure 4 and Figure 3 The task generation interface shown here will be described in detail. Figure 1 The illustrated embodiments and Figure 3 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0064] like Figure 2 As shown, the method also includes the following steps.

[0065] S310, receives a third input for the task generation mode display area in the task generation interface.

[0066] In some examples, the task generation interface may also include a task generation mode display area, for example... Figure 2 The task generation mode display area 25 shown is used to display the currently used task generation mode. Different task generation modes correspond to different algorithm models. Users can select the appropriate task generation mode as needed to simulate the behavior of objects in the target video using the corresponding algorithm model, thereby generating device performance tasks.

[0067] In this embodiment, the third input can be an input used to trigger the display of at least one candidate task generation mode, for example, such as... Figure 4 As shown, the third input can be the user's action of clicking on the task generation mode display area 25.

[0068] S320, in response to a third input, displays at least one task generation mode.

[0069] In some examples, the platform may provide at least one task generation mode for the user to select. For instance, the at least one task generation mode may include a fast mode and an enhanced mode, where the fast mode processes faster and is suitable for simulating more common daily behaviors, while the enhanced mode may be driven by a deep learning model and is suitable for situations that require capturing behavioral details.

[0070] In some specific examples, such as Figure 4 As shown, users can click on the task generation mode display area 25 to display at least one selectable task generation mode, such as quick mode and enhanced mode.

[0071] S330, receiving a fourth input for a target generation mode in at least one task generation mode.

[0072] In some examples, the fourth input can be an input used to select the target generation mode, for example, such as Figure 4 As shown, the fourth input can be the user clicking to select either the fast mode or the enhanced mode.

[0073] For example, if a user wants to generate a device performance task using the quick mode, they can click to select quick mode; if a user wants to generate a device performance task using the enhanced mode, they can click to select enhanced mode.

[0074] S340, in response to the fourth input, displays the target generation mode in the task generation mode display area.

[0075] In some examples, the target generation mode can be displayed in the task generation mode display area after the user selects the target generation mode.

[0076] like Figure 5 As shown, if the user selects the quick mode, the quick mode will be displayed in the task generation mode display area 25; if the user selects the enhanced mode, the enhanced mode will be displayed in the task generation mode display area 25.

[0077] Accordingly, the step of simulating the object behavior in the target video and generating the device performance task corresponding to the object behavior in S140 above may specifically include: simulating the object behavior in the target video based on the target generation mode and generating the device performance task corresponding to the object behavior.

[0078] For example, an algorithm model corresponding to the target generation mode selected by the user can be used to simulate the behavior of an object in the target video, thereby generating a device performance task that can imitate the behavior of that object.

[0079] In this embodiment, through the task generation interface, users can select different task generation modes according to their actual needs. By utilizing the advantages of different task generation modes, it is more convenient and faster to generate corresponding device performance tasks, thereby further improving the efficiency of secondary development of the device and reducing the difficulty of secondary development.

[0080] Figure 1 The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 5 Extending from the illustrated embodiment Figure 2 The illustrated embodiment will be discussed below in conjunction with... Figure 5 The task generation interface shown here will be described in detail. Figure 1 The illustrated embodiments and Figure 5 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0081] like Figure 2 As shown, the method also includes the following steps.

[0082] S510 receives a fifth input for adjusting video configuration parameters.

[0083] In some examples, the fifth input can be an input that triggers adjustments to video configuration parameters. Video configuration parameters can be parameters that adjust the target video, such as video adjustment parameters for static camera mode, and / or video rotation angles.

[0084] In some specific examples, such as Figure 2 As shown, the task generation interface may also include a static camera mode selection control 26 and a video rotation angle selection control 27. The fifth input can be an operation to turn the static camera mode selection control 26 on or off, or an operation to select the desired rotation angle by clicking the video rotation angle selection control 27. The static camera mode can be turned on or off depending on whether the camera in the target video is in motion, while the video rotation angle can be adjusted by rotating the target video depending on whether the object in the target video is in a vertical position.

[0085] S520, in response to the fifth input, determines the target configuration parameters, wherein the target configuration parameters are matched with the content of the target video.

[0086] In some examples, by allowing the user to select video configuration parameters, the platform can determine the target configuration parameters selected by the user. These target configuration parameters can be matched with the content of the target video in order to process the target video and better simulate the behavior of objects in the target video.

[0087] In some specific examples, such as Figure 2 As shown, when the target video is shot with a fixed camera and the subject is facing horizontally, the user can open the static camera mode by clicking the static camera mode selection control 26, and then click the rotate video angle selection control 27 to rotate 90° clockwise or counterclockwise, so that the subject in the target video changes from horizontal to vertical.

[0088] Accordingly, the step of simulating the object behavior in the target video and generating the device performance task corresponding to the object behavior in S140 above may specifically include: simulating the object behavior in the target video based on the target configuration parameters and generating the device performance task corresponding to the object behavior.

[0089] For example, the target video can be preprocessed using user-selected target configuration parameters, and then the object behavior can be simulated based on the preprocessed target video to generate a device performance task that can better mimic the object behavior.

[0090] In this embodiment, by adjusting the video configuration parameters and processing the target video using target configuration parameters that match the target video, the final simulation effect can be improved.

[0091] In other embodiments, the task generation interface may also include a timeline editor, for example, it may be... Figure 2 The timeline editor 28 is shown. Based on this, after S140 above, the method may further include: displaying task breakdown information corresponding to the device performance task in chronological order in the timeline editor.

[0092] In some examples, the timeline editor can be an editor with time as one of the axes, allowing the device performance task to be broken down and displayed along a timeline. For example, as... Figure 6 As shown, after a device performance task is generated, the task breakdown information corresponding to that device performance task can be displayed in the timeline editor 28 with time as the horizontal axis and facial expressions, actions, and audio as the vertical axis, in chronological order. The display effect can be as follows: Figure 7As shown, the timeline editor 28 displays facial expression decomposition information 281, action decomposition information 282, and audio decomposition information 283 corresponding to the facial expressions included in the device performance task.

[0093] Figure 1 The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 7 The embodiments shown are extensions of the embodiments described above. Figure 6 The illustrated embodiment will be discussed below in conjunction with... Figure 8 and Figure 7 The task generation interface shown here will be described in detail. Figure 1 The illustrated embodiments and Figure 7 The differences between the illustrated embodiments and the above embodiments are not repeated here.

[0094] like Figure 6 As shown, after the above steps of displaying the task breakdown information corresponding to the device performance task in chronological order, the method further includes the following steps.

[0095] S710 receives a sixth input for a target time frame in the timeline editor.

[0096] In some examples, the timeline editor can also be used to edit device performance tasks, such as inserting a performance clip at a specific time frame. The sixth input can be used to select the target time frame to be edited in the timeline editor; for example, the sixth input can be the user clicking on the target time frame in the timeline editor.

[0097] S720, in response to the sixth input, determines the target time frame.

[0098] In some examples, the target time frame can be the data frame corresponding to any time selected by the user in the timeline editor.

[0099] In some specific examples, such as Figure 6 As shown, when a user needs to edit a device performance task, they can click to select the target time frame 284 in the timeline editor, and then determine that the target time frame 284 is the time frame to be edited, so that the data corresponding to the time frame can be processed later.

[0100] S730, Display a preset material library, which includes at least one preset behavior material.

[0101] In some examples, the preset content library may be a pre-configured content library for intelligent agent devices, containing one or more behavioral content. The behavioral content may include at least one of audio content, motion content, and facial expression content. Specifically, motion content may include body movement content and displacement movement content. Body movement content may include, for example, content representing body movements such as clapping, waving, or stomping, while displacement movement content may include content representing device displacement movements such as taking three steps forward, two steps backward, five steps to the left, or four steps to the right.

[0102] In some specific examples, such as Figure 6 As shown, users can directly display the preset material library in a pop-up window after selecting the target time frame 284, or they can click on the material library to bring up a control, which will then display the material selection interface corresponding to the preset material library. For example, when a user clicks on the audio material library to bring up control 31, the audio material selection interface corresponding to the audio material library will be displayed; when a user clicks on the action material library to bring up control 32, the action material selection interface 30 corresponding to the action material library will be displayed; and when a user clicks on the emoticon material library to bring up control 33, the emoticon material selection interface corresponding to the emoticon material library will be displayed.

[0103] S740 receives the seventh input for the target behavior material in the preset material library.

[0104] In some examples, the seventh input can be an input used to select the target behavior material to be inserted from a preset material library. For example, the seventh input can be the user's action of clicking to select the target behavior material from the preset material library.

[0105] S750, in response to the seventh input, inserts target behavior footage into the device performance task based on the target time frame.

[0106] In some examples, once a user selects a target action clip, that clip can be inserted at the time position corresponding to the target time frame within the device performance task. In other examples, once a user selects a target action clip, it can also be inserted at the time position corresponding to an idle time frame following the target time frame.

[0107] In some specific examples, such as Figure 8 As shown, when a user selects clapping action material 301 from the action material selection interface 30, it can be displayed at the time position corresponding to the idle time frame after the target time frame 284, for example, in... Figure 6 Insert the clapping action footage 301 at the time position corresponding to the idle time frame 285 shown. Alternatively, it can be done directly in... Figure 2The clapping action footage 301 is inserted at the time position corresponding to the target time frame 284 shown, which is not limited here. In this way, the device performance task can be customized and edited through the timeline editor.

[0108] In this embodiment, by using the timeline editor to edit the device performance task, the generated device performance task can be modified quickly and conveniently. This can improve the flexibility of secondary development of intelligent agent devices, further improve the efficiency of secondary development, and reduce the difficulty of secondary development.

[0109] Based on the above embodiments, a specific example is given below to better illustrate the task generation method of this application.

[0110] In such Figure 4 In the task generation interface shown, users can upload target videos by dragging and dropping video files to the video acquisition control 21, or by clicking the video acquisition control 21 to find and select video files locally, or by clicking the video acquisition control 21 to trigger the connected camera to record video by capturing sound, in order to acquire the target video.

[0111] After acquiring the target video, users can also click on task generation mode display area 25 to select the desired mode, such as... Figure 2 The options are Quick Mode and Enhanced Mode. If the user selects Quick Mode, it will be displayed in the Task Generation Mode display area 25, and the device performance task will be generated in Quick Mode after the user clicks the Task Generation Control 22. If the user selects Enhanced Mode, it will be displayed in the Task Generation Mode display area 25, and the device performance task will be generated in Enhanced Mode after the user clicks the Task Generation Control 22.

[0112] Additionally, users can click the still camera mode selection control 26 to turn the still camera mode on or off based on the camera's movement in the target video. For example, if the target video is taken with a moving camera, the user can choose to turn off the still camera mode; if the target video is taken with a static camera, the user can choose to turn on the still camera mode.

[0113] Users can also select the appropriate rotation angle by clicking the rotation video angle selection control 27, based on the orientation of the subject in the target video. For example, if the subject in the target video is in a horizontal position, the user can choose to rotate the target video 90° clockwise or counterclockwise to make the subject vertical.

[0114] By turning the static camera mode on or off, and adjusting the video rotation angle, the final behavior simulation effect can be improved.

[0115] After uploading the target video and adjusting various parameters, the user can trigger the task generation process by clicking the task generation control 22. That is, using the set static camera mode, video rotation angle and other parameters, the user selects the fast mode or enhanced mode to transcribe the target video, simulate the behavior of objects in the target video, and generate a device performance task for the intelligent agent device to execute.

[0116] Based on this, after generating the device performance task, the device performance simulation screen corresponding to the device performance task can also be displayed on, for example... Figure 2 The simulated display area 23 shown allows users to preview the simulated behavior of the device.

[0117] In addition, users can also click on things like Figure 6 The task deployment control 24 shown in the figure sends the generated device performance task to the connected smart agent device so that the smart agent device can execute the device performance task and realize the simulation of the behavior of the object in the target video.

[0118] Furthermore, after generating a device performance task, the task decomposition information corresponding to that task can also be obtained, such as... Figure 6 The facial expression breakdown information 281, motion breakdown information 282, and audio breakdown information 283 shown are displayed in the timeline editor 28, so that the user can edit the device performance task based on the timeline editor 28.

[0119] For example, such as Figure 8 As shown, the user can click to select the target time frame 284 in the timeline editor 28, and click the pop-up control 32 of the motion material library to display the corresponding motion material selection interface 30. The user can then select the clapping action material 301 in this interface 30, thus inserting it into the corresponding position of the target time frame 284. In other examples, to avoid motion conflicts, the user-selected clapping action material 301 can also be inserted after the target time frame 284, such as... Figures 1 to 8 The location corresponding to the idle time frame 285 shown.

[0120] In this way, by leveraging the aforementioned task-generation interface-based interaction process, the goal of secondary development of intelligent agent device behavior can be achieved in an extremely convenient and rapid manner. This process greatly simplifies the originally complex and cumbersome secondary development operations, freeing developers from expending significant effort to overcome various difficulties and challenges when dealing with secondary development of intelligent agent device behavior, thus effectively reducing the difficulty level of secondary development. Simultaneously, this task-generation interface-based interaction method can also significantly improve the efficiency of secondary development, allowing developers to complete development tasks faster and save more time and energy for other related work or further optimization of development results.

[0121] The above text combined Figure 8 The following describes in detail the embodiments of the task generation method of this application, in conjunction with... Figure 9 This application provides a detailed description of embodiments of the task generation apparatus. It should be understood that the descriptions of the task generation method embodiments correspond to the descriptions of the task generation apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0122] Figure 9 The diagram shown is a structural schematic of a task generation apparatus provided in an embodiment of this application. Figure 10 As shown, the task generation apparatus 90 provided in this application embodiment includes: The first receiving module 910 is used to receive a first input for triggering video acquisition based on the task generation interface; The video acquisition module 920 is used to acquire the target video in response to the first input; The second receiving module 930 is used to receive a second input for triggering task generation based on the task generation interface; The task generation module 940 is used to respond to the second input, simulate the behavior of an object in the target video, and generate a device performance task corresponding to the object behavior.

[0123] In one embodiment of this application, the task generation device 90 further includes: a third receiving module, configured to receive a third input to a task generation mode display area in the task generation interface; a first display module, configured to display at least one task generation mode in response to the third input; a fourth receiving module, configured to receive a fourth input to a target generation mode among the at least one task generation mode; and a second display module, configured to display the target generation mode in the task generation mode display area in response to the fourth input. Correspondingly, the task generation module 940 is further configured to simulate object behavior in a target video based on the target generation mode, and generate a device performance task corresponding to the object behavior.

[0124] In one embodiment of this application, the task generation device 90 further includes: a fifth receiving module, configured to receive a fifth input for adjusting video configuration parameters; and a parameter determining module, configured to determine target configuration parameters in response to the fifth input, wherein the target configuration parameters match the content of the target video. Correspondingly, the task generation module 940 is further configured to simulate object behavior in the target video based on the target configuration parameters, and generate a device performance task corresponding to the object behavior.

[0125] In one embodiment of this application, the task generation device 90 further includes: a third display module, used to display task breakdown information corresponding to the device performance task in a timeline editor in chronological order.

[0126] In one embodiment of this application, the task generation device 90 further includes: a sixth receiving module for receiving a sixth input for a target time frame in a timeline editor; a target determination module for determining a target time frame in response to the sixth input; a fourth display module for displaying a preset material library, the preset material library including at least one preset behavior material; a seventh receiving module for receiving a seventh input for a target behavior material in the preset material library; and a material insertion module for inserting the target behavior material into the device performance task based on the target time frame in response to the seventh input.

[0127] In one embodiment of this application, the behavioral material includes at least one of audio material, motion material, and facial expression material.

[0128] In one embodiment of this application, the task generation device 90 further includes: a fifth display module, used to display a device performance simulation screen corresponding to the device performance task based on the task generation interface.

[0129] In one embodiment of this application, the task generation device 90 further includes: an eighth receiving module, configured to receive an eighth input for triggering task deployment; and a task deployment module, configured to deploy a device performance task to a target device in response to the eighth input, so that the target device can perform the device performance task.

[0130] In this way, by using the task generation interface, the target video that the device needs to imitate can be easily obtained, and the behavior of the object in the target video can be quickly simulated to generate a device performance task that simulates the behavior of the object. This allows intelligent devices such as robots to perform the device performance task, making it convenient and quick to complete the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0131] Below, for reference Figure 10 This describes an electronic device according to embodiments of the present application. Figure 10 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0132] like Figure 10 As shown, the electronic device 100 includes one or more processors 1001 and memory 1002.

[0133] The processor 1001 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0134] The memory 1002 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1001 may execute the program instructions to implement the task generation methods of the various embodiments of this application described above and / or other desired functions. The computer-readable storage medium may also store various contents such as data acquisition task templates, configuration information, instance tasks, etc.

[0135] In one example, the electronic device 100 may also include an input device 1003 and an output device 1004, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0136] The input device 1003 may include, for example, a keyboard, a mouse, etc.

[0137] The output device 1004 can output various information to the outside, including data acquisition task templates, configuration information, instance tasks, etc. The output device 1004 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0138] Of course, for the sake of simplicity, ​ Only some of the components of the electronic device 100 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 100 may include any other suitable components depending on the specific application.

[0139] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the task generation methods according to various embodiments of this application described above.

[0140] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0141] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the task generation methods according to various embodiments of this application described above.

[0142] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0143] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0144] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0145] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0146] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0147] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A task generation method, characterized in that, include: Based on the task generation interface, receive the first input used to trigger video acquisition; In response to the first input, the target video is acquired; Based on the task generation interface, a second input for triggering task generation is received; In response to the second input, the device performance task corresponding to the object behavior in the target video is generated by simulating the object behavior.

2. The method according to claim 1, characterized in that, Also includes: Receive a third input for the task generation mode display area in the task generation interface; In response to the third input, at least one task generation mode is displayed; Receive a fourth input for a target generation mode in the at least one task generation mode; In response to the fourth input, the target generation mode is displayed in the task generation mode display area; The step of simulating object behavior in the target video and generating a device performance task corresponding to the object behavior includes: Based on the target generation mode, simulate the object behavior in the target video to generate a device performance task corresponding to the object behavior.

3. The method according to claim 1, characterized in that, Also includes: Receive a fifth input for adjusting video configuration parameters; In response to the fifth input, target configuration parameters are determined, wherein the target configuration parameters match the content of the target video; The step of simulating object behavior in the target video and generating a device performance task corresponding to the object behavior includes: Based on the target configuration parameters, simulate the object behavior in the target video to generate a device performance task corresponding to the object behavior.

4. The method according to claim 1, characterized in that, The task generation interface includes a timeline editor. After generating the device performance task corresponding to the object behavior, the method further includes: In the timeline editor, task breakdown information corresponding to the device performance task is displayed in chronological order.

5. The method according to claim 4, characterized in that, After displaying the task breakdown information corresponding to the device performance task in chronological order, the method further includes: Receive a sixth input for the target time frame in the timeline editor; In response to the sixth input, the target time frame is determined; Display a preset material library, which includes at least one preset behavior material; Receive the seventh input for the target behavior material in the preset material library; In response to the seventh input, the target behavior material is inserted into the device performance task based on the target time frame.

6. The method according to claim 5, characterized in that, The behavioral materials include at least one of audio materials, action materials, and facial expression materials.

7. The method according to claim 1, characterized in that, After generating the device performance task corresponding to the object behavior, the method further includes: Based on the task generation interface, a simulated device performance screen corresponding to the device performance task is displayed.

8. The method according to claim 1, characterized in that, After generating the device performance task corresponding to the object behavior, the method further includes: Receive the eighth input used to trigger task deployment; In response to the eighth input, the device performance task is deployed to the target device so that the target device performs the device performance task.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the task generation method according to any one of claims 1 to 8.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the task generation method according to any one of claims 1 to 8.

11. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the task generation method according to any one of claims 1 to 8.

Images