Task generation method, storage medium, electronic equipment and program product

Abstract

The invention provides a task generation method, a storage medium, electronic equipment and a program product, and relates to the technical field of robots. The method comprises the following steps: receiving a first input for triggering audio acquisition based on a task generation interface; obtaining a target audio in response to the first input; receiving a second input for triggering task generation based on the task generation interface; and in response to the second input, generating an equipment performance task corresponding to the audio content according to the audio content of the target audio. According to the embodiment of the invention, the secondary development difficulty of intelligent agent equipment such as a robot can be reduced, and the secondary development efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, specifically to a task generation method, storage medium, electronic device, and program product. Background Technology

[0002] As robotics technology expands from structured factories to diverse scenarios, the need for customized secondary development of its behavior is becoming increasingly urgent. This process aims to endow general-purpose robots with specialized skills adapted to specific tasks, becoming a key bridge connecting hardware platforms and complex applications.

[0003] Currently, secondary development of robot behavior requires a series of cumbersome processes, such as offline motion capture and online model building, which makes secondary development difficult and inefficient. Summary of the Invention

[0004] In view of this, embodiments of this application provide a task generation method, a storage medium, an electronic device, and a program product.

[0005] In a first aspect, one embodiment of this application provides a task generation method, the method comprising: receiving a first input for triggering audio acquisition based on a task generation interface; acquiring target audio in response to the first input; receiving a second input for triggering task generation based on the task generation interface; and generating a device performance task corresponding to the audio content based on the audio content of the target audio in response to the second input.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: receiving a third input for the target text content; and in response to the third input, obtaining input text corresponding to the target text content. Correspondingly, obtaining the target audio includes: generating corresponding audio based on the input text to obtain the target audio.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, before generating corresponding audio based on the input text to obtain the target audio, the method further includes: receiving a fourth input for setting audio generation parameters; and in response to the fourth input, obtaining the target audio generation parameters. Accordingly, generating corresponding audio based on the input text to obtain the target audio includes: generating audio corresponding to the input text based on the target audio generation parameters to obtain the target audio.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the audio generation parameters include at least one of timbre parameters and emotion parameters.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, after acquiring the target audio, the method further includes: receiving a fifth input for triggering audio processing; in response to the fifth input, recognizing the semantic content of the target audio and generating recognized text corresponding to the target audio; and displaying the recognized text.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, after displaying the identified text, the method further includes: receiving a sixth input for target text in the identified text; responding to the sixth input, displaying a preset material library, wherein the preset material library includes at least one preset behavioral material; receiving a seventh input for target behavioral material in the preset material library; and responding to the seventh input, inserting the target behavioral material at the position corresponding to the target text. Correspondingly, generating a device performance task corresponding to the audio content of the target audio includes: generating a device performance task based on the target audio and the behavioral material inserted in the identified text.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: receiving an eighth input for triggering the generation of background behavior; and in response to the eighth input, generating background behavior corresponding to the target audio based on the audio content of the target audio. Correspondingly, generating a device performance task corresponding to the audio content of the target audio includes: generating a device performance task based on the audio content of the target audio and the background behavior.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, generating a device performance task corresponding to the audio content of the target audio includes: determining behavioral materials matching the audio content from a preset material library based on the audio content of the target audio; and generating a device performance task based on the target audio and the behavioral materials matching the audio content.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, behavioral material includes at least one of audio material, motion material, and facial expression material.

[0014] Secondly, one embodiment of this application provides a task generation device, which includes: a first receiving module, configured to receive a first input for triggering audio acquisition based on a task generation interface; an audio acquisition module, configured to acquire target audio in response to the first input; a second receiving module, configured to receive a second input for triggering task generation based on the task generation interface; and a task generation module, configured to generate a device performance task corresponding to the audio content based on the audio content of the target audio in response to the second input.

[0015] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a third receiving module, configured to receive a third input for the target text content; and a first acquiring module, configured to acquire input text corresponding to the target text content in response to the third input. Correspondingly, the audio acquiring module is further configured to: generate corresponding audio based on the input text to obtain the target audio.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a fourth receiving module, configured to receive a fourth input for setting audio generation parameters; and a second acquiring module, configured to acquire target audio generation parameters in response to the fourth input. Correspondingly, the audio acquiring module is also configured to: generate audio corresponding to the input text based on the target audio generation parameters, thereby obtaining the target audio.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the audio generation parameters include at least one of timbre parameters and emotion parameters.

[0018] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a fifth receiving module, configured to receive a fifth input for triggering audio processing; a semantic recognition module, configured to, in response to the fifth input, recognize the semantic content of the target audio and generate recognized text corresponding to the target audio; and a first display module, configured to display the recognized text.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: a sixth receiving module, configured to receive a sixth input for the target text in the recognized text; a second display module, configured to display a preset material library in response to the sixth input, wherein the preset material library includes at least one preset behavioral material; a seventh receiving module, configured to receive a seventh input for the target behavioral material in the preset material library; and a material insertion module, configured to insert the target behavioral material at the position corresponding to the target text in response to the seventh input. Correspondingly, the task generation module is also configured to: generate a device performance task based on the target audio and the behavioral material inserted into the recognized text.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the task generation device further includes: an eighth receiving module, configured to receive an eighth input for triggering the generation of background behavior; and a behavior generation module, configured to, in response to the eighth input, generate background behavior corresponding to the target audio based on the audio content of the target audio. Accordingly, the task generation module is further configured to: generate a device performance task based on the audio content of the target audio and the background behavior.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the task generation module is further configured to: determine behavioral materials matching the audio content from a preset material library based on the audio content of the target audio; and generate a device performance task based on the target audio and the behavioral materials matching the audio content.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, behavioral material includes at least one of audio material, motion material, and facial expression material.

[0023] Thirdly, one embodiment of this application provides a computer-readable storage medium storing a computer program for performing the task generation method described in the first aspect.

[0024] Fourthly, one embodiment of this application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute the task generation method described in the first aspect.

[0025] Fifthly, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the task generation method described in the first aspect.

[0026] In this application, the target audio that needs to be performed by the device is conveniently obtained by using the task generation interface, and the device performance task that is adapted to the audio content of the target audio is quickly generated so that intelligent devices such as robots can perform the device performance task. This facilitates and speeds up the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development. Attached Figure Description

[0027] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 The diagram shown is a flowchart of a task generation method provided in an embodiment of this application.

[0029] Figure 2 The diagram shown is a schematic of a task generation interface provided in an embodiment of this application.

[0030] Figure 3The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0031] Figure 4 The diagram shown is a schematic of a task generation interface provided in another embodiment of this application.

[0032] Figure 5 The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0033] Figure 6 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0034] Figure 7 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0035] Figure 8 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0036] Figure 9 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0037] Figure 10 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0038] Figure 11 The diagram shown is a flowchart of a task generation method provided in another embodiment of this application.

[0039] Figure 12 The diagram shown is a schematic diagram of a task generation interface provided in another embodiment of this application.

[0040] Figure 13 The diagram shown is a structural schematic of a task generation device provided in an embodiment of this application.

[0041] Figure 14 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In some instances, methods and means well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0045] Furthermore, the terms “first,” “second,” “third,” and “fourth” are used only for distinguishing descriptions and should not be interpreted as indicating or implying relative importance.

[0046] With the continuous advancement and development of robotics technology, more and more industries and fields are beginning to utilize robots to replace humans in performing specific tasks. Against this backdrop, in order to enable robots to better adapt to various complex and diverse application scenarios, it is often necessary to conduct secondary development and optimization of their behavior patterns. However, currently, the entire process of secondary development for the behavior of robots and other intelligent devices is particularly cumbersome and complex. Specifically, this process typically involves a series of meticulous and time-consuming operations, such as offline motion capture and online model building. These complex processes not only significantly increase the technical difficulty of secondary development but also significantly reduce overall development efficiency, posing certain obstacles and challenges to practical applications.

[0047] This application aims to construct a comprehensive and efficient one-stop platform, enabling users to easily perform secondary development of the behavior of intelligent agents such as robots by leveraging the various functional modules provided by the task generation interface. In the traditional model, users often need to go through a series of complex and cumbersome procedures to perform such secondary development, which not only consumes a lot of time and energy but also places high demands on the user's skill level. However, with the platform provided by this application, users can smoothly complete the secondary development process of intelligent agent behavior through intuitive and simple interface interaction, greatly simplifying the operation steps, lowering the technical threshold for secondary development, and significantly improving the overall efficiency of secondary development.

[0048] Specifically, the solution provided in this application can utilize the acquired audio and, through simple human-computer interaction, enable intelligent agents to interpret the behavior of audio content, thereby achieving the purpose of secondary development of the behavior of intelligent agents.

[0049] The following is combined Figures 1 to 12The task generation method provided in this application is described in detail.

[0050] Figure 1 The diagram shown is a flowchart illustrating a task generation method according to an embodiment of this application. This method can be applied to electronic devices; exemplarily, electronic devices may include smart devices such as computers and mobile phones. Figure 1 As shown, the method may include the following steps.

[0051] S110, based on the task generation interface, receives the first input used to trigger audio acquisition.

[0052] In some examples, the task generation interface can be an interface set up based on a preset secondary development platform. The task generation interface may include multiple functional controls, such as controls for triggering audio acquisition and controls for triggering task generation. Users can easily perform secondary development on the device behavior of intelligent devices such as robots by operating the functional controls in the task generation interface.

[0053] In some specific examples, the task generation interface could be, for example, like this: Figure 2 The interface shown includes an audio acquisition control 201 for triggering the acquisition of existing audio files and a task generation control 202 for triggering task generation.

[0054] In some examples, the first input can be an input that triggers the acquisition of audio, including but not limited to clicks, drags, etc. For example, Figure 2 As shown, the first input can be the user dragging and dropping the target audio to the audio acquisition control 201, or the user clicking the audio acquisition control 201 to select the corresponding audio file from local files, or the user clicking the audio acquisition control 201 to trigger the connected microphone to record audio to obtain the target audio. Alternatively, the first input could also be the user entering target text content in the task generation interface and then clicking the audio synthesis control to trigger the synthesis of the target audio.

[0055] S120, in response to the first input, acquires the target audio.

[0056] In some examples, the target audio can be audio selected or recorded by the user, or audio synthesized based on the target text content entered by the user; no limitation is made here.

[0057] like Figure 2As shown, in some specific examples, users can drag and drop the target audio file into the audio acquisition control 201, and the platform can then obtain the target audio through the task generation interface. In other specific examples, users can click on a specific area of ​​the audio acquisition control 201 to trigger the display of a local file browsing window. Through this window, users can select the desired target audio file from the corresponding audio storage path, and then obtain the target audio from that path after confirming the selection. In still other specific examples, users can also click on other specific areas of the audio acquisition control 201 to trigger the activation of the microphone connected to the platform, and then record audio through the microphone to obtain the target audio.

[0058] S130, based on the task generation interface, receives a second input to trigger task generation.

[0059] In some examples, the second input can be an input that triggers the generation of a task that the intelligent agent device can perform, including but not limited to actions such as clicking. For example, Figure 2 As shown, the second input can be the user's action of clicking the task generation control 202.

[0060] S140, in response to the second input, generates a device performance task corresponding to the audio content based on the audio content of the target audio.

[0061] In some examples, a device performance task can be a task that the platform can perform based on the audio content of a target audio file, generating a task that the intelligent agent device can execute. This behavioral performance includes, but is not limited to, action performance, facial expression performance, and voice performance.

[0062] like Figure 2 As shown, in some specific examples, users can click on the task generation control 202 to trigger the platform to perform a device performance task adapted to the target audio uploaded by the user, based on the audio content such as sound in the target audio, and combined with a specific algorithm to generate a device performance task, which can be deployed to the connected intelligent agent device, such as a robot, so that the intelligent agent device can perform the performance behavior by executing the device performance task.

[0063] In this way, by using the task generation interface, the target audio that needs to be performed by the device can be easily obtained, and a device performance task that is adapted to the audio content of the target audio can be quickly generated, so that intelligent devices such as robots can perform the device performance task. This facilitates the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0064] Based on this, in some embodiments, after S140 above, the method may further include: displaying a device performance simulation screen corresponding to the device performance task based on the task generation interface.

[0065] In some examples, the task generation interface may also include a display area for the device performance simulation, used to display the device performance simulation, or to display the device performance simulation in a pop-up window on the task generation interface. For example, Figure 2 As shown, the task generation interface may also include a simulation display area 203, and / or display a pop-up window (not shown in the figure) on the task generation interface to display a device performance simulation screen in the simulation display area 203 and / or the pop-up window.

[0066] For example, to facilitate user viewing and adjustment of simulation results after a device performance task is generated, the platform can simulate the intelligent agent device performing the task and display the corresponding device performance simulation screen when the user selects a preview. For instance, it can... Figure 2 The simulated display area 203 and / or the pop-up interface show the device performance simulation screen obtained after the task is executed.

[0067] Based on this, in some other embodiments, after S140 above, the method may further include: receiving a ninth input for triggering task deployment; and in response to the ninth input, deploying a device performance task to a target device so that the target device performs the device performance task.

[0068] In some examples, the ninth input can be an input that triggers the task deployment process. For example, such as... Figure 2 As shown, the task generation interface may also include a task deployment control 204, and the ninth input can be the user's action of clicking the task deployment control 204. The target device can be an intelligent agent device such as a robot, and the target device can be connected to the intelligent device where the platform is located via wired or wireless means.

[0069] For example, such as Figure 2 As shown, after a device performance task is generated, the user can click the task deployment control 204 to trigger the platform to deploy the device performance task to the connected target device, or display at least one connected device, from which the user can select the target device for task deployment. In this way, after receiving the device performance task, the target device will execute the task according to the triggering conditions, thereby realizing the behavioral interpretation of the target audio and enabling secondary development of the target device's functions.

[0070] Figure 3 The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 1 Extending from the illustrated embodiment Figure 3 The illustrated embodiment will be discussed below in conjunction with... Figure 2 and Figure 4 The task generation interface shown here will be described in detail. Figure 3 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0071] like Figure 3 As shown, the method also includes the following steps.

[0072] S310, receives a third input for the target text content.

[0073] In some examples, such as Figure 2 As shown, the task generation interface may also include an audio acquisition method switching control 205. Users can click this control to switch between uploading an audio file and synthesizing audio based on input text. Specifically, if the user chooses to acquire the target audio by uploading an audio file, the following will be displayed: Figure 2 The task generation interface shown; if the user selects to synthesize the target audio based on the input text, the interface will display as follows: Figure 4 The task generation interface shown may include a text input control 206 for inputting text content and an audio synthesis control 207 for triggering audio synthesis.

[0074] In this embodiment, the third input can be the user's input of the target text content. In some specific examples, such as... Figure 4 As shown, the third input can be the user's text input operation in the text input control 206, such as the operation of inputting the text "Hello everyone, I am a robot, it is nice to meet you all" in the text input control 206.

[0075] S320, in response to the third input, obtains the input text corresponding to the target text content.

[0076] In some examples, such as Figure 4 As shown, when a user inputs the target text content into the text input control 206, the platform can obtain the input text corresponding to the target text content through the text input control 206.

[0077] Accordingly, the step of obtaining the target audio in S120 above may specifically include: generating corresponding audio based on the input text to obtain the target audio.

[0078] For example, such as Figure 4As shown, the first input can be the user entering the target text content in the text input control 206 and then clicking the audio synthesis control 207 to trigger the synthesis of the target audio. In response to this first input, audio synthesis can be performed based on the obtained input text to obtain the target audio.

[0079] Based on this, in some embodiments, before generating the corresponding audio based on the input text to obtain the target audio, the method may further include: receiving a fourth input for setting audio generation parameters; and in response to the fourth input, obtaining the target audio generation parameters.

[0080] In some examples, the fourth input can be an input that triggers the setting of audio generation parameters. Audio generation parameters can be adjustable parameters during the audio generation process, and may include at least one of timbre parameters and emotion parameters. Timbre parameters may include, for example, female timbre, male timbre, etc., and emotion parameters may include, for example, happy, sad, angry, etc.

[0081] Accordingly, the above process of generating corresponding audio based on input text to obtain target audio can specifically include: generating audio corresponding to input text based on target audio generation parameters to obtain target audio.

[0082] In some examples, after the user selects the target audio generation parameters, the target audio with the characteristics of those parameters and whose audio content matches the input text can be generated by referring to those parameters.

[0083] In some specific examples, such as Figure 4 The task generation interface shown may also include a timbre parameter selection control 208 and an emotion parameter selection control 209. The fourth input can be the user clicking the timbre parameter selection control 208 and the emotion parameter selection control 209 to select timbre and emotion parameters. For example, the user can click the timbre parameter selection control 208 to select a female timbre and click the emotion parameter selection control 209 to select a happy timbre, thereby obtaining the timbre parameters corresponding to the female timbre and the emotion parameters corresponding to the happy emotion as the target audio generation parameters. When the user clicks the audio synthesis control 207, it will trigger the generation of audio with a female timbre, a happy emotion, and the target text content based on the target audio generation parameters, thus obtaining the target audio.

[0084] In this embodiment, by generating target audio using text content input by the user, and then generating corresponding device performance tasks based on the target audio, a variety of acquisition paths can be provided for audio acquisition, making secondary development of intelligent agent devices more flexible, thereby further reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0085] Figure 5The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 1 Extending from the illustrated embodiment Figure 5 The illustrated embodiment will be discussed below in conjunction with... Figure 6 and Figure 7 The task generation interface shown here will be described in detail. Figure 5 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0086] like Figure 5 As shown, after S120 above, the method further includes the following steps.

[0087] S510 receives a fifth input used to trigger audio processing.

[0088] In some examples, the fifth input can be an input that triggers entry into an audio processing procedure, which includes, but is not limited to, a semantic content recognition procedure.

[0089] In some specific examples, after obtaining the target audio, the task generation interface may be displayed as follows: Figure 6 As shown, the interface may include an audio processing control 210 for triggering audio processing. The fifth input may be, for example, the user clicking the audio processing control 210. By clicking the audio processing control 210, a semantic content recognition and other processing procedures for the target audio can be triggered.

[0090] S520, in response to the fifth input, recognizes the semantic content of the target audio and generates recognized text corresponding to the target audio.

[0091] In some examples, semantic recognition can be performed on the target audio to obtain the corresponding recognized text, which can then be displayed in a specific area for subsequent behavior editing.

[0092] S530 displays the recognized text.

[0093] In some examples, after obtaining the recognized text corresponding to the target audio, the recognized text can be displayed in a specific area of ​​the task generation interface. For example, in some specific examples, such as Figure 7 As shown, if the recognized text obtained by recognizing the target audio is "Hello everyone, I am a robot, it is nice to meet you all", then the recognized text can be displayed in the editable area 211 below the acquired target audio.

[0094] Based on this, in some other embodiments of this example, after S530 described above, the method may further include: receiving a sixth input for the target text in the identified text; in response to the sixth input, displaying a preset material library, wherein the preset material library includes at least one preset behavioral material; receiving a seventh input for the target behavioral material in the preset material library; and in response to the seventh input, inserting the target behavioral material at the position corresponding to the target text. Accordingly, the step of generating a device performance task corresponding to the audio content based on the audio content of the target audio in S140 described above may specifically include: generating a device performance task based on the target audio and the behavioral material inserted in the identified text.

[0095] In some examples, the identified text can be editable text, and the sixth input can be an input that selects the target text to be edited in the identified text, where the target text can be any one or a segment of characters in the identified text.

[0096] In addition, in this embodiment and other embodiments, the preset material library can be a material library pre-set for the intelligent agent device, containing one or more behavioral materials. The behavioral materials can include at least one of audio materials, motion materials, and facial expression materials. Specifically, motion materials can include motion materials representing body movements and motion materials representing displacement movements. For example, motion materials representing body movements can include clapping, waving, stomping, etc., while motion materials representing displacement movements can include materials representing device displacement movements such as taking three steps forward, two steps backward, five steps to the left, four steps to the right, etc.

[0097] For example, the preset material library can be displayed directly after receiving the sixth input, or it can be triggered by the user through a preset control. In some specific examples, the task generation interface can also be equipped with controls for triggering the display of the preset material library, for example, such as... Figure 7 As shown, the task generation interface can also be equipped with an emoticon material library trigger control 212 for triggering the display of the emoticon material library, and an action material library trigger control 213 for triggering the display of the action material library. Users can trigger the display of the emoticon material library by clicking the emoticon material library trigger control 212, and trigger the display of the action material library by clicking the action material library trigger control 213.

[0098] In some examples, the seventh input can be an input that selects the target behavior material to be inserted from the preset material library, where the target behavior material can be any one or more behavior materials from the preset material library.

[0099] Based on this, taking the addition of emoji materials as an example, such as Figure 8As shown, the sixth input can be an operation where the user selects the word "good" in the recognized text displayed within the editable area 211. Then, the user can trigger the display of the expression material library display interface 214 by clicking the expression material library trigger control 212. The seventh input can be an operation where the user clicks to select the happy behavior material 215 and clicks the confirmation selection button. The platform can add the selected happy behavior material 215 by the user to the position corresponding to the word "good". Furthermore, after generating the corresponding device performance task based on the inserted behavior material, when the intelligent device executes this device performance task, it can show a happy expression when outputting the sound of "good".

[0100] In addition, in order to make the intelligent device appear more natural when performing the behaviors corresponding to the device performance task, some background behaviors can be added to the intelligent device before generating the device performance task, so that the intelligent device can perform subtle behavior changes according to the rhythm, tempo, etc. of the target audio.

[0101] Based on this, in some other implementation manners of this embodiment, the method may further include: receiving an eighth input for triggering the generation of background behaviors; in response to the eighth input, generating background behaviors corresponding to the target audio according to the audio content of the target audio. Correspondingly, in the above S140, the step of generating a device performance task corresponding to the audio content according to the audio content of the target audio may specifically include: generating a device performance task according to the audio content of the target audio and the background behaviors.

[0102] In some examples, the eighth input can be an input for triggering the background behavior generation process. Among them, the background behavior can be a subtle behavior that moves rhythmically according to the rhythm, tempo, etc. of the target audio, such as slightly shaking the body when on standby, occasionally raising the hand when speaking, and other behaviors.

[0103] In some specific examples, as Figure 6 shown, when the user clicks the audio processing control 210, in addition to triggering the process of semantic recognition of the target audio, a background behavior generation prompt interface 216 as <## Figure 9 shown can also be pop-up displayed. The eighth input can be an operation where the user clicks the confirmation generation control 217 in the background behavior generation prompt interface 216. In this way, the platform can generate corresponding background behaviors according to the rhythm, tempo, etc. of the target audio, and then, on the basis of this background behavior, generate a device performance task including this background behavior. In addition, the user can also cancel the generation of the corresponding background behavior by clicking the cancel generation control 218 in the background behavior generation prompt interface 216, and then display a task generation interface as Figure 7 shown.

[0104] In this embodiment, by recognizing the semantic content of the target audio, and then performing custom editing of behavioral materials and adding background behaviors based on the recognized text, the flexibility of device performance task generation can be improved, thereby increasing the flexibility of secondary development.

[0105] Based on the above embodiments, in addition to generating device performance tasks through custom editing, users can also automatically match corresponding behavioral materials to the target audio through the platform to achieve intelligent arrangement. Therefore, S140 specifically includes: determining behavioral materials matching the audio content from a preset material library based on the audio content of the target audio; and generating a device performance task based on the target audio and the behavioral materials matching the audio content.

[0106] In some examples, such as Figure 7 As shown, after semantic recognition of the target audio and displaying the recognized text, a smart orchestration trigger control 219 can be displayed in the task generation interface. Correspondingly, the second input can also be the user clicking the smart orchestration trigger control 219 to trigger the smart orchestration processing. Specifically, the smart orchestration processing can involve retrieving behavioral materials matching the audio content of the target audio from a preset material library based on a preset algorithm model. For example, if the audio content contains greetings, waving or other behavioral materials can be inserted at the corresponding position; if the audio content contains expressions of happiness, happy or other facial expressions can be inserted at the corresponding position. Thus, after matching the audio content of the target audio and inserting the corresponding behavioral materials, a device performance task containing the target audio and the inserted behavioral materials can be generated.

[0107] Thus, through the aforementioned intelligent choreography function, the performance content of the device can be automatically choreographically arranged without user editing, thereby further improving the efficiency of secondary development of intelligent devices.

[0108] In addition, based on the above embodiments, the task generation interface may also include a timeline editor, for example, it could be... Figure 2 The timeline editor 221 is shown. Based on this, after S140 above, the method may further include: displaying task breakdown information corresponding to the device performance task in chronological order in the timeline editor.

[0109] In some examples, the timeline editor can be an editor with time as one of the axes, allowing the device performance task to be broken down and displayed along a timeline. For example, as... Figure 2As shown, after a device performance task is generated, the task breakdown information corresponding to that device performance task can be displayed in the timeline editor 221 with time as the horizontal axis and facial expressions, actions, and audio as the vertical axis, in chronological order. The display effect can be as follows: Figure 10 As shown, the timeline editor 221 displays facial expression decomposition information 2211, motion decomposition information 2212, and audio decomposition information 2213 corresponding to the facial expressions, motions, and audios included in the device performance task.

[0110] Figure 11 The diagram shown is a flowchart illustrating a task generation method provided in another embodiment of this application. Figure 1 The embodiments shown are extensions of the embodiments described above. Figure 11 The illustrated embodiment will be discussed below in conjunction with... Figure 10 and Figure 12 The task generation interface shown here will be described in detail. Figure 11 The illustrated embodiments and Figure 1 The differences between the illustrated embodiments and the above embodiments are not repeated here.

[0111] like Figure 11 As shown, after the above steps of displaying the task breakdown information corresponding to the device performance task in chronological order, the method further includes the following steps.

[0112] S1110 receives the tenth input for the target time frame in the timeline editor.

[0113] In some examples, the timeline editor can also be used to edit device performance tasks, such as inserting a performance clip at a specific time frame. The tenth input can be used to select the target time frame to be edited in the timeline editor; for example, the tenth input can be the user's action of clicking on the target time frame in the timeline editor.

[0114] S1120, in response to the tenth input, determines the target time frame.

[0115] In some examples, the target time frame can be the data frame corresponding to any time selected by the user in the timeline editor.

[0116] In some specific examples, such as Figure 10 As shown, when a user needs to edit a device performance task, they can click to select the target time frame 2214 in the timeline editor, and then determine that the target time frame 2214 is the time frame to be edited, so that the data corresponding to the time frame can be processed later.

[0117] S1130, Display the preset material library, which includes at least one preset behavior material.

[0118] In some specific examples, such as Figure 10 As shown, users can directly display a preset material library in a pop-up window after selecting the target time frame 2214, or they can click on the material library to bring up a control, which will then display the material selection interface corresponding to the preset material library. For example, when a user clicks on the audio material library to bring up control 31, the audio material selection interface corresponding to the audio material library will be displayed; when a user clicks on the action material library to bring up control 32, the action material selection interface 30 corresponding to the action material library will be displayed; and when a user clicks on the emoticon material library to bring up control 33, the emoticon material selection interface corresponding to the emoticon material library will be displayed.

[0119] S1140, receive the eleventh input for the first row of materials in the preset material library.

[0120] In some examples, the eleventh input can be an input used to select the first row of content to be inserted from a preset media library. For example, the eleventh input can be a user's action of clicking to select the first row of content from the preset media library. The first row of content can be any row of content in the preset media library.

[0121] S1150, in response to the seventh input, inserts target behavior material into the device performance task based on the target time frame.

[0122] In some examples, once the user selects the first line of footage, it can be inserted at the time position corresponding to the target time frame in the device performance task. In other examples, once the user selects the first line of footage, it can also be inserted at the time position corresponding to an idle time frame after the target time frame.

[0123] In some specific examples, such as Figure 10 As shown, when a user selects clapping action material 301 from the action material selection interface 30, it can be displayed at the time position corresponding to the idle time frame after the target time frame 2214, for example, in... Figure 12 Insert the clapping action footage 301 at the time position corresponding to the idle time frame 2215 shown. Alternatively, it can be done directly in... Figure 10 The clapping action footage 301 is inserted at the time position corresponding to the target time frame 2214 shown, which is not limited here. In this way, the device performance task can be customized and edited through the timeline editor.

[0124] In this embodiment, by using the timeline editor to edit the device performance task, the generated device performance task can be modified quickly and conveniently. This can improve the flexibility of secondary development of intelligent agent devices, further improve the efficiency of secondary development, and reduce the difficulty of secondary development.

[0125] Based on the above embodiments, a specific example is given below to better illustrate the task generation method of this application.

[0126] In such Figure 2 In the task generation interface shown, users can click the audio acquisition method switch control 205 and select the method of generating audio files to acquire the target audio. In this method, users can upload the target audio by dragging and dropping audio files to the audio acquisition control 201, or by clicking the audio acquisition control 201 to find and select audio files locally to upload the target audio, or by clicking the audio acquisition control 201 to trigger the connected microphone to record audio in the form of sound acquisition to acquire the target audio.

[0127] In addition, such as Figure 4 As shown, users can also switch the audio acquisition method by clicking control 205 and select the method of generating the target audio by inputting spoken text. In this method, users can input the target text content in the text input control 206, such as "Hello everyone, I am a robot, it's nice to meet you all." Then, by clicking the speech synthesis switch control 220, the speech synthesis function is turned on. By clicking the timbre parameter selection control 208, the female timbre parameter is selected, and by clicking the emotion parameter selection control 209, the happy emotion parameter is selected. After completing the configuration, by clicking the audio synthesis control 207, the target audio with the target text content "Hello everyone, I am a robot, it's nice to meet you all" can be generated using the female timbre characteristics and the happy emotion characteristics.

[0128] After obtaining the target audio, it can be displayed as follows: Figure 6 The task generation interface shown allows users to trigger semantic recognition processing of the target audio by clicking the audio processing control 210, thereby displaying... Figure 7 The task generation interface shown displays the recognized text in the editable area 211, allowing users to perform editing operations such as inserting behavioral materials into the recognized text.

[0129] In addition, during the semantic recognition processing of the target audio, a pop-up window can be displayed as follows: Figure 9 The background behavior generation prompt interface 216 shown allows users to choose to click the confirm generation control 217 to trigger the generation of background behavior corresponding to the target audio, or click the cancel generation control 218 to prevent the generation of background behavior corresponding to the target audio.

[0130] After the semantic recognition of the target audio is completed and the recognized text is displayed, operations such as inserting behavioral materials into the recognized text can be performed through custom editing. For example, ... Figure 8As shown in the figure, the user can select the word "good" in the recognized text displayed in the editable area 211, and click the expression material library trigger control 212 to trigger the display of the expression material library display interface 214. The user can select the required happy behavior material 215 in the expression material library display interface 214, thereby triggering the insertion of the happy behavior material 215 to the position corresponding to the word "good", realizing custom editing. Of course, corresponding expression, action and other behavior materials can also be added to the positions corresponding to other texts in the same or similar way, which will not be elaborated here. After editing the recognized text, the user can click the task generation control 202 to trigger the generation of the corresponding device performance task, that is, generate the corresponding device performance task according to the inserted behavior materials and the target audio.

[0131] In addition, after the semantic recognition of the target audio is completed and the recognized text is displayed, the insertion operation of the behavior materials and the generation of the corresponding device performance task can also be automatically performed in an intelligent arrangement manner. For example, as Figure 7 shown, the user can click the intelligent arrangement trigger control 219 to trigger the intelligent arrangement processing process. The platform automatically matches the corresponding behavior materials for the target audio and inserts them into the corresponding text, and then generates the corresponding device performance task according to the automatically inserted behavior materials and the target audio.

[0132] In this way, through the above interaction process based on the task generation interface, the secondary development of the intelligent device behavior can be conveniently and quickly realized, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0133] The above combines Figures 1 to 12 and details the embodiment of the task generation method of the present application. Next, in combination with Figure 13 , the embodiment of the task generation device of the present application is described in detail. It should be understood that the description of the embodiment of the task generation method corresponds to the description of the embodiment of the task generation device. Therefore, the parts not described in detail can be referred to the previous method embodiment.

[0134] Figure 13 As shown in the figure is the structural schematic diagram of the task generation device provided by an embodiment of the present application. As Figure 13 shown, the task generation device 130 provided by the embodiment of the present application includes: The first receiving module 1301 is configured to receive a first input for triggering audio acquisition based on the task generation interface; The audio acquisition module 1302 is configured to acquire a target audio in response to the first input; The second receiving module 1303 is configured to receive a second input for triggering task generation based on the task generation interface; The task generation module 1304 is used to generate a device performance task corresponding to the audio content based on the audio content of the target audio in response to the second input.

[0135] In one embodiment of this application, the task generation device 130 further includes: a third receiving module, configured to receive a third input for the target text content; and a first acquiring module, configured to acquire input text corresponding to the target text content in response to the third input. Correspondingly, the audio acquiring module 1302 is further configured to: generate corresponding audio based on the input text to obtain the target audio.

[0136] In one embodiment of this application, the task generation device 130 further includes: a fourth receiving module, configured to receive a fourth input for setting audio generation parameters; and a second acquiring module, configured to acquire target audio generation parameters in response to the fourth input. Correspondingly, the audio acquiring module 1302 is further configured to: generate audio corresponding to the input text based on the target audio generation parameters, thereby obtaining the target audio.

[0137] In one embodiment of this application, the audio generation parameters include at least one of timbre parameters and emotion parameters.

[0138] In one embodiment of this application, the task generation device 130 further includes: a fifth receiving module, configured to receive a fifth input for triggering audio processing; a semantic recognition module, configured to recognize the semantic content of the target audio in response to the fifth input and generate recognized text corresponding to the target audio; and a first display module, configured to display the recognized text.

[0139] In one embodiment of this application, the task generation device 130 further includes: a sixth receiving module for receiving a sixth input for target text in the identified text; a second display module for displaying a preset material library in response to the sixth input, wherein the preset material library includes at least one preset behavioral material; a seventh receiving module for receiving a seventh input for target behavioral material in the preset material library; and a material insertion module for inserting target behavioral material at the position corresponding to the target text in response to the seventh input. Accordingly, the task generation module 1304 is also used to generate a device performance task based on the target audio and the behavioral material inserted in the identified text.

[0140] In one embodiment of this application, the task generation device 130 further includes: an eighth receiving module, configured to receive an eighth input for triggering the generation of background behavior; and a behavior generation module, configured to generate background behavior corresponding to the target audio based on the audio content of the target audio in response to the eighth input. Accordingly, the task generation module 1304 is also configured to generate a device performance task based on the audio content of the target audio and the background behavior.

[0141] In one embodiment of this application, the task generation module 1304 is further configured to: determine behavioral materials matching the audio content from a preset material library based on the audio content of the target audio; and generate a device performance task based on the target audio and the behavioral materials matching the audio content.

[0142] In one embodiment of this application, the behavioral material includes at least one of audio material, motion material, and facial expression material.

[0143] Thus, based on the above embodiments, by utilizing the task generation interface, the target audio that needs to be performed by the device can be conveniently obtained, and a device performance task adapted to the audio content of the target audio can be quickly generated, so that intelligent devices such as robots can perform the device performance task. This facilitates and quickly completes the secondary development of device behavior, thereby reducing the difficulty of secondary development and improving the efficiency of secondary development.

[0144] Below, for reference Figure 14 This describes an electronic device according to embodiments of the present application. Figure 14 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0145] like Figure 14 As shown, the electronic device 140 includes one or more processors 1401 and memory 1402.

[0146] The processor 1401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 140 to perform desired functions.

[0147] The memory 1402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1401 may execute the program instructions to implement the data acquisition methods of the various embodiments of this application described above and / or other desired functions. The computer-readable storage medium may also store various contents such as data acquisition task templates, configuration information, instance tasks, etc.

[0148] In one example, the electronic device 140 may also include an input device 1403 and an output device 1404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0149] The input device 1403 may include, for example, a keyboard, a mouse, etc.

[0150] The output device 1404 can output various information to the outside, including data acquisition task templates, configuration information, instance tasks, etc. The output device 1404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0151] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 140 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 140 may include any other suitable components depending on the specific application.

[0152] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data acquisition methods according to various embodiments of this application described above.

[0153] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the data acquisition methods according to various embodiments of this application described above.

[0155] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0156] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0157] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0158] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0159] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A task generation method, characterized in that, include: Based on the task generation interface, receive the first input used to trigger audio acquisition; In response to the first input, acquire the target audio; Based on the task generation interface, a second input for triggering task generation is received; In response to the second input, a device performance task corresponding to the audio content of the target audio is generated.

2. The method according to claim 1, characterized in that, Also includes: Receive a third input regarding the target text content; In response to the third input, obtain the input text corresponding to the target text content; The acquisition of the target audio includes: The target audio is obtained by generating the corresponding audio based on the input text.

3. The method according to claim 2, characterized in that, Before generating the corresponding audio based on the input text to obtain the target audio, the method further includes: Receive a fourth input for setting audio generation parameters; In response to the fourth input, the target audio generation parameters are obtained; The step of generating corresponding audio based on the input text to obtain the target audio includes: Based on the target audio generation parameters, an audio corresponding to the input text is generated to obtain the target audio.

4. The method according to claim 3, characterized in that, The audio generation parameters include at least one of timbre parameters and emotion parameters.

5. The method according to claim 1, characterized in that, After acquiring the target audio, the method further includes: Receive the fifth input used to trigger audio processing; In response to the fifth input, the semantic content of the target audio is identified, and the identified text corresponding to the target audio is generated; Display the identified text.

6. The method according to claim 5, characterized in that, After displaying the identified text, the method further includes: Receive a sixth input for the target text in the identified text; In response to the sixth input, a preset material library is displayed, wherein the preset material library includes at least one preset behavior material; Receive the seventh input for the target behavior material in the preset material library; In response to the seventh input, the target behavior material is inserted at the position corresponding to the target text; The step of generating a device performance task corresponding to the audio content based on the target audio includes: A device performance task is generated based on the target audio and the behavioral material inserted into the recognized text.

7. The method according to claim 5, characterized in that, Also includes: Receive the eighth input used to trigger the background generation behavior; In response to the eighth input, a background behavior corresponding to the target audio is generated based on the audio content of the target audio. The step of generating a device performance task corresponding to the audio content based on the target audio includes: Based on the audio content of the target audio and the background behavior, a device performance task is generated.

8. The method according to claim 1, characterized in that, The step of generating a device performance task corresponding to the audio content based on the target audio includes: Based on the audio content of the target audio, determine the behavioral materials that match the audio content from the preset material library; Based on the target audio and behavioral material that matches the audio content, a device performance task is generated.

9. The method according to claim 6 or 8, characterized in that, The behavioral materials include at least one of audio materials, action materials, and facial expression materials.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the task generation method according to any one of claims 1 to 9.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the task generation method according to any one of claims 1 to 9.

12. A computer program product, characterized in that, The computer program product includes instructions that, when executed on an electronic device, cause the electronic device to perform the task generation method according to any one of claims 1 to 9.

Images