Virtual human development support system, virtual human development support method, and virtual human development support program

Abstract

A virtual human development support system according to the present disclosure includes a scenario specification unit and a data provision unit. At least the scenario specification unit specifies a plurality of services to be used in a scenario to be executed by a virtual human. The scenario specification unit specifies a plurality of data formats corresponding respectively to the plurality of services. The data provision unit provides data to be used in order to execute the scenario in the data format corresponding to each of services.

Description

[Technical Field]

[0001] The present disclosure relates to a virtual human development support system, a virtual human development support method, and a virtual human development support program. [Background technology]

[0002] Advances in ICT (Information and Communication Technology) have made it possible to create digital twins.

[0003] A digital twin is a digital replica that connects the real world with cyberspace. For example, a digital twin accurately represents a real-world object, such as a production machine in a factory, an aircraft engine, or an automobile. Such a digital twin maps, for example, the shape, state, or function of a real-world object in cyberspace.

[0004] The idea of ​​digital twins can also be applied to humans. For example, a human digital twin would digitally recreate a person's appearance, inner self, or memories. Such human digital twins have been attracting attention in recent years.

[0005] If the idea of ​​a human digital twin were to become a reality, humans would be able to create an avatar in cyberspace. This avatar would be able to carry out various activities in place of the person. Therefore, it is expected that human digital twins will expand the range of human activities and increase the time density of those activities.

[0006] Regarding the range of activities, for example, a human digital twin can substitute for handicapped physical functions. For example, even if a person has an upper body disability, this person can still speak in a VR (Virtual Reality) conference using gestures.

[0007] With regard to temporal density, for example, a person can have their human digital twin type an email on their behalf. As another example, a human digital twin can discuss with other people and gather information.

[0008] One way to build a human digital twin is to combine multiple deep learning models. A collection of multiple deep learning models forms a single human digital twin. Such a human digital twin can be built by the following process: (1) building / updating multiple deep learning models, and (2) combining the multiple deep learning models. The built human digital twin can be grown by updating the multiple deep learning models.

[0009] On the other hand, virtual humans, including human digital twins, are expected to be used in a variety of scenarios (e.g., application scenarios) in which virtual humans are needed.

[0010] Examples of scenarios include questionnaires and dialogues (e.g., Q&A, chat). In the questionnaire scenario, for example, an AI (artificial intelligence) of a person is created based on the person's life log data. The AI ​​reflects the person's personality. The AI ​​automatically answers the questionnaire.

[0011] The logic of the virtual human differs for each scenario, so the virtual human is developed separately for each scenario.

[0012] However, when a new scenario emerges, engineers must essentially develop a virtual human from scratch to accommodate the new scenario.

[0013] If the original scenario and the new scenario are dialogue scenarios, engineers can reuse the virtual human created for one scenario in another. In dialogue scenarios, engineers can replace the speech generation function, which is part of the virtual human's functions. For example, the Q&A function can be replaced with a chat function. [Prior art documents] [Non-patent literature]

[0014] [Non-Patent Document 1] “What are digital humans?” [online], [Retrieved April 12, 2022], Internet<https: / / digitalhumans.com / what-are-digital-humans / > [Non-patent document 2] “Plato Research Dialogue System” [online], [Retrieved April 12, 2022], Internet<https: / / github.com / uber-archive / plato-research-dialogue-system> Summary of the Invention [Problem to be solved by the invention]

[0015] However, the above prior art techniques may have difficulty supporting the development of virtual humans with a wide range of applications.

[0016] For example, the above prior art only considers dialogue scenarios. The interchangeable service is a service based on natural language processing whose input and output are text. The interchangeable service is connected in series with other services (e.g., image recognition, speech synthesis) in the dialogue scenario. Therefore, it may be difficult for engineers to apply the dialogue virtual human to scenarios other than dialogue.

[0017] Therefore, the present disclosure proposes a virtual human development support system, a virtual human development support method, and a virtual human development support program that can support the development of virtual humans with a wide range of applications. [Means for solving the problem]

[0018] In one aspect of the present disclosure, a virtual human development support system includes a first identification unit that identifies a plurality of services to be used in a scenario executed by a virtual human, a second identification unit that identifies a plurality of data formats corresponding to the plurality of services, and a providing unit that provides data to be used in executing the scenario to each service in the corresponding data format. [Effects of the Invention]

[0019] A virtual human development support system according to one or more embodiments of the present disclosure can support the development of virtual humans with a wide range of applications. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a block diagram of an example environment for assisting virtual human development. [Figure 2A] FIG. 2A shows an overview of the development support process according to the present disclosure. [Figure 2B] FIG. 2B shows an overview of the development support process according to the present disclosure. [Figure 3] FIG. 3 is a block diagram of an example of the configuration of a development support system according to the present disclosure. [Figure 4] FIG. 4 illustrates an example of functional units of a development support system according to the present disclosure. [Figure 5] FIG. 5 shows an example of the flow of data between functional units of the development support system according to the present disclosure. [Figure 6] FIG. 6 shows an example of setting information according to the present disclosure. [Figure 7]FIG. 7 is a block diagram of an example configuration of a first agent according to the present disclosure. [Figure 8] FIG. 8 illustrates an example of setting information for the first agent according to the present disclosure. [Figure 9] FIG. 9 illustrates an example of service collaboration performed by a first agent according to the present disclosure. [Figure 10] FIG. 10 illustrates an example of input and output of a first agent according to the present disclosure. [Figure 11] FIG. 11 is a block diagram of an example configuration of a second agent according to the present disclosure. [Figure 12] FIG. 12 illustrates an example of configuration information for a second agent according to the present disclosure. [Figure 13] FIG. 13 illustrates an example of service collaboration performed by a second agent according to the present disclosure. [Figure 14] FIG. 14 is a block diagram of an example configuration of a third agent according to the present disclosure. [Figure 15] FIG. 15 is a flowchart illustrating an example of a process for assisting the development of a virtual human. [Figure 16] FIG. 16 shows an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0021] Several embodiments will be described in detail below with reference to the drawings. However, the present invention is not limited to these several embodiments. Several features of various embodiments can be combined in various ways, provided that these several features are not mutually contradictory. Identical elements are indicated by the same reference numerals, and redundant descriptions will be omitted.

[0022] The following description is divided into nine sections: 1. Environment for Virtual Human Development Support, 2. Overview of Development Support Processing, 3. Configuration of Development Support System, 4. Details of Development Support Processing, 5. Flowchart of Development Support Processing, 6. Effects, 7. Other, 8. Hardware Configuration, and 9. Summary of Embodiments.

[0023] [1. Environment for supporting virtual human development] First, with reference to FIG. 1, an environment for assisting virtual human development according to the present disclosure will be described.

[0024] 1 is a block diagram of an environment 1, which is an example of an environment for virtual human development support. As shown in FIG. 1, the environment 1 includes a development support system 100, a network 200, and a plurality of services 300. The development support system 100 is an example of a virtual human development support system.

[0025] The development support system 100 is a system that performs one or more development support processes. The one or more development support processes include a process for supporting the development of a virtual human. An overview of the development support process according to the present disclosure will be described in the next section.

[0026] The development support system 100 includes one or more data processing devices, such as one or more servers. An example of the configuration of the development support system 100 is described in Section 3.

[0027] The network 200 is, for example, a local area network (LAN), a wide area network (WAN), the Internet, etc. The network 200 connects the development support system 100 and a plurality of services 300.

[0028] The plurality of services 300 are services that provide functions to the development support system 100. The plurality of services 300 are implemented by a data processing device such as a server.

[0029] [2. Overview of development support processing] Next, an overview of the development support process according to the present disclosure will be described with reference to Figures 2A and 2B. Note that this overview is not intended to limit the present invention or the embodiments described in the following sections.

[0030] 2A and 2B show an overview of the development support process according to the present disclosure. As shown in FIG. 2A, the development support system 100 includes the following functional units: (1) input data transfer 11, (2) service collaboration 12, (3) service invocation 13, and (4) data output 14.

[0031] The functional unit for input data delivery 11 delivers input data to the functional unit for service collaboration 12. The functional unit for service collaboration 12 links multiple services 300 together. The functional unit for service invocation 13 calls each service 300. The functional unit for data output 14 outputs data generated by the service collaboration.

[0032] For convenience, hereinafter, the "functional unit of input data transfer 11" will be referred to as "input data transfer 11." Also, the "functional unit of service collaboration 12" will be referred to as "service collaboration 12," the "functional unit of service invocation 13" will be referred to as "service invocation 13," and the "functional unit of data output 14" will be referred to as "data output 14."

[0033] First, the input data transfer 11 receives a stream of byte data, and then transfers the byte data to the service collaboration 12.

[0034] Byte data is data that has been encoded in a specific data format. Byte data is represented as a sequence of bytes. Examples of byte data include binary data such as text, audio, and video.

[0035] Next, the service collaboration 12 stores the byte data in a repository, and distributes the byte data to the service invocation 13.

[0036] The service call 13 converts the byte data into a data format required by each service 300. Therefore, each service 300 can be replaced with a service whose input and output are in a data format other than text.

[0037] The services 300 may include services that input or output byte stream data and services that input or output byte batch data. One example of a service 300 is Azure® speech to text, which inputs audio in a stream and then outputs text for each utterance.

[0038] 2A, the service collaboration 12 can input or output byte stream data. The service collaboration 12 can also input or output byte batch data. Therefore, the development support system 100 can combine various services 300 with different input / output methods.

[0039] The service call 13 provides data to the service 300. The provided data is byte data converted into a data format required by the service 300. The service 300 generates data using the provided data. The service 300 outputs the generated data to the development support system 100. The service call 13 converts the output data into byte data. The service call 13 then sends the byte data to the service collaboration 12.

[0040] Next, the service collaboration 12 assigns a name to the byte data. The service collaboration 12 manages the byte data within the service collaboration 12. This name can specify the data to be input to each service 300. Therefore, the development support system 100 can connect multiple services 300 in parallel.

[0041] 2B, the development support system 100 further includes a function unit for reading a setting file. For convenience, hereinafter, the "function unit for reading a setting file 15" will be referred to as the "setting file reader 15."

[0042] The setting file reader 15 obtains an external file, and reads information from the external file that is relevant to the processing described above with reference to Figure 2A.

[0043] In the example of Fig. 2B, the external files are configuration file 16a, configuration file 16b, and configuration file 16c. Configuration file 16a and configuration file 16b contain information related to collaboration between services 300. Configuration file 16c contains information related to the connection between services 300. The information related to collaboration between services 300 is called service collaboration information. Furthermore, the information related to the connection between services 300 is called service connection information.

[0044] These setting files contain various information such as the service 300 to be connected, the input / output data format, the input / output method, the name of the byte data, and the name of the data to be input to each service 300.

[0045] The configuration file does not need to specify the "name of data input to each service 300," which is part of the service collaboration information. If the configuration file does not specify the name of the data, the service collaboration 12 collaborates the services described in the service collaboration information (flow definition) in series from top to bottom. Specifically, the service collaboration 12 collaborates the described services in the order in which the output data format of the connected service 300 matches the input data format of the connected service 300.

[0046] As described above, development support system 100 can integrate various services whose input and output data formats are non-textual. The various services can be implemented in parallel in a virtual human. As a result, development support system 100 can support the development of virtual humans with a wide range of applicability.

[0047] [3. Configuration of the development support system] Next, an example of the configuration of the development support system 100 will be described with reference to FIG.

[0048] Fig. 3 is a block diagram of an example configuration of a development support system 100 according to the present disclosure. As shown in Fig. 3, the development support system 100 includes a communication unit 110, a control unit 120, and a storage unit 130. The development support system 100 may include an input unit (e.g., a keyboard, a mouse) that receives input from an administrator of the development support system 100. The development support system 100 may also include an output unit (e.g., a liquid crystal display, an organic EL (Electro Luminescence) display) that displays information to the administrator of the development support system 100.

[0049] [3-1. Communication Unit 110] The communication unit 110 is implemented by, for example, a network interface card (NIC). The communication unit 110 is connected to the network 200 by wire or wirelessly. The communication unit 110 can transmit and receive information to and from the service 300 via the network 200.

[0050] 3-2. Control Unit 120 The control unit 120 is a controller. The control unit 120 uses a random access memory (RAM) as a working area and is implemented by one or more processors (e.g., a central processing unit (CPU) or a micro processing unit (MPU)) that execute various programs stored in a storage device of the development support system 100. The control unit 120 may also be implemented by an integrated circuit such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a general purpose graphic processing unit (GPGPU).

[0051] 3, the control unit 120 includes a data acquisition unit 121, a setting acquisition unit 122, a scenario identification unit 123, a data provision unit 124, and a data output unit 125. One or more processors of the development support system 100 can implement each control unit by executing instructions stored in one or more memories of the development support system 100. The data processing performed by each control unit is an example, and each control unit (e.g., the scenario identification unit 123) may perform the data processing described in relation to other control units (e.g., the data provision unit 124).

[0052] 4 shows a functional unit 20, which is an example of a functional unit of the development support system 100 according to the present disclosure. The functional unit 20 includes the input data transfer 11, service collaboration 12, service invocation 13, data output 14, and setting file reading 15 described above with reference to FIGS. 2A and 2B. As shown in FIG. 4, the data acquisition unit 121 includes the input data transfer 11. Furthermore, the setting acquisition unit 122 includes the setting file reading 15, the scenario identification unit 123 includes the service collaboration 12, the data provision unit 124 includes the service invocation 13, and the data output unit 125 includes the data output 14.

[0053] 5 shows a data flow 21 that is an example of the data flow between functional units of the development support system 100 according to the present disclosure. The data flow 21 represents the relationship between the various data or information described above with reference to FIGS. 2A and 2B and the functional units described above with reference to FIGS. 2A and 2B. The configuration file includes service collaboration information, service connection information, and various data information.

[0054] [3-2-1. Data Acquisition Unit 121] 4, the data acquisition unit 121 acquires various data. The data acquisition unit 121 is an example of a first acquisition unit. The data acquisition unit 121 can acquire various data from the storage unit 130.

[0055] Data acquisition unit 121 acquires data used to execute a scenario to be executed by a virtual human, such as input data for the virtual human, such as a stream of byte data.

[0056] 3-2-2. Setting Acquisition Unit 122 The setting acquisition unit 122 acquires setting information that describes settings for various scenarios. The setting acquisition unit 122 is an example of a second acquisition unit. The setting acquisition unit 122 can acquire the setting information from the storage unit 130.

[0057] The setting acquisition unit 122 acquires setting information that describes the settings of a scenario to be executed by a virtual human. The setting information is, for example, a setting file such as service linkage information, service connection information, and the like.

[0058] Regarding the setting file read 15, the input of the setting acquisition unit 122 is a setting file. The output of the setting acquisition unit 122 is (1) information regarding collaboration between the services 300 (service collaboration information) and (2) service connection information.

[0059] For example, assume that the output of speech recognition is input to speech generation, and the output of speech generation is input to speech synthesis. In this example, the service collaboration information includes the output name of speech recognition (e.g., stt_result). The service collaboration information also includes the output name of speech generation (e.g., statement).

[0060] The service connection information includes various information such as (1) the service name and service ID (identifier), (2) the input / output data format, (3) the data input / output method (e.g., stream or batch), (4) the data format (e.g., if the data is audio, the format includes the sampling frequency and number of channels), and (5) information required to connect to the service (e.g., endpoint, API (Application Programming Interface) key).

[0061] The setting acquisition unit 122 analyzes the input setting file. The setting acquisition unit 122 transmits the output of the setting acquisition unit 122 to the scenario identification unit 123. The setting acquisition unit 122 transmits the service connection information to the data provision unit .

[0062] [3-2-3. Scenario Identification Unit 123] The scenario identification unit 123 identifies a scenario to be executed by a virtual human. The scenario identification unit 123 is an example of a first identification unit and a second identification unit.

[0063] For example, the scenario identification unit 123 identifies a plurality of data formats corresponding to a plurality of services, respectively. Also, the scenario identification unit 123 identifies a plurality of data formats corresponding to a plurality of services, respectively.

[0064] With respect to service collaboration 12, the inputs to the scenario identification unit 123 are (1) service collaboration information, (2) service connection information, and (3) byte data received from each service call 13. The output of the scenario identification unit 123 is byte data sent to each service call 13.

[0065] 6 shows an example of setting information 30 according to the present disclosure. The setting information 30 includes service collaboration information and service connection information.

[0066] When the service cooperation information describes the name of the data to be input to each service 300 (input / output data name), the scenario identification unit 123 performs the following five operations.

[0067] The scenario identification unit 123 receives byte data of the input data from the data acquisition unit 121 (first operation).

[0068] The scenario identification unit 123 identifies a service for which the name of the byte data is input from the service cooperation information. Then, the scenario identification unit 123 transmits the byte data to the service call 13 corresponding to the identified service (second operation).

[0069] When byte data is to be transmitted, the scenario identification unit 123 selects an appropriate function unit from four input / output function units based on the data input / output method (e.g., stream or batch) described in the service connection information. The four input / output function units are (1) a byte stream input / output function unit, (2) a byte stream input / byte batch output function unit, (3) a byte batch input / byte stream output function unit, and (4) a byte batch input / output function unit.

[0070] The scenario identification unit 123 receives byte data corresponding to the identified service from the service call 13. Then, the scenario identification unit 123 performs the second operation described above (third operation).

[0071] The scenario identification unit 123 repeats the second and third operations described above until the scenario identification unit 123 receives data from a service that outputs data to be transmitted to the data output unit 125 (fourth operation).

[0072] The scenario identification unit 123 transmits the byte data to the data output unit 125 (fifth operation).

[0073] In the scenario identification unit 123, the data acquisition unit 121 and the data output unit 125 can be treated on the same level as the service call 13 and other service call functions.

[0074] If the service cooperation information does not include the name of the data to be input to each service 300 (input / output data name), the scenario identification unit 123 performs the following six operations.

[0075] The scenario identification unit 123 receives byte data of the input data from the data acquisition unit 121 (first operation).

[0076] The scenario identification unit 123 identifies the service whose input is in the original data format of the byte data from the service connection information (second operation).

[0077] The scenario identification unit 123 transmits the byte data to the service call 13 of the service listed in the service collaboration information at the location closest to the location of the service that transmitted the byte data (third operation). When the byte data is transmitted, the scenario identification unit 123 selects an appropriate function unit from the four input / output function units described above based on the data input / output method listed in the service connection information.

[0078] After the scenario identification unit 123 receives the byte data from the service call 13, the scenario identification unit 123 performs the second operation described above (fourth operation).

[0079] The scenario identification unit 123 repeats the second, third, and fourth operations described above until the scenario identification unit 123 receives data from a service that outputs data to be transmitted to the data output unit 125 (fifth operation).

[0080] The scenario identification unit 123 transmits the byte data to the data output unit 125 (sixth operation).

[0081] [3-2-4. Data providing unit 124] 4, the data providing unit 124 provides data to the service. The data providing unit 124 is an example of a providing unit.

[0082] The data providing unit 124 provides data to each service in a corresponding data format. The data provided to each service is used to execute a scenario executed by the virtual human. For example, the data providing unit 124 provides input data for the virtual human to one service (e.g., a first service) of the multiple services. Then, the data providing unit 124 provides a response from this one service to another service (e.g., a second service). The data providing unit 124 can repeat this process. As a result, output data based on the input data is generated using the collaboration of the multiple services.

[0083] For service call 13, the inputs of data provider 124 are (1) service connection information extracted from the configuration file, (2) byte data, and (3) output data output from the corresponding service. The outputs of data provider 124 are (1) byte data and (2) input data to the corresponding service.

[0084] The data providing unit 124 receives byte data from the scenario identifying unit 123. The data providing unit 124 converts the byte data into a specific data format required by the service being used. The data providing unit 124 then inputs the converted byte data to the service being used based on the service connection information in the setting file.

[0085] The data providing unit 124 converts the output data of this service into byte data, and then transmits the byte data to the scenario identifying unit 123.

[0086] If a service requires a dedicated library or SDK (Software Development Kit), a separate service call 13 is required for each service. If a service does not require such an environment, the data provider 124 can use a common service call 13.

[0087] If a new service call 13 is needed, the data providing unit 124 creates a new service call 13. The new service call 13 is coupled to the service linkage 12 of the scenario identifying unit 123.

[0088] 3-2-5. Data Output Unit 125 The data output unit 125 outputs output data based on the input data. The data output unit 125 is an example of an output unit.

[0089] The output data is output via a specific interface. As described above, the output data is data generated using the cooperation of multiple services.

[0090] [3-3. Storage section 130] The storage unit 130 is implemented by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 130 can store data or information used in the development support process according to the present disclosure.

[0091] [4. Details of development support processing] An overview of the development support process according to the present disclosure has been described above with reference to Figures 2A and 2B. In this section, an example of the development support process according to the present disclosure will be described in more detail.

[0092] [4-1. First Agent] In this subsection, a first agent, which is an example of a virtual human, will be described with reference to FIGS.

[0093] FIG. 7 is a block diagram of a first agent 40, which is an example of the configuration of a first agent according to the present disclosure. The first agent 40 is a dialogue agent that conducts Q&A by voice. The first agent 40 uses three services: (1) Azure Speech to text (voice recognition), (2) Azure QnA maker (utterance generation), and (3) Google (registered trademark) CloudText to Speech (voice synthesis). The PC microphone acquires audio data. The speaker outputs sound.

[0094] 8 shows configuration information 41, which is an example of configuration information for the first agent 40 according to the present disclosure. The configuration information 41 includes service collaboration information and service connection information. As shown in FIG. 8, the configuration information 41 is created using YAML (YAML Ain't a Markup Language).

[0095] 9 shows a service collaboration 42 that is an example of service collaboration performed by the first agent 40 according to the present disclosure. The scenario identification unit 123 performs the service collaboration based on the setting information 41.

[0096] First, the scenario identification unit 123 searches for a key that matches the n-th child key of the "Flow-config" key from the service-config. Hereinafter, the key that matches the n-th child key of the "Flow-config" key will be referred to as "service n".

[0097] Next, the scenario identification unit 123 searches for services from service n+1 onwards that have a value in the child key "type" of the "input" key that matches the value of the child key "type" of the "output" key of service n.

[0098] The scenario identification unit 123 sets the output of service n to the "input of the service with the smallest k value among the detected services n+k."

[0099] 10 shows an input / output 43 that is an example of input / output of the first agent 40 according to the present disclosure. The scenario identification unit 123 identifies the method of inputting and outputting data (for example, stream, batch).

[0100] First, the scenario identification unit 123 identifies data to be treated as batch data based on the value of the "BatchData" key.

[0101] If the service connection information does not describe a method for inputting and outputting data, the scenario identification unit 123 treats the data as stream data.

[0102] The scenario identification unit 123 selects an appropriate input / output function unit corresponding to the service call 13 from the four input / output function units based on the data input / output method.

[0103] [4-2. Second Agent] In this subsection, a second agent, which is an example of a virtual human, will be described with reference to FIGS.

[0104] 11 is a block diagram of a second agent 50, which is an example of the configuration of a second agent according to the present disclosure. The second agent 50 is a dialogue agent that receives video and audio and outputs the movement and audio of a 3D character. As with the first agent 40, the second agent 50 is implemented by the development support system 100.

[0105] The second agent 50 recognizes faces from camera footage and estimates names. The second agent 50 converts audio into text using speech-to-text. The second agent 50 acquires utterances corresponding to the content of the text from DialogFlow (registered trademark). The second agent 50 synthesizes the name and the utterance to create the final utterance content. The second agent 50 then generates voice synthesis and movements in accordance with the voice synthesis.

[0106] 12 shows configuration information 51, which is an example of configuration information for a second agent according to the present disclosure. The configuration information 51 includes service linkage information. The configuration information 51 also includes service connection information (not shown). As shown in FIG. 12, the configuration information 51 is created using YAML.

[0107] 13 shows a service collaboration 52 that is an example of service collaboration performed by the second agent according to the present disclosure. The scenario identification unit 123 performs the service collaboration based on the setting information 51.

[0108] The scenario identification unit 123 uses the value of the child key "output-name" of each child key (service name) of the "Flow-config" key as the output data name of this service.The scenario identification unit 123 then uses data with a name that matches the value of the child key "input-name" of each child key (service name) of the "Flow-config" key as the input data of this service.

[0109] [4-3. The Third Agent] In this subsection, we will explain a third agent, which is an example of a virtual human, with reference to FIG.

[0110] 14 is a block diagram of a third agent 60, which is an example of the configuration of a third agent according to the present disclosure. The third agent 60 is an RPA (Robotic Process Automation) agent that performs email-related tasks and notifies users of completion by voice. As with the first agent 40, the third agent 60 is implemented by the development support system 100.

[0111] The third agent 60 stores the PDF (Portable Document Format) file attached to the email sent from the scanner in a specified folder, and changes the file name to specific characters in the file.

[0112] The third agent 60 detects newly received emails that contain the specified name.

[0113] The service collaboration method of the third agent 60 is the same as that of the second agent 50. Here, overlapping explanations will be omitted.

[0114] [5. Flowchart of development support process] Next, a flowchart of an example of development support processing according to the present disclosure will be described with reference to Fig. 15. The example of the development support processing includes processing for supporting the development of a virtual human. The processing for supporting the development of a virtual human is performed, for example, by development support system 100 of Fig. 1.

[0115] FIG. 15 is a flowchart showing process P100, which is an example of a process for supporting the development of a virtual human.

[0116] The data acquisition unit 121 of the development support system 100 acquires input data (step S101). The input data is, for example, the audio data described above with reference to FIG.

[0117] The scenario identification unit 123 of the development support system 100 identifies a first data format required by the first service (step S102). The first service is, for example, Azure Speech to text, as described above with reference to Fig. 7. In this case, the first data format is audio.

[0118] The data providing unit 124 of the development support system 100 provides the input data to the first service in the first data format (step S103).

[0119] The data providing unit 124 of the development support system 100 receives the first data from the first service (step S104). The first data is, for example, text output from Azure Speech to text, as described above with reference to FIG. 7.

[0120] The scenario identification unit 123 identifies a second data format required by the second service (step S105). The second service is, for example, Azure QnA maker described above with reference to Fig. 7. In this case, the second data format is text.

[0121] The data providing unit 124 provides the first data to the second service in the second data format (step S106).

[0122] The data providing unit 124 receives the second data from the second service (step S107).

[0123] The scenario identification unit 123 and the data providing unit 124 can repeat the processes from step S105 to step S107 until the data to be used by the data output unit 125 is output from the specific service.

[0124] [6. Effects] The development support system 100 can apply parts of a virtual human to scenarios other than dialogue. Therefore, the development support system 100 can support the development of a virtual human with a wide range of applications. As a result, the development support system 100 can reduce the development costs of a virtual human.

[0125] Furthermore, engineers can (1) configure service integration when a virtual human is constructed and (2) swap services when the virtual human is applied to another scenario simply by changing configuration files. This allows engineers to swap service integrations with fewer man-hours than if they had to code a virtual human program from scratch and swap service integrations.

[0126] [7. Other] A portion of a process described as being performed automatically may be performed manually. Alternatively, all or part of a process described as being performed manually may be performed automatically using known methods. Furthermore, information including the process procedures, specific names, various data, and parameters shown in this specification and drawings may be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0127] The illustrated system components conceptually represent the system functions. The components are not necessarily physically configured as shown in the drawings. In other words, the specific form of the distributed or integrated system is not limited to the system and device forms shown in the drawings. All or part of the system may be functionally or physically distributed or integrated depending on various loads and usage conditions.

[0128] [8. Hardware Configuration] 16 is a diagram showing a computer 1000, which is an example of a hardware configuration of a computer. The systems and methods described in this specification are implemented by, for example, the computer 1000 shown in FIG.

[0129] 16 shows an example of a computer on which a development support system 100 is implemented by executing a program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0130] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0131] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process of the development support system 100 are implemented as program modules 1093 in which code executable by the computer 1000 is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same processes as the functional configuration of the development support system 100 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0132] The hard disk drive 1090 can store a virtual human development support program for the development support process. The virtual human development support program can also be created as a program product. When the program product is executed, it performs one or more of the methods described above.

[0133] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0134] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN, WAN, etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0135] 9. Summary of Embodiments As described above, development support system 100 includes scenario identification unit 123 and data providing unit 124. In at least one embodiment, scenario identification unit 123 identifies a plurality of services to be used in a scenario executed by a virtual human. In at least one embodiment, scenario identification unit 123 identifies a plurality of data formats corresponding to the plurality of services, respectively. In at least one embodiment, data providing unit 124 provides data to be used to execute the scenario to each service in the corresponding data format.

[0136] As described above, development support system 100 includes data acquisition unit 121 and setting acquisition unit 122. In at least one embodiment, input data for a virtual human is acquired as data used to execute a scenario. In at least one embodiment, setting acquisition unit 122 acquires setting information describing the settings of the scenario. In some embodiments, scenario identification unit 123 identifies multiple services from the setting information. In some embodiments, scenario identification unit 123 identifies multiple data formats using the setting information. In some embodiments, data provision unit 124 provides input data to one of the multiple services such that output data based on the input data is generated using collaboration between the multiple services.

[0137] As described above, the development support system 100 according to the present disclosure includes the data output unit 125. In at least one embodiment, the data output unit 125 outputs the output data generated using the collaboration of multiple services via a specific interface.

[0138] In some embodiments, the data acquisition unit 121 acquires any one of text, audio, or video as input data, and in some embodiments, the data output unit 125 outputs any one of text, audio, or video as output data.

[0139] In some embodiments, data acquisition unit 121 acquires byte data as input data for a virtual human. In some embodiments, data provision unit 124 converts the byte data into a data format corresponding to one of the services, and provides the byte data converted into the data format corresponding to the one service to the one service.

[0140] In some embodiments, the setting acquisition unit 122 acquires setting information that describes information regarding the collaboration of multiple services. In some embodiments, the scenario identification unit 123 identifies multiple services that are to be linked together from the information regarding the collaboration.

[0141] Although various embodiments have been described in detail herein with reference to the drawings, these embodiments are merely examples and are not intended to limit the present invention. The features described herein can be realized in various ways, including various modifications and improvements based on the knowledge of those skilled in the art.

[0142] Furthermore, the above-mentioned "module (-er suffix, -or suffix)" can be read as a unit, means, circuit, etc. For example, a communication module, a control module, and a storage module can be read as a communication unit, a control unit, and a storage unit, respectively. [Explanation of symbols]

[0143] 1 Environment 100 Development Support System 110 Communications Department 120 control section 121 Data Acquisition Unit 122 Setting acquisition unit 123 Scenario Identification Section 124 Data Provision Department 125 Data output section 130 Storage section 200 Network 300 Service

Claims

1. a first identification unit that identifies a plurality of services to be used in a scenario executed by the virtual human; a second identification unit that identifies a plurality of data formats corresponding to the plurality of services, respectively; a providing unit that provides data used to execute the scenario to each service in a corresponding data format; Equipped with a first acquisition unit that acquires byte data of input data for the virtual human as the data used to execute the scenario; a second acquisition unit that acquires setting information describing the settings of the scenario, the setting information including the name of byte data to be input to each service; the first identification unit identifies, from the setting information, a service for which the name of the byte data acquired by the first acquisition unit is an input; the second identification unit uses the setting information to identify the plurality of data formats; The providing unit converts the byte data into a data format corresponding to one of the plurality of services, and provides the byte data converted into the data format corresponding to the one service to the one service.

2. An output unit that outputs output data generated using the collaboration of the plurality of services via a specific interface. The virtual human development support system according to claim 1 .

3. the first acquisition unit acquires one of text, audio, or video as the input data; The output unit outputs any one of text, audio, and video as the output data.

3. The virtual human development support system according to claim 2.

4. the second acquisition unit acquires the setting information describing information related to the cooperation of the plurality of services; The first identification unit identifies the plurality of services that are to be linked with each other from the information related to the linkage. The virtual human development support system according to claim 1 .

5. A computer-implemented virtual human development support method, comprising: a first identification step of identifying a plurality of services to be used in a scenario executed by the virtual human; a second specifying step of specifying a plurality of data formats corresponding to the plurality of services, respectively; a providing step of providing data used to execute the scenario to each service in a corresponding data format; Including, a first acquisition step of acquiring byte data of input data for the virtual human as the data to be used for executing the scenario; and a second acquisition step of acquiring setting information describing the settings of the scenario, the setting information including the name of byte data to be input to each service, the first identifying step identifies, from the setting information, a service for which the name of the byte data acquired in the first acquiring step is an input; the second specifying step uses the setting information to specify the plurality of data formats; A virtual human development support method in which the providing step converts the byte data into a data format corresponding to one of the plurality of services, and provides the byte data converted into the data format corresponding to the one service to the one service.

6. A virtual human development support program for causing a computer to function as the virtual human development support system according to any one of claims 1 to 4.

Images