Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program that are more useful for users.SOLUTION: An information processing system includes at least one processor. The processor reads a program to execute: a first reception step S001 of receiving voice information for a voice uttered by a user; an input step S002 of inputting a prompt including the voice information to an artificial intelligence module such as a conversational AI; a generation step S003 of dividing a reply from the artificial intelligence module into a plurality of segments and sequentially generating moving image data corresponding to each of the plurality of segments; and an output step S004 of sequentially outputting the moving image data. The generation step and the output step are executed in parallel. The moving image data includes animation of an avatar's uttering the reply and a synthetic voice corresponding to the reply.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 is a document related to efficiently obtaining a reply message that a user expects when the user sends a chat message related to a question. When a multi-cloud chat service providing device 10 disclosed in Patent Document 1 receives a question message from a user terminal 20, it transmits the question message to an AI chat cloud service system 30 and receives a reply message to the question message from the AI ​​chat cloud service system 30. Furthermore, when the multi-cloud chat service providing device 10 determines that a predetermined condition is satisfied, it transmits the question message to an operator terminal 40A operated by an operator and receives a reply message to the question message from the operator terminal 40A. The multi-cloud chat service providing device 10 then returns the reply message received from the AI ​​chat cloud service system 30 or the operator terminal 40A to the user terminal 20. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-128737 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is still room for improvement in the technology of AI-based response systems that respond to user utterances. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system. The information processing system includes at least one processor, and the processor is configured to execute the following steps by reading a program. In a first receiving step, voice information corresponding to a voice uttered by a user is received. In an input step, a prompt including the voice information is input to an artificial intelligence module. In a generating step, a response from the artificial intelligence module is divided into a plurality of segments, and video data corresponding to each of the plurality of segments is generated in sequence. The video data includes an animation of an avatar speaking the response and a synthesized voice corresponding to the response. In an output step, each piece of video data is output in sequence. The generating step and the output step are executed in parallel.

[0006] According to one aspect of the present invention, it is possible to provide an information processing system or the like that is more useful to users. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. FIG. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the server 2. [Figure 3] FIG. 2 is a block diagram showing a hardware configuration of an information processing device 3. [Figure 4] 2 is a flowchart showing an outline of processing executed by the information processing system 1. [Figure 5] FIG. 2 is a diagram illustrating an example of a usage mode of the information processing system 1. [Figure 6] 2 is a conceptual diagram showing an example of the flow of information processing in the information processing system 1. FIG. [Figure 7] 2 is an activity diagram showing an example of the flow of processing executed by the information processing system 1. FIG. [Figure 8] FIG. 10 is a diagram showing an example of dividing a reply 8 into multiple segments. [Figure 9] FIG. 9 is an explanatory diagram illustrating the relationship between generation and output of video data corresponding to each segment. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0009] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] In this embodiment, the term "unit" may also include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and this information may be represented by, for example, physical values ​​of signal values ​​representing voltages and currents, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

[0011] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0012] 1. Hardware Configuration This section explains the hardware configuration.

[0013] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes a server 2 and an information processing device 3. The server 2 and the information processing device 3 are configured to be able to communicate with each other via a telecommunications line (network). Here, the system exemplified as the information processing system 1 is made up of one or more devices or components. Therefore, it should be noted that the information processing system 1 includes either the server 2 alone or both the server 2 and the information processing device 3. More specifically, the information processing system 1 may include an element selected from the group consisting of the server 2 and the information processing device 3. The unselected element may not be included in the information processing system 1, but may be electrically connected to the selected element as an external element. These components will be described below.

[0014] <Server 2> 2 is a block diagram showing the hardware configuration of server 2. Server 2 includes a communication unit 21, a storage unit 22, and a control unit 23, and these components are electrically connected via a communication bus 20 inside server 2. Each component will be further described below.

[0015] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as needed. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the server 2 may communicate various information from the outside via the communication unit 21 and the network.

[0016] The memory unit 22 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the server 2 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program operations. The memory unit 22 stores various programs, variables, etc. related to the server 2 executed by the control unit 23.

[0017] The control unit 23 processes and controls the overall operations related to the server 2. The control unit 23 is, for example, a central processing unit (CPU) not shown. The control unit 23 realizes various functions related to the server 2 by reading out predetermined programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is specifically realized by the control unit 23, which is an example of hardware, and each step related to each function described below can be executed. This will be described in further detail in the next section. Note that the control unit 23 is not limited to being single, and multiple control units 23 may be provided for each function. A combination of these may also be used.

[0018] <Information processing device 3> 3 is a block diagram showing the hardware configuration of the information processing device 3. The information processing device 3 includes a communication unit 31, a storage unit 32, a control unit 33, a display unit 34, an input unit 35, and an output unit 36, and these components are electrically connected via a communication bus 30 inside the information processing device 3. Each component will be further described. The description of the communication unit 31, the storage unit 32, and the control unit 33 will be omitted because they are the same as the description of each unit in the server 2.

[0019] The display unit 34 may be included in the housing of the information processing device 3 or may be externally attached. The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. This is preferably implemented by selectively using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of the information processing device 3.

[0020] The input unit 35 may be included in the housing of the information processing device 3 or may be externally attached. The input unit 35 accepts voice and the like uttered by the user. The input unit 35 may be configured, for example, with a sound collector such as a microphone, and may collect external sound and output a sound signal indicating the collected sound. The sound signal is transferred as a command signal to the control unit 33 via the communication bus 30, and the control unit 33 may execute predetermined control or calculation as necessary. Furthermore, the information accepted by the input unit 35 is not limited to the above-mentioned voice and the like. Specifically, the input unit 35 may be configured to be integrated with the display unit 34 and be capable of accepting input from the user via a touch panel, switch buttons, a mouse, a QWERTY keyboard, or the like.

[0021] The output unit 36 ​​may be included in the housing of the information processing device 3 or may be externally attached. For example, it may be configured with a speaker or the like, and outputs voice, signal sounds, etc. generated by the information processing system 1.

[0022] The information processing device 3 may be a smartphone, a tablet terminal, a personal computer, a wearable device, or the like.

[0023] 2. Functional configuration of Server 2 The control unit 23 is configured to execute, for example, the following steps: The following steps can be optionally omitted.

[0024] The control unit 23 is configured to be able to receive information from the information processing device 3 or another device as a receiving step. The control unit 23 is also configured to be able to receive various pieces of information by reading out various pieces of information stored in a storage area that is at least a part of the memory unit 22 and writing the read out information in a working area that is at least a part of the memory unit 22. The storage area is, for example, an area of ​​the memory unit 22 that is implemented as a storage device such as an SSD. The working area is, for example, an area that is implemented as a memory such as a RAM. The receiving step may be the following first step or second receiving step. For example, the first receiving step may be a step of receiving voice information corresponding to a voice uttered by a user. The second receiving step may be a step of dividing a response from an artificial intelligence module into multiple segments and receiving the result.

[0025] In the input step, the control unit 23 inputs various information to the artificial intelligence module. The information includes a user's utterance. The information may be a prompt for the artificial intelligence module that is generated based on the user's utterance and reference information previously stored in the storage unit 22. The information may also include information other than the user's utterance.

[0026] In the generation step, the control unit 23 generates various information based on the response received from the artificial intelligence module via the communication unit 21 and the network. The information is video data including an animation of an avatar speaking and a synthetic voice corresponding to the response. In the generation step, the control unit 23 is configured to be able to generate the synthetic voice and animation. For example, in the generation step, the control unit 23 may divide the response from the artificial intelligence module into a plurality of segments and sequentially generate video data corresponding to each of the plurality of segments.

[0027] The control unit 23 outputs various information as an output step. The information can be output to the user via the display unit 34 or another device. In such a case, for example, the control unit 23 may control the display unit 34 to display visual information such as a screen, an image including a still image or a video, an icon, a message, etc. For example, the output step may be a step of outputting each piece of video data in sequence.

[0028] As a display control step, the control unit 23 controls the display unit 34 to display visual information such as a screen, an image including a still image or a video, an icon, a message, etc. The control unit 23 may generate only rendering information for displaying the visual information on the display unit 34. As a display control step, the control unit 23 may display an avatar in a manner that is visible to the user.

[0029] 3. Information processing flow This section describes the flow of an information processing method executed by the information processing system 1. As shown below, the information processing method includes each step executed by the information processing system. The information processing program of this embodiment causes a computer to execute each step of the information processing system. Note that the order of the processes can be changed as appropriate, multiple processes may be executed simultaneously, or some processes may be omitted.

[0030] 3.1 Overview FIG. 4 is a flowchart showing an outline of a process executed by the information processing system 1. In this process, first, the control unit 23 receives voice information corresponding to a voice uttered by a user as a receiving step (step S001). Next, the control unit 23 inputs a prompt including the voice information to the artificial intelligence module as an input step (step S002). Next, the control unit 23 divides a response from the artificial intelligence module into a plurality of segments as a generating step, and sequentially generates video data corresponding to each of the plurality of segments (step S003). Next, the control unit 23 outputs each of the video data sequentially as an output step (step S004). Here, since steps S003 and S004 are executed sequentially for each segment, while video data corresponding to the first segment is being output, video data corresponding to the second segment is being generated in parallel.

[0031] In summary, an information processing system according to one embodiment includes at least one processor, and the processor executes the following steps by reading a program. In a first receiving step, the control unit 23 receives voice information corresponding to a voice uttered by a user. In an input step, the control unit 23 inputs a prompt including the voice information to an artificial intelligence module. In a generating step, the control unit 23 divides a response from the artificial intelligence module into a plurality of segments and sequentially generates video data corresponding to each of the plurality of segments. The video data includes an animation of an avatar speaking the response and a synthesized voice corresponding to the response. In an output step, the control unit 23 sequentially outputs each piece of video data. The generating step and the output step are executed in parallel. According to this configuration, in an information processing system that causes an artificial intelligence module to generate a response in response to a voice received from a user and outputs video data of an avatar speaking the response to the user, because the response is divided into a plurality of segments and video data is generated for each segment, the time required to output the video data is shortened and usability is improved.

[0032] 3.2 Specific examples Hereinafter, details of the above information processing will be described as an example with reference to FIGS. 5 to 9. FIG. 5 is a diagram showing an example of how the information processing system 1 is used. FIG. 5 shows an information processing device 3 and a user Y. An avatar AV is displayed on the display unit 34 of the information processing device 3, and the user Y is speaking a voice 51. The control unit 33 of the information processing device 3 receives the voice 51 spoken by the user Y via the input unit 35. In this way, the information processing system 1 is a system in which, for example, the avatar AV replies with voice to the voice spoken by the user Y. By using an avatar, it is possible to provide the user Y with a sense of immersion that is not possible with a text-only conversation.

[0033] FIG. 6 is a conceptual diagram showing an example of the flow of information processing in the information processing system 1. When the control unit 23 receives the voice of user Y from the control unit 33, it inputs voice information corresponding to the received voice to the conversational AI. The conversational AI is an example of an artificial intelligence module. Here, the voice information may be information generated by performing voice recognition on the received voice. Next, the control unit 23 receives a response from the conversational AI and generates video data based on the response. The video data includes an animation of an avatar speaking based on the response, and a synthesized voice generated based on the response. Next, the control unit 23 outputs the video data.

[0034] FIG. 7 is an activity diagram showing an example of the flow of processing executed by the information processing system 1. This example of the flow may fall within the scope defined in the above-mentioned overview. The following description will be given along with each activity in this activity diagram. Note that the information processing may include any exception handling not shown. Exception handling includes the interruption of the information processing or the omission of each process. Selections or inputs made in the information processing may be based on user operation or may be made automatically without user operation.

[0035] First, the control unit 23 displays an avatar AV on the display unit 34 (activity A101). Here, the avatar AV may be a human figure, or an anthropomorphized animal or inanimate object. Preferably, the avatar AV is a realistic 3D model created to look exactly like a human and move in a human-like manner. A more human-like figure can enhance the user Y's sense of immersion and encourage deeper dialogue.

[0036] Next, as a first receiving step, the control unit 23 receives a voice uttered by the user Y via the input unit 35 (activity A102). In a more typical embodiment, the control unit 23 may convert the voice uttered by the user Y into text in real time. That is, the control unit 23 may input voice information corresponding to the voice of the user Y into a voice recognition model and convert the voice information into text data. The voice recognition model may be configured with elements such as a language model and an acoustic model, or a model that has been trained in advance using voice information and text data may be used.

[0037] As described above, when the control unit 23 converts user Y's speech into text in real time, if user Y's speech is interrupted, the control unit 23 may accept the user's speech that has been converted into text as speech information (activity A103). A case where user Y's speech is interrupted may be a case where user Y's speech is interrupted for a predetermined period of time. Specifically, for example, if user Y's speech is interrupted for 1 second or more, 2 seconds or more, 3 seconds or more, 4 seconds or more, or 5 seconds or more, the control unit 23 may determine that user Y's speech has been interrupted. With this configuration, user Y's speech can be quickly accepted as speech information when the user's speech is interrupted. Note that if an artificial intelligence module (described later) does not require a text prompt to output predetermined information, the conversion of user Y's speech into text (including the determination of a speech interruption) may be omitted. For example, the control unit 23 may input the speech itself to an artificial intelligence module having an internal voice recognition function.

[0038] Next, as an input step, the control unit 23 inputs a prompt including voice information to the artificial intelligence module (activity A104). The artificial intelligence module is typically a module that outputs a response to a prompt including voice information. Preferably, the artificial intelligence module is a module that outputs a response in natural language to a prompt including voice information. Furthermore, it is preferably a module that understands the content of user Y's utterance and responds according to that content. Therefore, the artificial intelligence module is preferably a conversational artificial intelligence module.

[0039] More typically, the artificial intelligence module may have a large-scale language model. That is, the artificial intelligence module may respond to input items based on the large-scale language model. Note that the large-scale language model is a deep learning model that pre-trains a language model that models human spoken language based on its occurrence probability from a huge amount of data. Note that the models that the artificial intelligence module can have are not limited to those described above.

[0040] Next, as a second receiving step, the control unit 23 divides the response from the artificial intelligence module into multiple segments and receives them (activity A105). By dividing the response into multiple segments at this stage, it is possible to shorten the time required for processing such as generating the corresponding video data in activity A106 and subsequent steps.

[0041] Here, if the artificial intelligence module outputs a response in streaming format, the control unit 23 may, in the second receiving step, divide the response transmitted from the artificial intelligence module in streaming format into a plurality of segments and receive the divided response. Outputting in streaming format means that the artificial intelligence module outputs a response to an input prompt in succession in stages. In such a case, the control unit 23 may divide the response that is output in stages into a plurality of segments and receive the divided response. In this manner, the control unit 23 can, in the second receiving step, receive the first segment of the response at an earlier timing than when all responses have been output.

[0042] The segments may be responses divided based on punctuation marks. Punctuation marks include periods, commas, periods, semicolons, colons, hyphens, and parentheses. Punctuation marks are used to organize sentences in an easy-to-understand manner and make them easier to read. In this manner, the control unit 23 can divide the responses into segments based on each clearly organized group.

[0043] Next, the control unit 23 generates video data corresponding to the response for each segment (activity A106) as a generation step. Here, the control unit 23 may generate video data corresponding to the received segments in order as the generation step. With this configuration, video data is generated in order from the received segments without waiting for the response from the artificial intelligence module until the end, thereby shortening the time required to generate video data corresponding to the first segment and improving usability.

[0044] Next, the control unit 23 outputs the video data for each segment in order as an output step (activity A107). Here, the video data includes an animation in which an avatar AV speaks a response and a synthesized voice corresponding to the response.

[0045] The animation of the avatar AV speaking may include at least one of the following: mouth movements, body movements, and facial expressions of the avatar AV, in accordance with the avatar AV's speech. This configuration further improves the realism of the animation of the avatar AV speaking, and further enhances the user's sense of immersion.

[0046] Next, the parallel execution of the generating step and the outputting step will be described in more detail with reference to FIGS.

[0047] FIG. 8 is a diagram showing an example of dividing reply 8 into multiple segments. Reply 8 includes segment S1, segment S2, segment S3, and segment Sn. Reply 8 is an example of a reply sent in streaming format from an artificial intelligence module. Segments S1 to Sn are reply 8 divided by periods.

[0048] FIG. 9 is an explanatory diagram illustrating the relationship between the generation and output of video data corresponding to each segment. The explanatory diagram shown as FIG. 9 illustrates the timing at which activities A105 to A107 are executed for each of segments S1 to Sn. When the control unit 23 first receives segment S1 from the artificial intelligence module, it generates video data AD1 corresponding to segment S1 and outputs the video data AD1 corresponding to segment S1. Here, the control unit 23 receives segment S2 following segment S1, generates video data AD1 corresponding to segment S1, and then generates video data AD2 corresponding to segment S2. Furthermore, the control unit 23 generates video data AD2 corresponding to segment S2, and then generates video data AD3 corresponding to segment S3. Thus, as a generation step, the control unit 23 generates video data ADn-1 corresponding to segment Sn-1, and then generates video data ADn corresponding to segment Sn. Meanwhile, as a generation step, the control unit 23 outputs video data AD1 corresponding to segment S1 in parallel with generating video data AD2 corresponding to segment S2.

[0049] However, the time required to generate the video data AD1 corresponding to the segment S1 is not necessarily the same as the time required to output the video data AD1 corresponding to the segment S1.

[0050] If the time required for generation is longer than the time required for output, generation of video data AD2 will not be completed when video data AD1 is output, resulting in a time gap between the output of video data AD1 and the start of output of video data AD2. Therefore, it is preferable that the control unit 23 generates the nth video data in the generation step so that the nth video data corresponding to the nth segment can be output consecutively with the n-1th video data corresponding to the n-1th segment. This configuration allows video data corresponding to multiple segments to be output continuously and without interruption. Because the video data is not interrupted, user Y can converse with the avatar AV without stress.

[0051] On the other hand, if the time required for generation is shorter than the time required for output, a situation may arise in which, when generation of video data AD2 is completed, output of video data AD1 has not yet been completed, and video data AD2 cannot be output immediately. In this embodiment, the control unit 23 may be configured such that, when the nth video data is generated in the generation step, if output of the (n-1)th video data has not yet been completed, the processor queues output of the nth video data in the output step. With this configuration, if the nth video data is generated before output of the (n-1)th video data, the data can be queued and then output in order.

[0052] As shown in FIG. 9, if the time required for generation is shorter than the time required for output, queuing Q2 occurs after generating video data AD2 corresponding to segment S2 and before outputting the video data AD2; queuing Q3 occurs after generating video data AD3 corresponding to segment S3 and before outputting the video data AD3; and queuing Qn occurs after generating video data ADn corresponding to segment Sn and before outputting the video data ADn. In such cases, the number of video data being queued may increase. Therefore, in the generation step, the control unit 23 may adjust the generation speed for generating video data so as to optimize the number of video data being queued. With this configuration, video data is generated in accordance with the output speed, thereby optimizing resource utilization.

[0053] 5. Variations Furthermore, the following aspects may be adopted: The above-described information processing aspects are merely examples, and the present invention is not limited to these, and can be modified as appropriate within the scope of the technical concept of the invention.

[0054] In the above embodiment, in activity A102, the control unit 23 receives the voice uttered by the user Y as the first receiving step, but the control unit 23 may also receive data other than the voice information and input a prompt including the voice information and the data other than the voice information to the artificial intelligence module as the input step. With this configuration, it is possible to receive information other than voice from the user and to have a conversation regarding the information other than voice.

[0055] Here, data other than voice information includes image data, text data, and data acquired by sensors such as temperature and humidity sensors. For example, when image data is received along with a user's voice saying, "I'd like your opinion on this photo," a response including the user's opinion on the photo can be provided. When text data is received along with a user's voice saying, "I'd like your thoughts on this article," a response can be provided with the user's thoughts on the article. When a user's voice saying, "It's cold, isn't it?" is received, a response can be provided based on information acquired from a temperature sensor. In such cases, a multimodal AI module can be used as the AI ​​module. A multimodal AI module is an AI module that integrates and processes data in multiple different formats, such as text, voice, images, and videos. By using a multimodal AI module, it is possible to, for example, associate images with voice and then provide a response.

[0056] In the above embodiment, it has been described that an avatar is displayed on the display unit 34 in activity A101, but the avatar does not have to be displayed on the display unit 34 at the stage of activity A101. The control unit 23 may output an avatar in response to an utterance from the user.

[0057] In the above embodiment, it is described that user Y's speech is interrupted in activity A102, but the control unit 23 may determine whether the user's speech is interrupted by using a voice recognition system to determine the context of user Y's speech, and if it determines that a series of questions from user Y has been interrupted, determine that user Y's speech is interrupted.

[0058] In the activity A104 of the above embodiment, the control unit 23 may include a requirement regarding the language to be output by the AI ​​module when inputting voice information to the AI ​​module as an input step. In this manner, a response can be obtained in a language different from the language input to the AI ​​module. The language requirement may be specified, for example, by a language tag.

[0059] Furthermore, in activity A105, when the control unit 23 divides the response from the artificial intelligence module into multiple segments and accepts the response as the second accepting step, the control unit 23 may accept the response by dividing the response into multiple segments according to the language of the response. In other words, the control unit 23 may divide the response by punctuation marks according to the language of the response.

[0060] The overall configuration shown in Fig. 1 is an example and is not limited to this. For example, the server 2 may be distributed across two or more devices, or may be replaced by a cloud computing system. Furthermore, all processing may be performed by the server 2, or all processing may be performed by the information processing device 3. An application may be installed on the information processing device 3, and the information processing device 3 and the server 2 may work together to execute the processing described above.

[0061] The server 2 may be an on-premise server or a cloud server. The cloud server 2 may provide the above functions and processes in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0062] In the above embodiment, the server 2 performs various storage and control operations, but multiple external devices may be used instead of the server 2. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0063] It may be provided in the following manner.

[0064] (1) An information processing system comprising at least one processor, the processor being configured to execute each of the following steps by reading a program: a first receiving step receiving voice information corresponding to a voice uttered by a user; an input step inputting a prompt including the voice information into an artificial intelligence module; a generation step dividing a response from the artificial intelligence module into a plurality of segments and sequentially generating video data corresponding to each of the plurality of segments, wherein the video data includes an animation of an avatar speaking the response and a synthesized voice corresponding to the response; and an output step outputting each of the video data sequentially, the generation step and the output step being executed in parallel.

[0065] According to this configuration, in an information processing system in which an artificial intelligence module generates a response to voice input from a user and outputs video data of an avatar speaking that response to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time required to output the video data and improving usability.

[0066] (2) In the information processing system described in (1) above, in the first reception step, the user's voice is converted into text in real time, and when the user's voice is interrupted, the user's voice that has been converted into text up to that point is accepted as the voice information.

[0067] According to this configuration, the user's voice can be quickly accepted as voice information at the timing when the user finishes speaking.

[0068] (3) In the information processing system described in (1) or (2) above, further, in the second receiving step, the response transmitted in streaming format from the artificial intelligence module is divided into the plurality of segments and received, and in the generating step, video data corresponding to the segments is generated in order from the received segments.

[0069] With this configuration, the response from the artificial intelligence module is not waited for until the end, but is divided into multiple segments and accepted, and video data is generated in order from the accepted segments, thereby shortening the time required to generate video data corresponding to the first segment and improving usability.

[0070] (4) In the information processing system described in any one of (1) to (3) above, the generation step generates the nth video data so that the nth video data corresponding to the nth segment can be output consecutively with the n-1th video data corresponding to the n-1th segment.

[0071] With this configuration, video data can be output continuously, and since there is no interruption in the video data, the user can converse with the avatar without stress.

[0072] (5) In the information processing system described in (4) above, when the nth video data is generated in the generation step, if the output of the n-1th video data has not been completed, the processor queues the output of the nth video data in the output step.

[0073] According to this configuration, if the n-th video data is generated before the (n-1)-th video data is output, the data can be temporarily queued and then output in order.

[0074] (6) In the information processing system described in (5) above, in the generating step, a generation speed for generating the video data is adjusted so as to optimize the number of video data in the queue.

[0075] According to this configuration, video data is generated in accordance with the output speed, thereby optimizing resource usage.

[0076] (7) In the information processing system according to any one of (1) to (6) above, the segments are obtained by dividing the response based on punctuation marks.

[0077] With this configuration, the response received from the artificial intelligence module can be divided into segments for each group of meanings.

[0078] (8) In the information processing system described in any one of (1) to (7) above, the animation includes at least one of the avatar's mouth movements, body movements, and facial expressions in accordance with the avatar's speech.

[0079] This configuration further improves the realism of the animation of the avatar speaking, further increasing the user's sense of immersion.

[0080] (9) In the information processing system described in any one of (1) to (8) above, in the first receiving step, data different from the voice information is further received, and in the input step, the prompt including the voice information and the data different from the voice information is input to the artificial intelligence module.

[0081] According to this configuration, it is possible to receive information other than voice from the user and to have a conversation regarding the information other than voice.

[0082] (10) An information processing method, comprising the steps of the information processing system according to any one of (1) to (9) above.

[0083] According to this configuration, in an information processing system in which an artificial intelligence module generates a response to voice input from a user and outputs video data of an avatar speaking that response to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time required to output the video data and improving usability.

[0084] (11) A program that causes at least one computer to execute each step in the information processing system according to any one of (1) to (9) above.

[0085] According to this configuration, in an information processing system in which an artificial intelligence module generates a response to voice input from a user and outputs video data of an avatar speaking that response to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time required to output the video data and improving usability. Of course, this is not the case.

[0086] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the accompanying claims. [Explanation of symbols]

[0087] 1: Information processing system 2: Server 20: Communication bus 21: Communications Department 22: Storage section 23: Control section 3: Information processing equipment 30: Communication bus 31: Communications Department 32: Storage section 33: Control section 34: Display section 35: Input section 36: Output section 51: Audio 8: Reply 9: Explanatory diagram AD: Video data AD1: Video data AD2: Video data AD3: Video data ADn: Video data AV:Avatar Q2: Queuing Q3: Queuing Qn: Queuing S1: Segment S2: Segment S3: Segment Sn: Segment Y: User

Claims

1. An information processing system, At least one processor is provided, the processor being configured to execute the following steps by reading a program: In the first receiving step, voice information corresponding to a voice uttered by the user is received; In the input step, a prompt including the voice information is input to an artificial intelligence module; the generating step divides the response from the artificial intelligence module into a plurality of segments, and generates video data corresponding to each of the plurality of segments in order, wherein the video data includes an animation of an avatar uttering the response and a synthesized voice corresponding to the response; In the output step, each of the video data is output in order; The generating step and the outputting step are performed in parallel.

2. 2. The information processing system according to claim 1, In the first receiving step, the system converts the user's voice into text in real time, and when the user's voice is interrupted, accepts the user's voice that has been converted into text up to that point as the voice information.

3. 2. The information processing system according to claim 1, Furthermore, in the second receiving step, the response transmitted in a streaming format from the artificial intelligence module is divided into the plurality of segments and received; In the generating step, video data corresponding to the received segments is generated in order from the received segments.

4. 2. The information processing system according to claim 1, In the generation step, the system generates the nth video data so that the nth video data corresponding to the nth segment can be output consecutively with the n-1th video data corresponding to the n-1th segment.

5. 5. The information processing system according to claim 4, A system in which, when the nth video data is generated in the generation step, if the output of the n-1th video data has not been completed, the processor queues the output of the nth video data in the output step.

6. 6. The information processing system according to claim 5, In the generating step, the system adjusts a generation rate at which the video data is generated so as to optimize the number of video data in the queue.

7. 2. The information processing system according to claim 1, The segments are divisions of the response based on punctuation marks.

8. 2. The information processing system according to claim 1, The system, wherein the animation includes at least one of mouth movements, body movements, and facial expressions of the avatar in sync with the avatar's speech.

9. 2. The information processing system according to claim 1, In the first receiving step, data different from the voice information is further received, In the input step, the prompt including the voice information and the data different from the voice information is input to the artificial intelligence module.

10. An information processing method, comprising: A method comprising the steps of the information processing system according to any one of claims 1 to 9.

11. A program, A program that causes at least one computer to execute each step in the information processing system according to any one of claims 1 to 9.

Images