Information processing system, information processing method, and program

Abstract

An information processing system, an information processing method, and a program are provided that shorten the time required to output video data and improve usability. [Solution] In an information processing system having at least one processor, a method executed by the processor reading a program includes a first receiving step (a step of receiving voice information) of receiving voice information corresponding to a voice uttered by a user, an input step of inputting a prompt including the voice information to an artificial intelligence module (conversational AI), and a generating step of dividing a response from the artificial intelligence module into a plurality of segments and generating video data corresponding to each of the plurality of segments in sequence. The video data includes an animation of an avatar speaking the response and a synthetic voice corresponding to the response. Furthermore, an output step executed in parallel with the generating step of outputting each of the video data in sequence.

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 is a document related to efficiently obtaining a reply message expected by a user when the user sends a chat message related to a question. When a multi-cloud chat service providing device 10 shown in Patent Document 1 receives a question message from a user terminal 20, the multi-cloud chat service providing device 10 transmits the question message to an AI chat cloud service system 30 and receives a reply message to the question message from the AI ​​chat cloud service system 30. When the multi-cloud chat service providing device 10 determines that a predetermined condition is satisfied, the multi-cloud chat service providing device 10 transmits the question message to an operator terminal 40A operated by an operator and receives a reply message to the question message from the operator terminal 40A. Then, the multi-cloud chat service providing device 10 replies to the user terminal 20 with the reply message received from the AI ​​chat cloud service system 30 or the operator terminal 40A. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-128737 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is still room for improvement in the technology of AI-based response systems that respond to user utterances. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system. The information processing system includes at least one processor, and the processor is configured to execute the following steps by reading a program. In a first receiving step, voice information corresponding to a voice uttered by a user is received. In an input step, a prompt including the voice information is input to an artificial intelligence module. In a generating step, a response from the artificial intelligence module is divided into a plurality of segments, and video data corresponding to each of the plurality of segments is generated in sequence. The video data includes an animation in which an avatar speaks the response, and a synthetic voice corresponding to the response. In an output step, each of the video data is output in sequence. The generating step and the output step are executed in parallel.

[0006] According to one aspect of the present invention, it is possible to provide an information processing system etc. which is more beneficial to the user. [Brief description of the drawings]

[0007] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the server 2. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of an information processing device 3. [Figure 4] 2 is a flowchart showing an overview of processing executed by the information processing system 1. [Diagram 5] FIG. 2 is a diagram showing an example of a usage mode of the information processing system 1. [Figure 6] 2 is a conceptual diagram showing an example of the flow of information processing in the information processing system 1. FIG. [Figure 7] 2 is an activity diagram showing an example of a flow of processing executed by the information processing system 1. FIG. [Figure 8] FIG. 13 is a diagram showing an example of dividing a reply 8 into multiple segments. [Figure 9] FIG. 9 is an explanatory diagram for explaining the relationship between generation and output of video data corresponding to each segment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0009] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, in this embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0011] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0012] 1. Hardware Configuration This section describes the hardware configuration.

[0013] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes a server 2 and an information processing device 3. The server 2 and the information processing device 3 are configured to be able to communicate with each other through an electric communication line (network). Here, the system exemplified as the information processing system 1 includes one or more devices or components. Therefore, it should be noted that the information processing system 1 includes the server 2 alone or both the server 2 and the information processing device 3. More specifically, the information processing system 1 may include an element selected from the group consisting of the server 2 and the information processing device 3. The unselected element may not be included in the information processing system 1, but may be electrically connected to the selected element as an external element. These components will be described below.

[0014] <Server 2> 2 is a block diagram showing a hardware configuration of the server 2. The server 2 includes a communication unit 21, a storage unit 22, and a control unit 23, and these components are electrically connected via a communication bus 20 inside the server 2. Each component will be further described.

[0015] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the server 2 may communicate various information from the outside via the communication unit 21 and the network.

[0016] The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the server 2 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. The storage unit 22 stores various programs, variables, etc. related to the server 2 executed by the control unit 23.

[0017] The control unit 23 performs processing and control of the overall operation related to the server 2. The control unit 23 is, for example, a central processing unit (CPU) not shown. The control unit 23 realizes various functions related to the server 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by the software stored in the storage unit 22 is specifically realized by the control unit 23, which is an example of hardware, so that each step related to each function described below can be executed. This will be described in further detail in the next section. Note that the control unit 23 is not limited to being single, and may be implemented with multiple control units 23 for each function. Also, a combination of these may be used.

[0018] <Information processing device 3> 3 is a block diagram showing a hardware configuration of the information processing device 3. The information processing device 3 includes a communication unit 31, a storage unit 32, a control unit 33, a display unit 34, an input unit 35, and an output unit 36, and these components are electrically connected via a communication bus 30 inside the information processing device 3. Each component will be further described. The description of the communication unit 31, the storage unit 32, and the control unit 33 will be omitted because they are similar to the description of each unit in the server 2.

[0019] The display unit 34 may be included in the housing of the information processing device 3 or may be externally attached. The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display according to the type of the information processing device 3.

[0020] The input unit 35 may be included in the housing of the information processing device 3 or may be externally attached. The input unit 35 accepts voice and the like uttered by the user. The input unit 35 may be configured, for example, by a sound collector such as a microphone, and may collect external sound and output a sound signal indicating the collected sound. The sound signal is transferred as a command signal to the control unit 33 via the communication bus 30, and the control unit 33 may execute a predetermined control or calculation as necessary. In addition, the information accepted by the input unit 35 is not limited to the above-mentioned voice and the like. Specifically, the input unit 35 may be configured to be integrated with the display unit 34 and to be capable of accepting input from the user via a touch panel, a switch button, a mouse, a QWERTY keyboard, or the like.

[0021] The output unit 36 ​​may be included in the housing of the information processing device 3 or may be externally attached. For example, the output unit 36 ​​is configured with a speaker or the like, and outputs voice, signal sound, or the like generated by the information processing system 1.

[0022] The information processing device 3 may be a smartphone, a tablet terminal, a personal computer, a wearable device, or the like.

[0023] 2. Functional configuration of Server 2 The control unit 23 is configured to execute, for example, the following steps. The following steps can be optionally omitted.

[0024] The control unit 23 is configured to be able to receive information from the information processing device 3 or other devices as a receiving step. The control unit 23 is also configured to be able to receive various information by reading out various pieces of information stored in a storage area that is at least a part of the memory unit 22 and writing the read out information in a working area that is at least a part of the memory unit 22. The storage area is, for example, an area of ​​the memory unit 22 that is implemented as a storage device such as an SSD. The working area is, for example, an area that is implemented as a memory such as a RAM. The receiving step may be the following first step or second receiving step. For example, the first receiving step may be a step of receiving voice information corresponding to a voice uttered by the user. The second receiving step may be a step of dividing the response of the artificial intelligence module into a plurality of segments and receiving the response.

[0025] As an input step, the control unit 23 inputs various information to the artificial intelligence module. The information includes the user's utterance. The information may be a prompt for the artificial intelligence module that is generated based on the user's utterance and reference information previously stored in the storage unit 22. The information may include information other than the user's utterance.

[0026] As a generation step, the control unit 23 generates various information based on the response received from the artificial intelligence module via the communication unit 21 and the network. The information is video data including an animation of an avatar speaking and a synthetic voice corresponding to the response. As a generation step, the control unit 23 is configured to be able to generate a synthetic voice and an animation. For example, as a generation step, the control unit 23 may divide the response from the artificial intelligence module into a plurality of segments and generate video data corresponding to each of the plurality of segments in sequence.

[0027] The control unit 23 outputs various information as an output step. The information can be output to the user via the display unit 34 or another device. In this case, for example, the control unit 23 may control the display unit 34 to display visual information such as a screen, an image including a still image or a video, an icon, a message, etc. For example, the output step may be a step of outputting each of the video data in sequence.

[0028] As a display control step, the control unit 23 controls the display unit 34 to display visual information such as a screen, an image including a still image or a video, an icon, a message, etc. The control unit 23 may generate only rendering information for displaying the visual information on the display unit 34. As a display control step, the control unit 23 may display an avatar in a manner that is visible to the user.

[0029] 3. Information processing flow In this section, the flow of the information processing method executed by the information processing system 1 will be described. As shown below, the information processing method includes each step executed by the information processing system. The information processing program of this embodiment causes a computer to execute each step of the information processing system. The order of the processes can be changed as appropriate, multiple processes can be executed simultaneously, or some processes can be omitted.

[0030] 3.1 Overview FIG. 4 is a flowchart showing an outline of the process executed by the information processing system 1. In this process, first, the control unit 23 receives voice information corresponding to the voice uttered by the user as a receiving step (step S001). Next, the control unit 23 inputs a prompt including the voice information to the artificial intelligence module as an input step (step S002). Next, the control unit 23 divides the response from the artificial intelligence module into a plurality of segments as a generating step, and generates video data corresponding to each of the plurality of segments in order (step S003). Next, the control unit 23 outputs each of the video data in order as an output step (step S004). Here, since steps S003 and S004 are executed in order for each segment, while the video data corresponding to the first segment is being output, the video data corresponding to the second segment is generated in parallel.

[0031] In summary, an information processing system according to an embodiment includes at least one processor, and the processor executes the following steps by reading a program. The control unit 23 receives voice information corresponding to a voice uttered by a user as a first receiving step. The control unit 23 inputs a prompt including voice information to the artificial intelligence module as an input step. The control unit 23 divides a response from the artificial intelligence module into a plurality of segments as a generating step, and generates video data corresponding to each of the plurality of segments in order. The video data includes an animation in which an avatar speaks the response, and a synthetic voice corresponding to the response. The control unit 23 outputs each of the video data in order as an output step. The generating step and the output step are executed in parallel. According to this configuration, in an information processing system in which an artificial intelligence module generates a response to a voice received from a user, and video data in which an avatar speaks the response is output to the user, the response is divided into a plurality of segments, and video data is generated for each segment, so that the time required to output the video data is shortened and usability is improved.

[0032] 3.2 Specific examples Hereinafter, the details of the above information processing will be described as an example with reference to Figs. 5 to 9. Fig. 5 is a diagram showing an example of a usage mode of the information processing system 1. Fig. 5 shows an information processing device 3 and a user Y. An avatar AV is displayed on the display unit 34 of the information processing device 3, and the user Y is speaking a voice 51. The control unit 33 of the information processing device 3 receives the voice 51 spoken by the user Y via the input unit 35. In this way, the information processing system 1 is a system in which, for example, the avatar AV responds with a voice to the voice uttered by the user Y. By using an avatar, it is possible to provide the user Y with a sense of immersion that is not possible with a conversation that is only text.

[0033] FIG. 6 is a conceptual diagram showing an example of the flow of information processing of the information processing system 1. When the control unit 23 receives the voice of the user Y from the control unit 33, it inputs voice information corresponding to the received voice to the conversational AI. The conversational AI is an example of an artificial intelligence module. Here, the voice information may be information generated by performing voice recognition on the received voice. Next, the control unit 23 receives a response from the conversational AI, and generates video data based on the response. The video data includes an animation in which an avatar generated based on the response speaks, and a synthetic voice generated based on the response. Next, the control unit 23 outputs the video data.

[0034] FIG. 7 is an activity diagram showing an example of the flow of processing executed by the information processing system 1. This example of the flow may fall within the scope defined in the above-mentioned overview. Below, an explanation will be given along with each activity of this activity diagram. The information processing may include any exception processing not shown. Exception processing includes interruption of the information processing and omission of each process. Selection or input performed in the information processing may be based on a user operation or may be performed automatically without relying on a user operation.

[0035] First, the control unit 23 causes the display unit 34 to display an avatar AV (activity A101). Here, the avatar AV may be a human figure, or an anthropomorphized animal or inanimate object. Preferably, the avatar AV is a realistic 3D model created to look exactly like a human and move in a human-like manner. A more human-like appearance can enhance the user Y's sense of immersion and encourage deeper conversation.

[0036] Next, as a first receiving step, the control unit 23 receives the voice uttered by the user Y via the input unit 35 (activity A102). In a more typical embodiment, the control unit 23 may convert the voice uttered by the user Y into text in real time. That is, the control unit 23 may input voice information corresponding to the voice of the user Y into a voice recognition model and convert the voice information into text data. The voice recognition model may be configured with elements such as a language model and an acoustic model, and a model previously trained using voice information and text data may be used.

[0037] As described above, when the control unit 23 converts the user Y's speech into text in real time, the control unit 23 may accept the user's speech that has been converted into text as speech information when the user Y's speech is interrupted (activity A103). The case where the user Y's speech is interrupted may be a case where the speech from the user Y is interrupted for a predetermined period of time. Specifically, for example, when the speech from the user Y is interrupted for 1 second or more, 2 seconds or more, 3 seconds or more, 4 seconds or more, or 5 seconds or more, the control unit 23 may determine that the user Y's speech is interrupted. According to this configuration, the voice of the user Y can be quickly accepted as speech information at the timing when the user's speech is interrupted. Note that, when the artificial intelligence module described later does not require a text-converted prompt to output predetermined information, the conversion of the user Y's speech into text (including the determination of interruption of speech, etc.) may be omitted. The control unit 23 may input the voice itself to an artificial intelligence module having a voice recognition function therein, for example.

[0038] Next, as an input step, the control unit 23 inputs a prompt including voice information to the artificial intelligence module (activity A104). The artificial intelligence module is typically a module that outputs a response to a prompt including voice information. Preferably, the artificial intelligence module is a module that outputs a response in natural language to a prompt including voice information. Furthermore, it is preferably a module that understands the content of user Y's utterance and responds according to that content. Therefore, the artificial intelligence module is preferably a conversational artificial intelligence module.

[0039] More typically, the artificial intelligence module may have a large-scale language model. That is, the artificial intelligence module may respond to an input item based on the large-scale language model. Note that the large-scale language model is a deep learning model that pre-learns from a huge amount of data about a language model that models human spoken words based on their occurrence probability. Note that the models that the artificial intelligence module can have are not limited to the above.

[0040] Next, as a second receiving step, the control unit 23 divides the response from the artificial intelligence module into multiple segments and receives them (activity A105). By dividing the response into multiple segments at this stage, it is possible to shorten the time required for processing such as generating the corresponding video data in activity A106 and subsequent activities.

[0041] Here, when the artificial intelligence module outputs a response in streaming format, the control unit 23 may, as a second receiving step, divide the response transmitted from the artificial intelligence module in streaming format into a plurality of segments and receive it. Outputting in streaming format means, in other words, that the artificial intelligence module outputs a response to an input prompt in succession in stages. In such a case, the control unit 23 may receive the response that is output in stages by dividing it into a plurality of segments. With this aspect, the control unit 23 can receive the first segment of the response as a second receiving step at a timing earlier than when the control unit 23 receives the response after all responses have been output.

[0042] The segments may be responses divided based on punctuation marks. Punctuation marks include periods, commas, periods, semicolons, colons, hyphens, parentheses, etc. Punctuation marks are used to organize sentences in an easy-to-understand manner and make them easier to read. In this manner, the control unit 23 can divide the responses into segments for each group that is organized in an easy-to-understand manner.

[0043] Next, the control unit 23 generates video data corresponding to the response for each segment as a generation step (activity A106). Here, the control unit 23 may generate video data corresponding to the received segments in order as a generation step. According to such a configuration, video data is generated in order from the received segments without waiting for the response from the artificial intelligence module until the end, so that the time required to generate video data corresponding to the first segment is shortened, and usability is improved.

[0044] Next, the control unit 23 outputs the video data for each segment in order as an output step (activity A107). Here, the video data includes an animation in which an avatar AV speaks a response, and a synthesized voice corresponding to the response.

[0045] The animation of the avatar AV speaking may include at least one of the mouth movement, body movement, and facial expression of the avatar AV in accordance with the speech of the avatar AV. With this configuration, the reality of the animation of the avatar AV speaking is further improved, and the user's sense of immersion is further improved.

[0046] Next, the parallel execution of the generating step and the outputting step will be described in more detail with reference to FIGS.

[0047] FIG. 8 is a diagram showing an example of dividing reply 8 into a plurality of segments. Reply 8 includes segment S1, segment S2, segment S3, and segment Sn. Reply 8 is an example of a reply transmitted in streaming format from an artificial intelligence module. Segments S1 to Sn are reply 8 divided by periods.

[0048] FIG. 9 is an explanatory diagram 9 for explaining the relationship between the generation and output of video data corresponding to each segment. The explanatory diagram 9 shown as FIG. 9 explains the timing of execution of activities A105 to A107 for each of segments S1 to Sn. When the control unit 23 first receives a segment S1 from the artificial intelligence module, it generates video data AD1 corresponding to the segment S1 and outputs the video data AD1 corresponding to the segment S1. Here, the control unit 23 receives a segment S2 following the segment S1, and generates video data AD1 corresponding to the segment S1, and then generates video data AD2 corresponding to the segment S2. The control unit 23 further generates video data AD2 corresponding to the segment S2, and then generates video data AD3 corresponding to the segment S3. In this way, the control unit 23 generates video data ADn-1 corresponding to the segment Sn-1 as a generation step, and then generates video data ADn corresponding to the segment Sn. Meanwhile, the control unit 23 outputs video data AD1 corresponding to the segment S1 in parallel with generating video data AD2 corresponding to the segment S2 as a generation step.

[0049] However, the time required to generate the video data AD1 corresponding to the segment S1 and the time required to output the video data AD1 corresponding to the segment S1 are not necessarily the same.

[0050] If the time required for generation is longer than the time required for output, the generation of the video data AD2 is not completed when the video data AD1 is output, so there will be a time gap between the output of the video data AD1 and the start of output of the video data AD2. Therefore, it is preferable that the control unit 23 generates the n-th video data as a generation step so that the n-th video data corresponding to the n-th segment can be output consecutively with the n-1-th video data corresponding to the n-1-th segment. With this configuration, video data corresponding to multiple segments can be output consecutively without interruption. Since the video data is not interrupted, the user Y can talk to the avatar AV without stress.

[0051] On the other hand, if the time required for generation is shorter than the time required for output, a situation may occur in which when generation of video data AD2 is completed, output of video data AD1 is not completed, and video data AD2 cannot be output immediately. In this embodiment, the control unit 23 may be configured such that when the nth video data is generated in the generation step, if output of the (n-1)th video data is not completed, the processor queues output of the nth video data in the output step. With this configuration, if the nth video data is generated before output of the (n-1)th video data, it can be queued and then output in order.

[0052] However, as shown in FIG. 9, when the time required for generation is shorter than the time required for output, after generating video data AD2 corresponding to segment S2, queuing Q2 occurs before outputting video data AD2, after generating video data AD3 corresponding to segment S3, queuing Q3 occurs before outputting video data AD3, and after generating video data ADn corresponding to segment Sn, queuing Qn occurs before outputting video data ADn. In such a case, the number of video data being queued may increase. Therefore, as a generation step, the control unit 23 may adjust the generation speed of generating video data so as to optimize the number of video data being queued. According to such a configuration, video data is generated in accordance with the output speed, so that resources can be optimized.

[0053] 5. Variations Furthermore, the following aspects may be adopted. The above-described information processing aspects are merely examples. The present invention is not limited to these aspects, and can be modified as appropriate without departing from the technical spirit of the invention.

[0054] In the above embodiment, in activity A102, the control unit 23 receives the voice uttered by the user Y as the first receiving step, but the control unit 23 may further receive data other than the voice information and input a prompt including the voice information and the data other than the voice information to the artificial intelligence module as the input step. With such a configuration, it is possible to receive information other than the voice from the user and have a conversation regarding the information other than the voice.

[0055] Here, the data different from the voice information includes image data, text data, data acquired by a sensor such as temperature and humidity data, etc. For example, when image data is received together with a voice from a user saying "I want to hear your opinion on this photo," a reply including the opinion on the photo can be made, when text data is received together with a voice from a user saying "I want your thoughts on reading this text," a reply can be made with the user's thoughts on the text, and when a voice from a user saying "It's cold, isn't it?" is received, a reply can be made based on information acquired by a temperature sensor. In such a case, it is preferable to use a multimodal type artificial intelligence module as the artificial intelligence module. A multimodal type artificial intelligence module is an artificial intelligence module that integrates and processes data in a plurality of different formats such as text, voice, image, and video. By using a multimodal type artificial intelligence module, for example, it is possible to associate an image with a voice and then reply.

[0056] In the above embodiment, it has been described that an avatar is displayed on the display unit 34 in the activity A101, but the avatar does not have to be displayed on the display unit 34 at the stage of the activity A101. The control unit 23 may output an avatar in response to an utterance from the user.

[0057] In the above embodiment, it is described that user Y's speech is interrupted in activity A102, but the control unit 23 may determine whether the user's speech is interrupted by using a voice recognition system to determine the context of user Y's speech, and if it determines that a series of questions from user Y has been interrupted, determine that user Y's speech has been interrupted.

[0058] In the activity A104 of the above embodiment, the control unit 23 may include a requirement regarding the language to be output by the artificial intelligence module when inputting the voice information to the artificial intelligence module as an input step. In this manner, a response can be obtained in a language different from the language input to the artificial intelligence module. The requirement regarding the language may be specified by, for example, a language tag.

[0059] Furthermore, in activity A105, when the control unit 23 divides the response from the artificial intelligence module into a plurality of segments and accepts the response as the second accepting step, the control unit 23 may accept the response by dividing the response into a plurality of segments according to the language of the response. In other words, the control unit 23 may divide the response by punctuation marks according to the language of the response.

[0060] The overall configuration shown in Fig. 1 is an example, and is not limited thereto. For example, the server 2 may be distributed to two or more devices, or may be replaced by a cloud computing system. In addition, all processing may be performed by the server 2, or all processing may be performed by the information processing device 3. An application may be installed in the information processing device 3, and the information processing device 3 and the server 2 may cooperate to execute the above-mentioned processing.

[0061] The server 2 may be in an on-premise form or in a cloud form. As the server 2 in a cloud form, the above functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0062] In the above embodiment, the server 2 performs various storage and control, but multiple external devices may be used instead of the server 2. In other words, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0063] It may be provided in any of the following ways:

[0064] (1) An information processing system comprising at least one processor, the processor being configured to execute each of the following steps by reading a program, wherein a first receiving step receives voice information corresponding to a voice uttered by a user, an input step inputs a prompt including the voice information to an artificial intelligence module, a generating step divides a response from the artificial intelligence module into a plurality of segments and sequentially generates video data corresponding to each of the plurality of segments, wherein the video data includes an animation of an avatar speaking the response and a synthesized voice corresponding to the response, and an output step outputs each of the video data in sequence, the generating step and the output step being executed in parallel.

[0065] According to such a configuration, in an information processing system in which an artificial intelligence module generates a response to voice received from a user and video data of an avatar speaking the response is output to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time it takes to output the video data and improving usability.

[0066] (2) In the information processing system described in (1) above, in the first reception step, the user's voice is converted into text in real time, and when the user's voice is interrupted, the user's voice that has been converted into text up to that point is accepted as the voice information.

[0067] According to such a configuration, the user's voice can be quickly accepted as voice information at the timing when the user finishes speaking.

[0068] (3) In the information processing system described in (1) or (2) above, further, in a second receiving step, the response transmitted in streaming format from the artificial intelligence module is divided into the plurality of segments and received, and in the generating step, video data corresponding to the segments is generated in order from the received segments.

[0069] With this configuration, the artificial intelligence module does not wait until the end for a response, but instead accepts the data divided into multiple segments and generates video data in the order of the accepted segments, thereby shortening the time it takes to generate video data corresponding to the first segment and improving usability.

[0070] (4) In an information processing system described in any one of (1) to (3) above, in the generation step, the nth video data is generated so that the nth video data corresponding to the nth segment can be output consecutively to the n-1th video data corresponding to the n-1th segment.

[0071] According to this configuration, the video data can be output continuously, and since there is no interruption in the video data, the user can converse with the avatar without stress.

[0072] (5) In the information processing system described in (4) above, when the nth video data is generated in the generation step, if the output of the n-1th video data has not been completed, the processor queues the output of the nth video data in the output step.

[0073] According to this configuration, if the n-th video data is generated before the (n-1)-th video data is output, the data can be temporarily queued and then output in order.

[0074] (6) In the information processing system described in (5) above, in the generating step, a generation speed for generating the video data is adjusted so as to optimize the number of video data in the queue.

[0075] According to such a configuration, video data is generated in accordance with the output speed, making it possible to optimize resources.

[0076] (7) In the information processing system according to any one of (1) to (6) above, the segments are formed by dividing the reply based on punctuation marks.

[0077] According to this configuration, the reply received from the artificial intelligence module can be divided into segments according to the unit of meaning.

[0078] (8) In the information processing system described in any one of (1) to (7) above, the animation includes at least one of the avatar's mouth movements, body movements, and facial expressions in accordance with the avatar's speech.

[0079] With this configuration, the realism of the animation of the avatar speaking is improved, and the user's sense of immersion is further enhanced.

[0080] (9) In an information processing system described in any one of (1) to (8) above, in the first receiving step, data different from the voice information is further received, and in the input step, the prompt including the voice information and the data different from the voice information is input to the artificial intelligence module.

[0081] According to such a configuration, it is possible to receive information other than voice from the user and to have a conversation regarding the information other than voice.

[0082] (10) An information processing method, comprising each step in the information processing system according to any one of (1) to (9) above.

[0083] According to such a configuration, in an information processing system in which an artificial intelligence module generates a response to voice received from a user and video data of an avatar speaking the response is output to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time it takes to output the video data and improving usability.

[0084] (11) A program that causes at least one computer to execute each step in the information processing system according to any one of (1) to (9) above.

[0085] According to such a configuration, in an information processing system in which an artificial intelligence module generates a response to voice received from a user and video data of an avatar speaking the response is output to the user, the response is divided into multiple segments and video data is generated for each segment, thereby shortening the time it takes to output the video data and improving usability. Of course, this is not the case.

[0086] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are included within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0087] 1: Information processing system 2: Server 20: Communication bus 21: Communications Department 22: Storage section 23: Control section 3: Information processing equipment 30: Communication bus 31: Communications Department 32: Storage section 33: Control section 34: Display section 35: Input section 36: Output section 51: Audio 8: Reply 9:Explanatory diagram AD:Video data AD1: Video data AD2: Video data AD3: Video data ADn: Video data AV:Avatar Q2: Queuing Q3: Queuing Qn: Queuing S1: Segment S2: Segment S3: Segment Sn: Segment Y :User

Claims

1. An information processing system, The method includes at least one processor, the processor being configured to execute the following steps by reading a program: In the first receiving step, voice information corresponding to a voice uttered by a user is received; In the input step, a prompt including the voice information is input to an artificial intelligence module; In the generating step, the response from the artificial intelligence module is divided into a plurality of segments, and video data corresponding to each of the plurality of segments is generated in sequence, where the video data includes an animation of an avatar speaking the response and a synthetic voice corresponding to the response; In the output step, each of the video data is output in order; The generating step and the outputting step are performed in parallel, wherein: In the generating step, the n-th video data is generated so that the n-th video data corresponding to the n-th segment can be output consecutively to the n-1-th video data corresponding to the n-1-th segment; When the n-th video data is generated in the generating step, if the output of the (n-1)-th video data is not completed, the processor queues the output of the n-th video data in the output step; In the generating step, a generation rate of the video data is adjusted so as to optimize the number of video data in the queue.

2. 2. The information processing system according to claim 1, In the first receiving step, the system converts the user's voice into text in real time, and when the user's voice is interrupted, accepts the user's voice that has been converted into text up to that point as the voice information.

3. 2. The information processing system according to claim 1, Furthermore, in the second receiving step, the response transmitted in a streaming format from the artificial intelligence module is divided into the plurality of segments and received; In the generating step, the system generates video data corresponding to the received segments in order.

4. 2. The information processing system according to claim 1, The segments are divisions of the response based on punctuation marks.

5. 2. The information processing system according to claim 1, The animation includes at least one of mouth movements, body movements, and facial expressions of the avatar in sync with the speech of the avatar.

6. 2. The information processing system according to claim 1, In the first receiving step, data different from the voice information is further received, The input step includes inputting the prompt, which includes the voice information and the data different from the voice information, to the artificial intelligence module.

7. 1. An information processing method, comprising: A method comprising the steps of the information processing system according to any one of claims 1 to 6.

8. A program, A program for causing at least one computer to execute each step in the information processing system according to any one of claims 1 to 6.

Images