Data processing device, communication terminal, data processing method, and program

Abstract

To provide a data processing device, a communication terminal, a data processing method, and a program that can contribute to reducing the burden on an answerer and a questioner when the answerer answers questions from the questioner and the questioner collects answers from the answerer, and to collecting highly reliable answers.SOLUTION: A data processing device includes an execution unit that interactively asks questions to an answerer and makes answers to the questions from the answerer, a generation unit that generates summary data that summarizes the answers by using a summary data generation model, and an output unit that outputs at least one of the summary data and data based on the summary data.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The technology disclosed herein relates to a data processing device, a communication terminal, a data processing method, and a program. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a survey such as a questionnaire or an interview is conducted face-to-face, a psychological barrier may arise between the questioner and the respondent due to tension or distrust. When a psychological barrier arises between the questioner and the respondent, it becomes difficult for the questioner to find out the respondent's true feelings. This results in a decline in the quality of the survey results.

[0005] There are many people in the world who would like to enjoy daily conversations with their favorite people or characters (for example, characters appearing in games, manga, anime, etc.) online rather than face-to-face. Online conversations with favorite people or characters make it easier to open up and reveal one's true feelings compared to face-to-face conversations with strangers, so the psychological barrier between the conversational parties is lower compared to conversations in face-to-face surveys.

[0006] In surveys such as multiple choice or written questionnaires, it is difficult to motivate respondents. The same is true when respondents are busy. If respondents are not motivated to participate in the survey, the survey results are likely to be of low quality.

[0007] The technology disclosed herein provides a data processing device, a communication terminal, a data processing method, and a program that can contribute to reducing the burden on the answerer and the questioner when the answerer answers a question from the questioner and the questioner collects an answer from the answerer, and to collecting reliable answers. [Means for solving the problem]

[0008] A first aspect of the technology of the present disclosure is a data processing device including an execution unit that interactively asks questions to a respondent and receives answers to the questions from the respondent, a generation unit that generates summary data that summarizes the answers by using a summary data generation model, and an output unit that outputs at least one of the summary data and data based on the summary data.

[0009] A second aspect of the technology disclosed herein is the data processing device according to the first aspect, wherein the execution unit outputs question voice data indicating the voice of the question and obtains answer voice data indicating the voice of the answer.

[0010] A third aspect of the technology of the present disclosure is the data processing device according to the second aspect, wherein the execution unit executes the output of the question voice data and the acquisition of the answer voice data in the interactive format.

[0011] A fourth aspect of the technology of the present disclosure is the data processing device according to the first aspect, further comprising an identification unit that identifies an emotion of the respondent based on the answer.

[0012] A fifth aspect of the technology disclosed herein is the data processing device according to the fourth aspect, wherein the execution unit acquires answer voice data indicating a voice of the answer, and the identification unit identifies the emotion by using an emotion identification model that responds to an input of the answer voice data and outputs emotion data related to the emotion corresponding to the answer voice data.

[0013] A sixth aspect of the technology of the present disclosure is the data processing device according to the fourth aspect, wherein the data based on the summary data is data generated based on the emotion identified by the identification unit and the summary data.

[0014] A seventh aspect of the technology of the present disclosure is the data processing device according to the sixth aspect, wherein the data based on the summary data is an analysis result of analyzing the summary data based on the emotion identified by the identification unit.

[0015] An eighth aspect of the technology disclosed herein is the data processing device of the seventh aspect, further comprising an acquisition unit that acquires attribute data indicating attributes of the respondent, and data based on the summary data is an analysis result of the summary data based on the emotion identified by the identification unit and the attribute data acquired by the acquisition unit.

[0016] A ninth aspect of the technology disclosed herein is a data processing device according to the first aspect, further comprising an acquisition unit that acquires attribute data indicating attributes of the respondent, and the execution unit assigns characteristics determined according to the attribute data acquired by the acquisition unit to a form of the question, and outputs the question with the characteristics assigned to the form of the question.

[0017] A tenth aspect of the technology disclosed herein is a data processing device according to the ninth aspect, wherein the expression form is speech, and the execution unit assigns the features to the speech and outputs question speech data indicating the speech to which the features have been assigned.

[0018] An eleventh aspect of the technology disclosed herein is a data processing device according to the first aspect, in which the questions and answers are asked with a survey as one of the purposes, and the execution unit, on condition that an exchange between the answer and the question has been conducted in the interactive format, negotiates with the respondent in the interactive format as to whether or not a summary of the answer obtained in the exchange may be used as a result of the survey.

[0019] A twelfth aspect of the technology of the present disclosure is the data processing device of the eleventh aspect, wherein the summary used as the result is summary data, and the execution unit executes presenting the summary data to the respondent.

[0020] A thirteenth aspect of the technology disclosed herein is a data processing device according to the eleventh aspect, wherein the execution unit obtains consent data indicating that the respondent has consented to the use of the summary as a result of the survey.

[0021] A fourteenth aspect of the technology disclosed herein is a data processing device according to the thirteenth aspect, further comprising an awarding unit that awards a reward to the respondent on condition that the acceptance data is obtained by the execution unit.

[0022] A fifteenth aspect of the technology disclosed herein is a data processing device according to the thirteenth aspect, wherein the output unit outputs at least one of the summary data and data based on the summary data, on condition that the consent data has been obtained by the execution unit.

[0023] A sixteenth aspect of the technology disclosed herein is a data processing device according to the first aspect, wherein the generation unit generates a response to the answer by using a response generation model, the execution unit presents the response generated by the generation unit to the answerer, and the response is part of the question.

[0024] A seventeenth aspect of the technology disclosed herein is a data processing device according to the sixteenth aspect, wherein the questions and answers are asked with a survey as one of the purposes, and at least one of the questions and answers includes content that is in line with the theme of the survey.

[0025] An 18th aspect of the technology disclosed herein is a data processing device according to the first aspect, wherein the questions and answers are asked with a survey as one of the purposes, and the questions include content that is in line with the theme of the survey.

[0026] A 19th aspect of the technology of the present disclosure is a data processing device comprising an execution unit that executes questions for a respondent and an answer to the question from the respondent in an interactive format, and an identification unit that identifies the respondent's emotions based on the answer.

[0027] A twentieth aspect of the technology disclosed herein is a data processing device according to the nineteenth aspect, wherein the execution unit acquires answer voice data indicating a voice of the answer, and the identification unit identifies the emotion by using an emotion identification model that responds to an input of the answer voice data and outputs emotion data related to the emotion corresponding to the answer voice data.

[0028] A twenty-first aspect of the technology disclosed herein is a data processing device according to the first aspect, wherein the respondent is a user of a chat application.

[0029] A 22nd aspect of the technology disclosed herein is a communications terminal capable of communicating with a data processing device relating to any one of the 1st to 21st aspects, the communications terminal having a user interface, the user interface presenting the question executed in the interactive format by the execution unit, and accepting the answer executed in the interactive format by the execution unit.

[0030] A twenty-third aspect of the technology of the present disclosure is a data processing method including interactively asking questions to a respondent and receiving answers to the questions from the respondent, generating summary data summarizing the answers by using a summary data generation model, and outputting at least one of the summary data and data based on the summary data.

[0031] A twenty-fourth aspect of the technology of the present disclosure is a data processing method including interactively asking questions to a respondent and receiving answers to the questions from the respondent, and identifying emotions of the respondent based on the answers.

[0032] A 25th aspect of the technology of the present disclosure is a program for causing a computer to execute a process including: interactively asking questions to a respondent and receiving answers to the questions from the respondent; generating summary data that summarizes the answers by using a summary data generation model; and outputting at least one of the summary data and data based on the summary data.

[0033] A 26th aspect of the technology of the present disclosure is a program for causing a computer to execute a process including interactively asking questions to a respondent and receiving answers to the questions from the respondent, and identifying the respondent's emotions based on the answers. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2] FIG. 2 is a conceptual diagram illustrating an example of a database configuration. [Diagram 3] FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a smart device. [Figure 4] A conceptual diagram showing an example of processing contents in a data processing system when a survey implementation process is performed by a processor of a data processing device and a reception output process is performed by a processor of a smart device. [Diagram 5] FIG. 2 is a conceptual diagram showing an example of a data format (expression form) of dialogue partner data generated by the dialogue generation model. [Figure 6] FIG. 13 is a conceptual diagram showing an example of processing contents of a smart device. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the processing contents performed by the data processing device when responses are obtained from a plurality of people who are the subjects of a questionnaire survey. [Figure 8A] 13 is a flowchart showing an example of the flow of a survey implementation process. [Figure 8B] This is a continuation of the flowchart of FIG. 8A. [Figure 9A] 13 is a flowchart showing an example of the flow of a reception and output process. [Figure 9B] This is a continuation of the flowchart of FIG. 9A. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Hereinafter, examples of embodiments of a data processing device, a communication terminal, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the attached drawings.

[0036] First, the terms used in the following description will be explained.

[0037] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0038] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0039] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0040] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0041] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0042] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0043] As shown in Fig. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server. An example of the smart device 14 is a smartphone. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure, and the smart device 14 is an example of a "communication terminal" according to the technology of the present disclosure.

[0044] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0045] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52. The reception device 38 and the output device 40 are an example of a "user interface" according to the technology of the present disclosure.

[0046] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives instructions from the person 20. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive instructions by touch of the pointer. The microphone 38B detects the voice of the person 20 to receive instructions by voice.

[0047] The output device 40 includes a display 40A and a speaker 40B, and presents data to the person 20 by outputting the data in a form of expression that the person 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0048] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0049] The data processing system 10 is a system used by the survey conducting company 16 to conduct a survey requested by the survey requesting company 18. For example, the survey is a questionnaire survey in which questions are answered (e.g., a questionnaire survey about a company or a product, etc.). The questionnaire survey is conducted online using the data processing device 12 and the smart device 14. The survey conducting company 16 possesses the data processing device 12, and the person 20 possesses the smart device 14. The person 20 is an example of a "respondent" according to the technology of the present disclosure. The survey conducting company 16 is an example of a "questioner" according to the technology of the present disclosure.

[0050] The person 20 uses the smart device 14 to answer questions posed by the survey company 16 via the data processing device 12 and the smart device 14. The exchange of questions and answers is carried out in an interactive chat format. The chat uses text and voice. The chat is realized by executing a chat application on the smart device 14.

[0051] As will be described in more detail below, in the questionnaire survey, the questions posed by the survey implementing company 16 to the person 20 are questions based on the survey theme etc. determined in advance by the survey requesting company 18, and are generated by the data processing device. In addition, a reply to the answer from the person 20 is generated by the data processing device. The reply includes part of the question. For example, the question is divided into multiple sub-questions, and depending on the content of the answer, a reply including the sub-questions is generated by the data processing device. The survey implementing company 16 uses the data processing device 12 to analyze (for example, analyze using statistical methods) the multiple answers obtained from the multiple persons 20, and delivers the results of the analysis to the survey requesting company 18 as survey results.

[0052] 2 shows an example of the configuration of the database 24. The database 24 stores various data related to a plurality of persons 20. Examples of the various data include ID (IDentification) 24A, attribute data 24B, and remuneration data 24C.

[0053] The ID 24A is a number capable of identifying a person 20 (here, as an example, a user of the chat application described above) and is assigned to each of the multiple persons 20. The attribute data 24B is data indicating the attributes of the person 20. Examples of the attributes include gender, age, address, occupation, purchased product (e.g., a product purchased through online shopping), purchase date and time (e.g., the date and time when a product was purchased through online shopping), and favorite character (e.g., a character appearing in a game, a character appearing in a manga, a character appearing in an anime, or a real celebrity, etc.). The reward data 24C is data indicating the reward granted to the person 20.

[0054] Here, the reward refers to, for example, rewards that can be obtained by using online services. One example of rewards is points that can be used in various commercial transactions (for example, points that can be used in place of cash when purchasing products).

[0055] Attribute data 24B and reward data 24C are associated with each ID 24A assigned to each person 20. For example, the attribute data 24B and reward data 24C associated with the ID 24A are updated by the processor 28 (see FIG. 1) in response to the actions of the person 20 (e.g., the online behavior of the person 20).

[0056] 3 shows an example of main functions of the data processing device 12 and the smart device 14. As shown in FIG. 3, in the data processing device 12, a processor 28 performs a survey implementation process.

[0057] A survey implementation program 56 is stored in the storage 32. The survey implementation program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads out the survey implementation program 56 from the storage 32, and executes the read out survey implementation program 56 on the RAM 30. The survey implementation process is realized by the processor 28 operating as an execution unit 28A, a generation unit 28B, an identification unit 28C, an acquisition unit 28D, an analysis unit 28E, an output unit 28F, and an assignment unit 28G in accordance with the survey implementation program 56 executed on the RAM 30.

[0058] The storage 32 stores a dialogue generation model 58 and an emotion identification model 60. The dialogue generation model 58 is an example of a "summary data generation model" and a "reply generation model" according to the technology of the present disclosure, and the emotion identification model 60 is an example of an "emotion identification model" according to the technology of the present disclosure. As will be described in detail later, the dialogue generation model 58 is used by the generation unit 28B, and the emotion identification model 60 is used by the identification unit 28C.

[0059] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 62. The reception output program 62 is an example of the above-mentioned chat application, and is used together with the survey implementation program 56 by the data processing system 10. The reception output program 62 is a program created based on an existing chat application (for example, the conventionally known LINE (registered trademark)). The processor 46 reads out the reception output program 62 from the storage 50, and executes the read reception output program 62 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0060] FIG. 4 shows an example of the processing contents in the data processing system 10 when the processor 28 of the data processing device 12 performs a survey implementation process and the processor 46 of the smart device 14 performs a reception output process.

[0061] As shown in Fig. 4, the execution unit 28A communicates with the smart device 14 via the network 54 (see Fig. 1) to interactively ask questions to the person 20 and receive responses to the questions from the person 20. First, the execution unit 28A determines whether or not a condition for starting a questionnaire survey (hereinafter referred to as a "survey start condition") is satisfied in order to interactively ask questions to the person 20 and receive responses to the questions from the person 20. One example of the survey start condition is that a period set in advance by an administrator of the data processing device 12 or the like has arrived.

[0062] The storage 32 stores a template 64 and survey request data 66. As will be described in detail later, the template 64 is used to generate first instruction data 68 to be input to the dialogue generation model 58. The survey request data 66 is data indicating the contents of a request for a questionnaire survey from the survey requesting company 18. The survey request data 66 includes a survey theme 66A and target attribute data 66B. The survey theme 66A is the theme of the questionnaire survey. The survey theme 66A includes the name of the survey requesting company 18. The target attribute data 66B is data indicating the attributes that are the subject of the questionnaire survey. For example, the attributes that are the subject of the questionnaire survey are one or more specified attributes among the multiple attributes indicated by the attribute data 24B stored in the database 24.

[0063] The execution unit 28A acquires the template 64 and the research topic 66A from the storage, on condition that the research start conditions are satisfied.

[0064] The acquisition unit 28D acquires attribute data 24B corresponding to the target attribute data 66B stored in the storage 32 from the database 24, and acquires ID 24A associated with the attribute data 24B acquired from the database 24. For example, if the attributes indicated by the target attribute data 66B are "male" and "22 years old", the acquisition unit 28D acquires attribute data 24B indicating the attribute "male" and attribute data 24B indicating the attribute "22 years old" from the database 24. The acquisition unit 28D also acquires ID 24A associated with attribute data 24B indicating the attribute "male" and attribute data 24B indicating the attribute "22 years old" from the database 24. The acquisition unit 28D also acquires attribute data indicating "favorite character" associated with ID 24A acquired from the database 24 from the database 24.

[0065] The execution unit 28A generates first instruction data 68 indicating instructions to be given to the dialogue generation model 58 by incorporating the research theme 66A obtained from the storage 32 and the attribute data 24B obtained by the acquisition unit 28D into the template 64 obtained from the storage 32.

[0066] The dialogue generation model 58 is a so-called generative AI (Artificial Intelligence). One example of the dialogue generation model 58 is ChatGPT-4 (Internet search<https: / / openai.com / gpt-4> ) or ChatGPT-4-32k. The dialogue generation model 58 is obtained by performing deep learning on a neural network. The dialogue generation model 58 is input with first instruction data 68, and also with inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image. The dialogue generation model 58 infers the input inference data according to the instruction indicated by the first instruction data 68, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0067] An example of the template 64 is text data showing the following text:

[0068] "You are the person asking the survey. Ask questions that are in line with the "survey topic...". Break down your questions into small parts. Respond to the answers of the survey respondents at least three times. Include questions in your answers. Respond to the respondents in a way that allows for a natural conversation between you and the respondent. Make your answers easy to answer the questions. Questions and answers should be done in voice and text. Use the voice of the "character...". Add the characteristics of the "character..." to the text. When all questions in line with the "survey topic..." have been asked, tell the respondent in voice and text that this is a survey for "company...". Tell the respondent in voice and text whether they agree to provide a summary of all their answers to all questions to "company...". Tell the respondent in voice and text that they will receive reward points if they agree. If the respondent does not agree, tell the respondent in voice and text that you understand that they will not agree. If you have obtained consent from the respondent, communicate the summary to the respondent by voice and text. Ask the respondent by voice and text whether the content of the summary is appropriate. If the respondent responds that the content of the summary is not appropriate, communicate the re-created summary to the respondent by voice and text. If the respondent responds that the content of the summary is appropriate, communicate to the respondent by voice and text that they have been awarded reward points. The questions and responses to the respondent should be tailored to make the respondent feel good, using Information 1 to Information N below. "Information 1:..." "Information 2:..." to "Information N:..."

[0069] Execution unit 28A generates first instruction data 68 by incorporating text obtained from research theme 66A into "Research theme..." and "Company...", and incorporating text obtained from attribute data 24B into "Character..." and "Information 1:...", "Information 2:..." to "Information N:...". Note that template 64 specifically illustrated here is merely an example, and any instruction may be used as long as it does not deviate from the spirit of the technology of the present disclosure.

[0070] The generation unit 28B generates dialogue partner data 70 by using the dialogue generation model 58. The dialogue partner data 70 is data indicating the content of a conversation with a person 20 who is the subject of a questionnaire survey. The dialogue partner data 70 is roughly divided into dialogue start data 70A, response data 70B, and dialogue end data 70C.

[0071] The dialogue start data 70A is data indicating the content to be spoken first in a dialogue with the person 20. The reply data 70B is data indicating a reply to a reply to a question from the person 20. The reply data 70B is broadly divided into question-and-reply data 70B1 and summary-and-reply data 70B2. The question-and-reply data 70B1 is data indicating a reply that contains a part of a question along the survey theme 66A (i.e., at least one of a plurality of sub-questions into which a question along the survey theme 66A is divided). The summary-and-reply data 70B2 is data indicating a reply that contains a summary of the answers to all questions asked to the person 20. The summary-and-reply data 70B2 includes summary data 70B2a. The summary data 70B2a is data indicating a summary of the answers to all questions asked to the person 20. The dialogue end data 70C is data indicating the content to be spoken last in a dialogue with the person 20 (i.e., the content of the last reply in the dialogue).

[0072] When the first instruction data 68 is input to the dialogue generation model 58, dialogue start data 70A is generated by the dialogue generation model 58. The execution unit 28A acquires the dialogue start data 70A generated by the dialogue generation model 58. Then, the execution unit 28A transmits the dialogue start data 70A to the smart device 14 identified from the ID 24A acquired by the acquisition unit 28D. In the smart device 14, the control unit 46A causes the output device 40 to output the dialogue start data 70A. The microphone 38B acquires a voice indicating a response by the person 20 to the dialogue start data 70A. The control unit 46A transmits answer voice data 72 (hereinafter also referred to as "first answer voice data") indicating the voice acquired by the microphone 38B in response to the dialogue start data 70A to the data processing device 12. In the data processing device 12, the execution unit 28A acquires the first answer voice data.

[0073] When the first answer voice data is input to the dialogue generation model 58, the dialogue generation model 58 generates question-and-answer data 70B1 that matches the first answer voice data. The question-and-answer data 70B1 that matches the first answer voice data refers to data indicating a reply that includes a question determined according to the answer identified from the first answer voice data. The execution unit 28A acquires the question-and-answer data 70B1 generated by the dialogue generation model 58. The execution unit 28A then transmits the question-and-answer data 70B1 to the smart device 14 identified from the ID 24A acquired by the acquisition unit 28D. In the smart device 14, the control unit 46A causes the output device 40 to output the question-and-answer data 70B1. The microphone 38B acquires a voice indicating the reply by the person 20 to the question-and-answer data 70B1. The control unit 46A transmits answer voice data 72 indicating the voice picked up by the microphone 38B in response to the question-and-answer data 70B1 (hereinafter also referred to as "second answer voice data") to the data processing device 12. In the data processing device 12, the execution unit 28A acquires the second answer voice data.

[0074] When the second answer voice data is input to the dialogue generation model 58, the dialogue generation model 58 generates question-and-answer data 70B1 that matches the second answer voice data. Here, the question-and-answer data 70B1 that matches the second answer voice data refers to data indicating a reply that includes a question determined according to the answer identified from the second answer voice data. The exchange between the question-and-answer data 70B1 and the second answer voice data continues until all questions according to the survey theme 66A are completed. The last question among all questions according to the survey theme 66A is a question regarding negotiation as to whether or not a summary of answers identified from the multiple second answer voice data obtained by multiple exchanges between the question-and-answer data 70B1 and the second answer voice data may be used as a result of a questionnaire survey.

[0075] Furthermore, the final answer among all the answers to all the questions in line with survey theme 66A is an answer (i.e., an answer indicating whether or not to agree) to a question regarding a negotiation as to whether or not a summary of answers identified from multiple second answer voice data obtained by multiple exchanges between question-and-reply data 70B1 and second answer voice data may be used as the result of a questionnaire survey.

[0076] When the final second answer voice data obtained through the exchange between the question-and-answer data 70B1 and the second answer voice data is input to the dialogue generation model 58, summary-and-reply data 70B2 is generated by the dialogue generation model 58. The summary data 70B2a included in the summary-and-reply data 70B2 is data indicating a summary of answers identified from all of the second answer voice data obtained through the exchange between the question-and-reply data 70B1 and the second answer voice data until all questions related to the survey theme 66A are completed.

[0077] The execution unit 28A acquires the summary-including response data 70B2 generated by the dialogue generation model 58. Then, the execution unit 28A transmits the summary-including response data 70B2 to the smart device 14 identified from the ID 24A acquired by the acquisition unit 28D. In the smart device 14, the control unit 46A causes the output device 40 to output the summary-including response data 70B2. The microphone 38B acquires a voice indicating a response by the person 20 to the summary-including response data 70B2. The control unit 46A transmits response voice data 72 indicating the voice acquired by the microphone 38B in response to the summary-including response data 70B2 (hereinafter also referred to as "third response voice data") to the data processing device 12. In the data processing device 12, the execution unit 28A acquires the third response voice data.

[0078] Here, if the answer indicated by the third answer voice data is a negative answer to the summary indicated by the summary data 70B2a, the summary-including answer data 70B2 is regenerated, and the exchange between the summary-including answer data 70B2 and the third answer voice data continues.

[0079] If the answer indicated by the third answer voice data is an affirmative answer to the summary indicated by the summary data 70B2a (i.e., if the person 20 agrees with the content of the summary), then when the third answer voice data is input to the dialogue generation model 58, dialogue end data 70C matching the third answer voice data is generated by the dialogue generation model 58. The dialogue end data 70C includes data indicating a message that a reward will be given for cooperating with the questionnaire survey.

[0080] The execution unit 28A acquires the dialogue end data 70C generated by the dialogue generation model 58. Then, the execution unit 28A transmits the dialogue end data 70C to the smart device 14 identified from the ID 24A acquired by the acquisition unit 28D. In the smart device 14, the control unit 46A causes the output device 40 to output the dialogue end data 70C.

[0081] Fig. 5 shows an example of a data format (expression form) of the dialogue partner data 70 generated by the dialogue generation model 58. As shown in Fig. 5, the dialogue generation model 58 generates, as dialogue start data 70A, dialogue start voice data 70Aa indicating the first thing to be said in a dialogue with the person 20 in the voice of a character (here, as an example, character A) that the person 20 likes, and dialogue start text data 70Ab indicating the first thing to be said in a dialogue with the person 20 in text to which the characteristics of character A have been added.

[0082] Furthermore, the dialogue generation model 58 generates, as response data 70B, response voice data 70Ba indicating a response to the answer to the question from the person 20 in the voice of character A, and response text data 70Bb indicating a response to the answer to the question from the person 20 in text to which the characteristics of character A have been added. Furthermore, the dialogue generation model 58 generates, as dialogue end data 70C, dialogue end voice data 70Ca indicating the last spoken content in the dialogue with the person 20 in the voice of character A, and dialogue end text data 70Cb indicating the last spoken content in the dialogue with the person 20 in text to which the characteristics of character A have been added.

[0083] In the example shown in FIG. 5, the dialogue start voice data 70Aa and the reply voice data 70Ba are examples of "question voice data" according to the technique of the present disclosure.

[0084] Fig. 6 shows an example of the processing contents of the smart device 14. As shown in Fig. 6, the speaker 40B outputs the voice indicated by the voice data (i.e., dialogue start voice data 70Aa, response voice data 70Ba, and dialogue end voice data 70Ca) indicated by the dialogue partner data 70 transmitted from the execution unit 28A to the smart device 14 under the control of the control unit 46A. The voice output from the speaker 40B is the voice of the character A.

[0085] In addition, under the control of the control unit 46A, the display 40A outputs text indicated by the voice data (i.e., dialogue start text data 70Ab, response text data 70Bb, and dialogue end text data 70Cb) indicated by the dialogue partner data 70 transmitted from the execution unit 28A to the smart device 14.

[0086] In this way, by person 20 responding by voice to the voice and text content output from speaker 40B, a voice dialogue between character A and person 20 (i.e., a voice chat between character A and person 20) is established.

[0087] The microphone 38B acquires the voice response of the person 20. The control unit 46A then performs a conventionally known voice-to-text conversion process on the response acquired by the microphone 38B to convert the voice response into a text response, and displays the text response on the display 40A.

[0088] On the display 40A, the speech of the character A and the speech of the person 20 are displayed alternately in a distinguishable manner, so that a text dialogue between the character A and the person 20 (i.e., a text chat between the character A and the person 20) is established.

[0089] FIG. 7 shows an example of the process carried out by the data processing device 12 when responses are obtained from a plurality of persons 20 who are the subjects of a questionnaire survey.

[0090] 7, the identification unit 28C generates second instruction data 74 indicating an instruction to be given to the emotion identification model 60. An example of an instruction indicated by the second instruction data 74 is text data indicating the text "You have the ability to identify the emotion of a respondent from the voice of the respondent in a questionnaire survey. Please analyze the respondent's voice data that you have received and identify the respondent's emotion. Please use Plutchik's emotion wheel to identify the respondent's emotion."

[0091] The identification unit 28C uses the emotion identification model 60 to identify, from the voice represented by the answer voice data 72, the emotion of the person 20 who answered the question based on the survey theme 66A.

[0092] The emotion identification model 60 is a so-called generation AI. An example of the emotion identification model 60 is a generation AI such as ChatGPT-4 or ChatGPT-4-32k. The emotion identification model 60 can be obtained by performing deep learning on a neural network. The identification unit 28C inputs the second instruction data 74 and the answer voice data 72 to the emotion identification model 60.

[0093] In response to the input of the answer voice data 72, the emotion identification model 60 outputs emotion data related to the emotion corresponding to the answer voice data 72. In the example shown in Fig. 7, the emotion identification model 60 identifies the emotion and the degree of emotion of the person 20 who answered the question based on the survey theme 66A in accordance with the instruction given by the second instruction data 74 for the input answer voice data 72, and outputs emotion identification data 76 indicating the identified emotion and the degree of emotion. For example, the emotion identification data 76 is expressed as a numerical value and / or text data, etc.

[0094] For example, the emotion identified by the emotion identification model 60 is a positive emotion or a negative emotion in response to a question in a survey. Positive emotion and negative emotion are obtained by, for example, dividing a plurality of emotions defined in Plutchik's emotion wheel. Positive emotion and negative emotion are merely examples, and the emotion identification model 60 may identify a specific emotion defined in Plutchik's emotion wheel.

[0095] The analysis unit 28E generates a data set 78 for each person 20 who is the subject of the questionnaire survey. The data set 78 is data in which the emotion identification data 76 output from the emotion identification model 60, the attribute data 24B used in generating the first instruction data 68 by the execution unit 28A, and the summary data 70B2a generated by the generation unit 28B are associated with each other.

[0096] The data set 78 is generated for each of the multiple persons 20 who were the subjects of the questionnaire survey. The analysis unit 28E analyzes the multiple data sets 78 corresponding to the responses of the multiple persons 20 who were the subjects of the questionnaire survey, and generates analysis result data 80 indicating the analysis result.

[0097] An example of the analysis of the data set 78 is an analysis using a statistical method. The analysis unit 28E classifies the multiple pieces of summary data 70B2a or deletes unnecessary summary data 70B2a from the multiple pieces of summary data 70B2a based on the most common emotion, the average emotion, the least common emotion, and statistical values ​​of the degree of emotion (e.g., average value, median value, maximum value, minimum value, and / or mode) identified from the multiple emotion identification data 76 included in the multiple data sets 78.

[0098] The output unit 28F acquires the analysis result data 80 obtained by the analysis unit 28E, and outputs the acquired analysis result data 80 to a predetermined output destination. A first example of the predetermined output destination is a server of the survey requesting company 18. A second example of the predetermined output destination is storage 32. The output of the analysis result data 80 to the server of the survey requesting company 18 is realized by transmission via the network 54 using the communication I / F 26.

[0099] The granting unit 28G grants a reward to the person 20 who has agreed to use a summary of answers to questions based on the survey theme 66A as a result of the questionnaire survey, on the condition that the execution unit 28A acquires answer voice data 72 indicating an answer given by the person 20 to the person 20 who has agreed to use a summary of answers to questions based on the survey theme 66A as a result of the questionnaire survey. For example, the granting unit 28G grants a reward to the person 20 who is the owner of the smart device 14 to which the dialogue partner data 70 is transmitted in order to obtain the summary data 70B2a included in the data set 78 at the timing when the data set 78 is generated. In the example shown in FIG. 7, the granting unit 28G updates the reward data 24C in the database 24 so as to increase the reward indicated by the reward data 24C corresponding to the ID 24 currently acquired by the acquisition unit 28D. Note that, although an example has been given here in which a reward is given to person 20 when data set 78 is generated, this is merely one example, and reward may also be given to person 20 when consent to the contents of summary data 70B2a is obtained from person 20, or reward may be given to person 20 when analysis result data 80 is delivered to survey requesting company 18.

[0100] Next, the operation of the data processing system 10 will be described with reference to FIGS. 8A to 9B.

[0101] First, an example of the flow of a survey implementation process will be described with reference to Figures 8A and 8B. Note that the flow of the survey implementation process shown in Figures 8A and 8B is an example of a "data processing method" according to the technology of the present disclosure.

[0102] In the investigation execution process shown in Fig. 8A, first, in step ST10, the execution unit 28A judges whether or not the investigation start condition is satisfied. If the investigation start condition is not satisfied in step ST10, the judgment is negative, and the judgment of step ST10 is performed again. If the investigation start condition is satisfied in step ST10, the judgment is positive, and the investigation execution process proceeds to step ST12.

[0103] In step ST12, the execution unit 28A acquires a survey theme 66A from the storage 32. In addition, the acquisition unit 28D acquires target attribute data 66B from the storage 32. After the process of step ST12 is executed, the survey implementation process proceeds to step ST14.

[0104] In step ST14, the acquisition unit 28D acquires attribute data 24B corresponding to the target attribute data 66B acquired in step ST12 from the database 24. The acquisition unit 28D also acquires ID 24A associated with the attribute data 24B acquired from the database 24 from the database 24. After the processing of step ST14 is executed, the survey implementation processing proceeds to step ST16.

[0105] In the following, it is assumed that a plurality of attribute data 24B and a plurality of IDs 24A corresponding to a plurality of persons 20 are acquired from the database 24 in step ST14.

[0106] In step ST16, the execution unit 28A acquires the template 64 from the storage 32. After the process of step ST16 is executed, the survey implementation process proceeds to step ST18.

[0107] In step ST18, the execution unit 28A acquires corresponding unused attribute data 24B and IDs 24A from the plurality of attribute data 24B and the plurality of IDs 24A acquired by the acquisition unit 28D in step ST14. After the processing of step ST18 is executed, the survey implementation processing proceeds to step ST20.

[0108] In step ST20, the execution unit 28A generates first instruction data 68 using the template 64 acquired in step ST16, the research topic 66A acquired in step ST12, and the attribute data 24B acquired in step ST18. After the process of step ST20 is executed, the research implementation process proceeds to step ST22.

[0109] In step ST22, the generation unit 28B generates dialogue start data 70A based on the first instruction data 68 by using the dialogue generation model 58. That is, the generation unit 28B inputs the first instruction data 68 to the dialogue generation model 58, and the dialogue generation model 58 generates and outputs the dialogue start data 70A in response to the input of the first instruction data 68 by the generation unit 28B. The execution unit 28A transmits the dialogue start data 70A in accordance with the ID 24A acquired in step ST18. That is, the dialogue start data 70A is transmitted to the smart device 14 identified by the ID 24A. After the processing of step ST22 is executed, the survey implementation processing proceeds to step ST24.

[0110] In response to the dialogue start data 70A being transmitted to the smart device 14 as a result of the processing of step ST22 being executed, the smart device 14 transmits answer voice data 72 (see step ST108 shown in FIG. 9A).

[0111] In step ST24, the execution unit 28A judges whether or not the answer voice data 72 transmitted from the smart device 14 has been received by the communication I / F 26. In step ST24, if the answer voice data 72 transmitted from the smart device 14 has not been received by the communication I / F 26, the judgment is negative, and the judgment of step ST24 is performed again. In step ST24, if the answer voice data 72 transmitted from the smart device 14 has been received by the communication I / F 26, the judgment is positive, and the survey implementation process proceeds to step ST26.

[0112] In step ST26, the execution unit 28A determines whether or not all questions according to the survey theme 66A acquired in step ST12 have been completed. If all questions according to the survey theme 66A acquired in step ST12 have been completed in step ST26, the determination is negative, and the survey implementation process proceeds to step ST30. If all questions according to the survey theme 66A acquired in step ST12 have not been completed in step ST26, the determination is positive, and the survey implementation process proceeds to step ST28.

[0113] In step ST28, the generation unit 28B generates question-and-answer data 70B1 corresponding to the answer voice data 72 acquired in step ST24 by using the dialogue generation model 58. The execution unit 28A transmits the question-and-answer data 70B1 in accordance with the ID 24A acquired in step ST18. That is, the question-and-answer data 70B1 is transmitted to the smart device 14 identified by the ID 24A. After the processing of step ST28 is executed, the survey implementation processing proceeds to step ST24.

[0114] As a result of the processing of step ST28 being executed, the question-and-answer data 70B1 is sent to the smart device 14, and in response thereto, answer voice data 72 is sent from the smart device 14 (see step ST108 shown in FIG. 9A).

[0115] In step ST30, the execution unit 28A inputs the answer voice data 72 acquired in step ST24 to the dialogue generation model 58. The dialogue generation model 58 judges whether or not consent has been obtained from the person 20 to using the summary of the answers to the questions along the survey theme 66A as the results of the questionnaire survey, based on the answer voice data 72 input from the execution unit 28A. If consent has not been obtained in step ST30, the judgment is negative, and the process proceeds to step ST46 shown in FIG. 8B. If consent has been obtained in step ST30, the judgment is positive, and the survey implementation process proceeds to step ST32. The answer voice data 72 indicating that the person 20 has consented to using the summary of the answers to the questions along the survey theme 66A as the results of the questionnaire survey is an example of "consent data" according to the technology of the present disclosure.

[0116] In step ST32, the generation unit 28B generates summary-including response data 70B2 including summary data 70B2a summarizing responses identified from all answer voice data 72 obtained to all questions related to the survey topic 66A, by using the dialogue generation model 58. The execution unit 28A transmits the summary-including response data 70B2 according to the ID 24A acquired in step ST18. That is, the summary-including response data 70B2 is transmitted to the smart device 14 identified by the ID 24A. After the process of step ST32 is executed, the survey implementation process proceeds to step ST34 shown in FIG. 8B.

[0117] In response to the summary-containing reply data 70B2 being transmitted to the smart device 14 as a result of the processing of step ST32 being executed, reply voice data 72 is transmitted from the smart device 14 (see step ST108 shown in FIG. 9A).

[0118] In step ST34 shown in Fig. 8B, the execution unit 28A judges whether or not the answer voice data 72 transmitted from the smart device 14 has been received by the communication I / F 26. In step ST34, if the answer voice data 72 transmitted from the smart device 14 has not been received by the communication I / F 26, the judgment is negative, and the judgment of step ST34 is performed again. In step ST34, if the answer voice data 72 transmitted from the smart device 14 has been received by the communication I / F 26, the judgment is positive, and the survey implementation process proceeds to step ST36.

[0119] In step ST36, the execution unit 28A inputs the answer voice data 72 acquired in step ST34 to the dialogue generation model 58. The dialogue generation model 58 determines whether or not consent has been obtained from the person 20 for the summary of answers to questions related to the survey theme 66A, based on the answer voice data 72 input from the execution unit 28A. If consent has not been obtained in step ST36, the determination is negative, and the process proceeds to step ST38. If consent has been obtained in step ST36, the determination is positive, and the survey implementation process proceeds to step ST40.

[0120] In step ST38, the generation unit 28B regenerates summary-including response data 70B2 including summary data 70B2a summarizing responses identified from all answer voice data 72 obtained for all questions related to the survey topic 66A, by using the dialogue generation model 58. The execution unit 28A transmits the summary-including response data 70B2 in accordance with the ID 24A acquired in step ST18. That is, the summary-including response data 70B2 is transmitted to the smart device 14 identified by the ID 24A. After the processing of step ST38 is executed, the survey implementation processing proceeds to step ST34.

[0121] In step ST40, the identification unit 28C generates emotion identification data 76 based on the second instruction data 74 and the answer voice data 72 acquired in step ST34 by using the emotion identification model 60. After the process of step ST40 is executed, the survey implementation process proceeds to step ST42.

[0122] In step ST42, the analysis unit 28E generates a data set 78 based on the summary data 70B2a included in the summary-included reply data 70B2 generated in step ST32, the emotion identification data 76 generated in step ST40, and the attribute data 24B acquired in step ST14. After the process of step ST42 is executed, the survey implementation process proceeds to step ST44.

[0123] In step ST44, the granting unit 28G grants a reward to the person 20 corresponding to the data set 78 generated in step ST42 (i.e., the owner of the smart device 14 identified by the ID 24A acquired in step ST14). After the process of step ST44 is executed, the survey implementation process proceeds to step ST46.

[0124] In step ST46, the generation unit 28B generates dialogue end data 70C by using the dialogue generation model 58. The execution unit 28A transmits the dialogue end data 70C according to the ID 24A acquired in step ST18. That is, the dialogue end data 70C is transmitted to the smart device 14 identified by the ID 24A. After the processing of step ST46 is executed, the survey implementation processing proceeds to step ST48.

[0125] In step ST48, the analysis unit 28E judges whether the number of the generated data sets 78 is equal to or greater than a predetermined number (e.g., 1000). In step ST48, if the number of the generated data sets 78 is less than the predetermined number, the judgment is negative, and the survey implementation process proceeds to step ST18 shown in Fig. 8A. In step ST48, if the number of the generated data sets 78 is equal to or greater than the predetermined number, the judgment is positive, and the survey implementation process proceeds to step ST50.

[0126] In step ST50, the analysis unit 28E performs analysis on at least the predetermined number of data sets 78. After the process of step ST50 is executed, the survey implementation process proceeds to step ST52.

[0127] In step ST52, the analysis unit 28E generates analysis result data 80 indicating the analysis results performed in step ST50. The output unit 28F outputs the analysis result data 80 to a predetermined output destination (e.g., a server of the survey requesting company 18). After the process of step ST52 is performed, the survey implementation process ends.

[0128] Next, an example of the flow of the reception and output process will be described with reference to FIGS. 9A and 9B.

[0129] In the reception output process shown in Fig. 9A, first, in step ST100, the control unit 46A determines whether or not the dialogue partner data 70 has been received by the communication I / F 44. In step ST100, if the dialogue partner data 70 has not been received by the communication I / F 44, the determination is negative, and the determination of step ST100 is performed again. In step ST100, if the dialogue partner data 70 has been received by the communication I / F 44, the determination is positive, and the reception output process proceeds to step ST102.

[0130] In step ST102, the control unit 46A determines whether or not the dialogue partner data 70 received by the communication I / F 44 in step ST100 is dialogue start data 70A. In step ST102, if the dialogue partner data 70 received by the communication I / F 44 in step ST100 is not dialogue start data 70A, the determination is negative, and the reception output process proceeds to step ST112. In step ST102, if the dialogue partner data 70 received by the communication I / F 44 in step ST100 is dialogue start data 70A, the determination is positive, and the reception output process proceeds to step ST104.

[0131] In step ST104, the control unit 46A causes the output device 40 to output the dialogue start data 70A. After the process of step ST104 is executed, the reception and output process proceeds to step ST106.

[0132] In step ST106, the control unit 46A judges whether or not the microphone 38B has acquired a voice indicating a reply by the person 20 to the dialogue start data 70A, the reply data with question 70B1, or the reply data with summary 70B2 output from the output device 40. In step ST106, if the microphone 38B has not acquired a voice indicating a reply by the person 20 to the dialogue start data 70A, the reply data with question 70B1, or the reply data with summary 70B2 output from the output device 40, the judgment is negative, and the judgment of step ST106 is performed again. In step ST106, if the microphone 38B has acquired a voice indicating a reply by the person 20 to the dialogue start data 70A, the reply data with question 70B1, or the reply data with summary 70B2 output from the output device 40, the judgment is positive, and the reception and output process proceeds to step ST108.

[0133] In step ST108, the control unit 46A transmits answer voice data 72 indicating the voice picked up by the microphone 38B to the data processing device 12. After the process of step ST108 is executed, the reception and output process proceeds to step ST110.

[0134] In step ST110, the control unit 46A determines whether a condition for terminating the acceptance and output process (hereinafter referred to as the "termination condition") is satisfied. A first example of the termination condition is that an instruction to terminate the acceptance and output process is accepted by the acceptance device 38. A second example of the termination condition is that an instruction to terminate the acceptance and output process is given to the smart device 14 from an external device such as the data processing device 12.

[0135] In step ST110, if the end condition is not satisfied, the determination is negative, and the reception and output process proceeds to step ST100. In step ST110, if the end condition is satisfied, the determination is positive, and the reception and output process ends.

[0136] In step ST112, the control unit 46A determines whether the dialogue partner data 70 received by the communication I / F 44 in step ST100 is question-containing reply data 70B1. If the dialogue partner data 70 received by the communication I / F 44 in step ST100 is not question-containing reply data 70B1 in step ST112, the determination is negative, and the reception output process proceeds to step ST116 shown in Fig. 9B. If the dialogue partner data 70 received by the communication I / F 44 in step ST100 is question-containing reply data 70B1 in step ST112, the determination is positive, and the reception output process proceeds to step ST114.

[0137] In step ST114, the control unit 46A causes the output device 40 to output question-and-reply data 70B1. After the process of step ST114 is executed, the reception and output process proceeds to step ST106.

[0138] 9B, the control unit 46A judges whether or not the dialogue partner data 70 received by the communication I / F 44 in step ST100 is summary-including reply data 70B2. In step ST116, if the dialogue partner data 70 received by the communication I / F 44 in step ST100 is not summary-including reply data 70B2 (i.e., if the dialogue partner data 70 received by the communication I / F 44 in step ST100 is dialogue end data 70C), the judgment is negative and the reception output process proceeds to step ST120. In step ST116, if the dialogue partner data 70 received by the communication I / F 44 in step ST100 is summary-including reply data 70B2, the judgment is positive and the reception output process proceeds to step ST118.

[0139] In step ST118, the control unit 46A causes the output device 40 to output the summary-included response data 70B2. After the process of step ST118 is executed, the reception and output process proceeds to step ST106 shown in FIG. 9A.

[0140] In step ST120, the control unit 46A outputs the dialogue end data 70C to the output device 40. After the process of step ST120 is executed, the reception and output process ends.

[0141] As described above, in the data processing device 12, the execution unit 28A executes, in an interactive format, questions to the person 20 who is a respondent to the questionnaire survey, and the person 20 answers the questions. Furthermore, the generation unit 28B uses the dialogue generation model 58 to generate summary data 70B2a summarizing the answers from the person 20. Then, the output unit 28F outputs the analysis result data 80 to a predetermined output destination as data based on the summary data 70B2a. This can contribute to reducing the burden on the survey implementing company 16 and the person 20 when collecting answers from the person 20 in accordance with the survey theme 66A in the questionnaire survey, and to collecting highly reliable answers.

[0142] Furthermore, in the data processing device 12, the execution unit 28A outputs the dialogue partner data 70 as voice data to the smart device 14, and acquires answer voice data 72 from the smart device 14. This enables the person 20 to respond to the questionnaire survey in a hands-free manner.

[0143] Furthermore, in the data processing device 12, the execution unit 28A interactively outputs the dialogue partner data 70 as voice data and acquires the answer voice data 72. This makes it easier to obtain answers that are closer to the true feelings of the person 20 who is responding to the questionnaire survey than when a descriptive or multiple choice questionnaire survey is conducted.

[0144] Furthermore, in the data processing device 12, the identification unit 28C identifies the emotion of the person 20 who is the respondent to the questionnaire survey. This allows the emotion of the person 20 to be reflected in the results of the questionnaire survey. Furthermore, the results of the questionnaire survey can be analyzed based on the emotion of the person 20.

[0145] Furthermore, in the data processing device 12, the emotion identification data 76 is generated by using the emotion identification model 60. Therefore, the emotion of the person 20 who is the respondent to the questionnaire survey can be easily identified.

[0146] Furthermore, in the data processing device 12, the data generated based on the emotion identified by the identification unit 28C and the summary data 70B2a is used as the result of the questionnaire survey. Therefore, it is possible to understand the relationship between the answer of the person 20 who is the respondent to the questionnaire survey and the emotion of the person 20.

[0147] Furthermore, in the data processing device 12, analysis result data 80 obtained by analyzing summary data 70B2a based on the emotion identified by identification unit 28C is used as the result of the questionnaire survey. Therefore, a more reliable result of the questionnaire survey can be obtained compared to when the result of the questionnaire survey is summary data 70B2a itself.

[0148] Furthermore, in the data processing device 12, analysis result data 80 obtained by analyzing summary data 70B2a based on the emotion identified by identification unit 28C and the attributes of person 20 who is the respondent to the questionnaire survey is used as the result of the questionnaire survey. Therefore, a more reliable result of the questionnaire survey can be obtained compared to the case where the result of the questionnaire survey is an analysis result obtained by analyzing summary data 70B2a without considering the emotion identified by identification unit 28C and the attributes of person 20 who is the respondent to the questionnaire survey.

[0149] Furthermore, in the data processing device 12, the characteristic of the favorite character A, which is one of the attributes of the person 20 who is the respondent to the questionnaire survey, is imparted to the expression form of the question (e.g., voice and text). In this way, the person 20 who is the respondent to the questionnaire survey will unconsciously answer the questions in the questionnaire survey while enjoying a natural dialogue with character A. This means that the psychological barriers to the questionnaire survey (e.g., aversion, distrust, or resistance, etc.) are lowered. By lowering the psychological barriers to the questionnaire survey, it is possible to obtain answers that are closer to the person 20's true feelings as the result of the questionnaire survey.

[0150] Furthermore, on the condition that the questionnaire survey questions and answers have been exchanged in an interactive format, the data processing device 12 negotiates with the respondent person 20 in an interactive format as to whether or not a summary of the answers obtained in the interactive format may be used as the questionnaire survey results. Therefore, compared to a case where the questionnaire survey questions and answers are exchanged in an interactive format after the person 20 is notified in advance that it is a questionnaire survey, it is possible to obtain answers from the person 20 who is unaware that it is a questionnaire survey as the questionnaire survey results.

[0151] Furthermore, in the data processing device 12, a summary of the answers obtained through the exchange of questions and answers in the questionnaire survey is presented to the person 20 who is the respondent to the questionnaire survey. Thus, the person 20 can confirm what kind of summary will be used as the result of the questionnaire survey.

[0152] Furthermore, the data processing device 12 provides a reward to the person 20 on the condition that the person 20 agrees to a summary of the person's answer to the question from the character A being used as the result of the questionnaire survey. This can contribute to improving the motivation of the person 20 to take the questionnaire survey.

[0153] Furthermore, in the data processing device 12, the analysis result data 80 is output on the condition that the execution unit 28A acquires answer voice data 72 indicating that the person 20 has consented to the summary of the person 20's answer to the question from the character A being used as the result of the questionnaire survey. Therefore, it is possible to prevent the leakage of summaries to which the person 20 has not consented.

[0154] Furthermore, in the data processing device 12, questions that are in line with the survey theme 66A are divided into small pieces and included in the dialogue start data 70A and the question-and-reply data 70B1 generated by using the dialogue generation model 58. Therefore, the person 20 who is responding to the questionnaire survey can enjoy multiple exchanges between questions and answers (e.g., exchanges with character A that the person 20 likes), and the survey implementing company 16 can obtain the responses to the questionnaire survey from the person 20.

[0155] Furthermore, the data processing device 12 includes questions that are in line with the survey theme 66A in the questions posed to the person 20. Therefore, the survey implementing company 16 can obtain answers from the person 20 that are in line with the survey theme 66A.

[0156] In addition, in the data processing device 12, the reception output program 62 is created based on an existing chat application, so it is possible to subject users of the existing chat application to a questionnaire survey. Therefore, the more users of the existing chat application there are, the larger the scale of the questionnaire survey can be. It is also possible to conduct a questionnaire survey using the attributes of users of the chat application.

[0157] In the above embodiment, an example was given in which analysis result data 80 based on summary data 70B2a is output to a default output destination, but the technology of the present disclosure is not limited to this. For example, summary data 70B2a itself may be output to a default output destination, data set 78 itself may be output to a default output destination, or data in which summary data 70B2a is associated with emotion identification data 76 or attribute data 24B may be output to a default output destination.

[0158] In the above embodiment, an example has been given in which the answer voice data 72 is input to the emotion identification model 60, and the emotion identification data 76 corresponding to the answer voice data 72 is output from the emotion identification model 60, but the technology of the present disclosure is not limited to this. For example, image data and answer voice data 72 obtained by capturing an image of the person 20 with the camera 42 (see FIG. 1 ) may be input to the emotion identification model 60, and data indicating the emotion and the degree of the emotion identified from the image data and the answer voice data 72 may be generated and output by the emotion identification model 60.

[0159] In the above embodiment, an example in which the emotion identification data 76 is used in the data set 78 has been given, but the technology of the present disclosure is not limited to this. The emotion identification data 76 may be input to the dialogue generation model 58 together with the answer voice data 72. In this case, the dialogue generation model 58 generates dialogue partner data 70 whose contents correspond to the answer indicated by the answer voice data 72 and the emotion identified from the emotion identification data 76. That is, the dialogue partner data 70 is generated by the dialogue generation model 58 so that a reply that is in line with the emotion of the person 20 is provided to the person 20 as a reply to the answer indicated by the answer voice data 72. In order to achieve this, for example, the template 64 is made to include text data such as "Please create a reply that is in line with the emotion of the respondent as a reply to the answer from the respondent."

[0160] Furthermore, the emotion identification model 60 may determine the emotion of the respondent according to a specific mapping. Specifically, the emotion identification model 60 may determine the emotion of the respondent according to an emotion map (see FIG. 10), which is a specific mapping.

[0161] FIG. 10 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0162] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0163] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0164] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0165] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0166] The emotion identification model 60 inputs the answer voice data 72 into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the respondent. This neural network is pre-trained based on a plurality of training data that are combinations of the answer voice data 72 input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 11. FIG. 11 shows an example in which multiple emotions, "peace of mind," "calm," and "reassuring," have similar emotion values.

[0167] In the above embodiment, an example in which the data processing device 12 assuming a server is implemented in the data processing system 10 is given, but the technology of the present disclosure is not limited to this. For example, the technology of the present disclosure may be implemented in a general information processing system. Also, for example, the technology of the present disclosure may be implemented in a software program that runs on a server and / or a personal computer, or an application that runs on a smart device. Also, the technology of the present disclosure may be provided to a user in the form of SaaS (Software as a Service). Also, in the above embodiment, a server is given as an example of the data processing device 12, but this is merely an example, and the technology of the present disclosure may be realized by a device other than a server (for example, a personal computer, etc.).

[0168] In the above embodiment, an example was given in which the survey implementation processing is performed by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the survey implementation processing may be performed by multiple computers, including computer 22.

[0169] In the above embodiment, an example in which the survey implementation program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the survey implementation program 56 may be stored in a portable computer-readable non-transient storage medium such as a Universal Serial Bus (USB) memory. The survey implementation program 56 stored in the non-transient storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the survey implementation process in accordance with the survey implementation program 56. Alternatively, the survey implementation program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the survey implementation program 56 may be downloaded in response to a request from the data processing device 12 and installed in the computer 22.

[0170] It is not necessary to store all of the survey implementation program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the survey implementation program 56 in the storage 32; only a portion of the survey implementation program 56 may be stored.

[0171] The hardware resources that execute the survey implementation process can be various processors, as shown below. An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource that executes the survey implementation process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, which is a processor having a circuit configuration that is specially designed to execute a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the survey implementation process by using the memory.

[0172] The hardware resource for executing the survey execution process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the survey execution process may be a single processor.

[0173] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes the survey execution process. Second, there is a configuration in which a processor is used that realizes the functions of the entire system including multiple hardware resources that execute the survey execution process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, the survey execution process is realized using one or more of the above-mentioned various processors as hardware resources.

[0174] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor elements. The above-mentioned investigation execution process is merely an example. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processing may be changed without departing from the spirit of the invention.

[0175] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0176] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0177] <Additional Notes> (1) an execution unit that executes questions to a respondent and answers to the questions from the respondent in an interactive format; a generation unit that generates summary data by using a summary data generation model, the summary data being a summary of the answers; an output unit that outputs at least one of the summary data and data based on the summary data. Data processing device.

[0178] (2) The execution unit is outputting question voice data indicating the voice of the question; Acquire answer voice data indicating the answer voice. A data processing device as described in (1).

[0179] (3) The execution unit executes the output of the question voice data and the acquisition of the answer voice data in the interactive manner. A data processing device according to (2).

[0180] (4) The method further includes: A data processing device according to any one of (1) to (3).

[0181] (5) The execution unit acquires answer voice data indicating a voice of the answer, The identification unit identifies the emotion by using an emotion identification model that outputs emotion data related to the emotion corresponding to the answer voice data in response to the input of the answer voice data. A data processing device according to (4).

[0182] (6) The data based on the summary data is data generated based on the emotion identified by the identification unit and the summary data. A data processing device according to (4) or (5).

[0183] (7) The data based on the summary data is an analysis result of the summary data being analyzed based on the emotion identified by the identification unit. A data processing device according to (6).

[0184] (8) An acquisition unit for acquiring attribute data indicating attributes of the respondent, The data based on the summary data is an analysis result of the summary data being analyzed based on the emotion identified by the identification unit and the attribute data acquired by the acquisition unit. A data processing device according to (7).

[0185] (9) An acquisition unit for acquiring attribute data indicating attributes of the respondent, The execution unit is imparting a feature determined according to the attribute data acquired by the acquisition unit to an expression of the question; Outputting the query with the features added to the expression. A data processing device according to any one of (1) to (8).

[0186] (10) the expression is speech, The execution unit is imparting said features to said audio; Outputting question voice data indicating the voice to which the features have been added. A data processing device according to (9).

[0187] (11) The questions and the answers are asked in part for research purposes; The execution unit executes, in the interactive format, a negotiation with the respondent as to whether or not a summary of the answer obtained in the exchange may be used as a result of the survey, on condition that the exchange between the answer and the question has been carried out in the interactive format. A data processing device according to any one of (1) to (10).

[0188] (12) the resulting summary is the summary data; The execution unit executes presentation of the summary data to the respondent. A data processing device according to (11).

[0189] (13) The execution unit obtains consent data indicating that the respondent has consented to the use of the summary as a result of the survey. A data processing device according to (11) or (12).

[0190] (14) The system further includes a rewarding unit that rewards the answerer on condition that the consent data is acquired by the execution unit. A data processing device according to (13).

[0191] (15) The output unit outputs at least one of the summary data and data based on the summary data on condition that the execution unit has acquired the consent data. A data processing device according to (13) or (14).

[0192] (16) the generation unit generates a response to the answer by using a response generation model; The execution unit presents the response generated by the generation unit to the respondent, The answer is part of the question A data processing device according to any one of (1) to (15).

[0193] (17) The questions and the answers are asked in part for research purposes; At least one of the questions and the responses includes content that is in line with the topic of the survey. A data processing device according to (16).

[0194] (18) The questions and the answers are asked in part for research purposes; The questions include content related to the theme of the survey. A data processing device according to any one of (1) to (17).

[0195] (19) an execution unit that executes questions to a respondent and answers to the questions from the respondent in an interactive format; and a determination unit for determining the respondent's emotion based on the response. Data processing device.

[0196] (20) The execution unit acquires answer voice data indicating a voice of the answer, The identification unit identifies the emotion by using an emotion identification model that outputs emotion data related to the emotion corresponding to the answer voice data in response to the input of the answer voice data. A data processing device according to (19).

[0197] (twenty one) The respondent is a user of a chat application. A data processing device according to any one of (1) to (20). [Explanation of symbols]

[0198] 10 Data Processing System 12 Data Processing Device 14 Smart Devices 16 Surveyed companies 18 Survey requesting companies 20 people 22,36 Computer 24 Database 24A ID 24B Attribute Data 24C Remuneration Data 26,44 Communication I / F 28,46 processor 28A Executive Department 28B Generation part 28C Specific part 28D Acquisition Department 28E Analysis Department 28F Output section 28G Granting Section 30,48 RAM 32,50 Storage 34,52 Bus 38 Reception equipment 38A Touch Panel 38B Microphone 40 Output Device 40A Display 40B Speaker 42 Camera 46A Control Unit 54 Network 56 Research Implementation Program 58 Dialogue Generation Model 60 Emotion identification model 62 Reception output program 64 Templates 66 Survey request data 66A Research Topics 66B Subject attribute data 68 First instruction data 70 Interlocutor Data 70A Dialogue initiation data 70Aa Dialogue start voice data 70Ab Dialogue start text data 70B Reply data 70B1 Question and Answer Data 70B2 Summary response data 70Ba Reply voice data 70Bb Reply text data 70B2a Summary Data 70C Dialogue End Data 70Ca End of dialogue voice data 70Cb End of dialogue text data 72 Answer audio data 74 Second instruction data 76 Emotion-specific data 78 Datasets 80 Analysis result data A character

Claims

1. an execution unit that executes questions to a respondent and answers to the questions from the respondent in an interactive format; a generation unit that generates summary data by using a summary data generation model, the summary data being a summary of the answers; an output unit that outputs at least one of the summary data and data based on the summary data. Data processing device.

2. The execution unit is outputting question voice data indicating the voice of the question; Acquire answer voice data indicating the answer voice.

2. A data processing apparatus according to claim 1.

3. The execution unit executes the output of the question voice data and the acquisition of the answer voice data in the interactive manner.

3. A data processing apparatus according to claim 2.

4. The method further includes:

2. A data processing apparatus according to claim 1.

5. The execution unit acquires answer voice data indicating a voice of the answer, The identification unit identifies the emotion by using an emotion identification model that outputs emotion data related to the emotion corresponding to the answer voice data in response to the input of the answer voice data.

5. A data processing apparatus according to claim 4.

6. The data based on the summary data is data generated based on the emotion identified by the identification unit and the summary data.

5. A data processing apparatus according to claim 4.

7. The data based on the summary data is an analysis result of the summary data being analyzed based on the emotion identified by the identification unit.

7. A data processing apparatus according to claim 6.

8. An acquisition unit for acquiring attribute data indicating attributes of the respondent, The data based on the summary data is an analysis result of the summary data based on the emotion identified by the identification unit and the attribute data acquired by the acquisition unit.

8. A data processing apparatus according to claim 7.

9. An acquisition unit for acquiring attribute data indicating attributes of the respondent, The execution unit is imparting a feature determined according to the attribute data acquired by the acquisition unit to an expression of the question; Outputting the query with the features added to the expression.

2. A data processing apparatus according to claim 1.

10. the expression is speech, The execution unit is imparting said features to said audio; Outputting question voice data indicating the voice to which the features have been added.

10. A data processing apparatus according to claim 9.

11. The questions and the answers are asked in part for research purposes; The execution unit executes, in the interactive form, a negotiation with the respondent as to whether or not a summary of the answer obtained in the exchange may be used as a result of the survey, on condition that the exchange between the answer and the question has been carried out in the interactive form.

2. A data processing apparatus according to claim 1.

12. the resulting summary is the summary data; The execution unit executes presentation of the summary data to the respondent. A data processing apparatus according to claim 11.

13. The execution unit obtains consent data indicating that the respondent has consented to the use of the summary as a result of the survey. A data processing apparatus according to claim 11.

14. The system further includes a rewarding unit that rewards the answerer on condition that the consent data is acquired by the execution unit.

14. A data processing apparatus according to claim 13.

15. The output unit outputs at least one of the summary data and data based on the summary data on condition that the execution unit has acquired the consent data.

14. A data processing apparatus according to claim 13.

16. the generation unit generates a response to the answer by using a response generation model; The execution unit presents the response generated by the generation unit to the respondent, The answer is part of the question 2. A data processing apparatus according to claim 1.

17. The questions and the answers are asked in part for research purposes; At least one of the questions and the responses includes content that is in line with the theme of the survey.

17. A data processing apparatus according to claim 16.

18. The questions and the answers are asked in part for research purposes; The questions include content related to the theme of the survey.

2. A data processing apparatus according to claim 1.

19. an execution unit that executes questions to a respondent and answers to the questions from the respondent in an interactive format; and a determination unit for determining the respondent's emotion based on the response. Data processing device.

20. The execution unit acquires answer voice data indicating a voice of the answer, The identification unit identifies the emotion by using an emotion identification model that outputs emotion data related to the emotion corresponding to the answer voice data in response to the input of the answer voice data.

20. A data processing apparatus according to claim 19.

21. The respondent is a user of a chat application.

2. A data processing apparatus according to claim 1.

22. A communication terminal capable of communicating with the data processing device according to any one of claims 1 to 21, A user interface is provided. The user interface includes: posing the question to be executed in the interactive manner by the execution unit; and accepting the answer to be executed in the interactive manner by the execution unit. Communications terminal.

23. interactively conducting questions to respondents and responses to said questions from said respondents; generating summary data summarizing the responses by using a summary data generation model; and outputting at least one of the summary data and data based on the summary data. Data processing methods.

24. interactively asking questions to respondents and receiving answers to said questions from said respondents; and determining a sentiment of the respondent based on the response. Data processing methods.

25. interactively conducting questions to respondents and responses to said questions from said respondents; generating summary data summarizing the responses by using a summary data generation model; and A program for causing a computer to execute a process including outputting at least one of the summary data and data based on the summary data.

26. interactively asking questions to respondents and receiving answers to said questions from said respondents; and A program for causing a computer to execute a process including identifying the respondent's emotion based on the answer.

Images