Data processing apparatus, data processing method, and program

Abstract

To obtain a result of specific processing on a user to be analyzed, from interaction data, to be utilized for improving communication.SOLUTION: A data processing apparatus includes: an input unit for receiving a user input; a processing unit which performs specific processing using a generation model that generates a predetermined inference result according to input data; and an output unit which outputs a result of the specific processing to a predetermined device. The input unit receives, as the user input, interaction data related to a user to be analyzed. The processing unit acquires a result of the specific processing on the user to be analyzed, using an output of the generation model that uses information obtained from the user input as the input data. The output unit visualizes the result of the specific processing, for output.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The technology disclosed herein relates to a data processing device, a data processing method, and a program. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Recently, various social issues related to one-on-one communication have arisen. For example, in communication between managers and subordinates in a company, communication issues have arisen due to the increase in text-based exchanges accompanying the spread of remote work. For example, it is difficult for a superior to understand the situation or feelings of a subordinate, and it is difficult for a subordinate to consult with a superior. When applying conventional technology to improve such communication issues, a mechanism that encourages appropriate communication by speech generation is assumed, but there is room for improvement in obtaining appropriate processing results for user input. [Means for solving the problem]

[0005] A data processing device of a first aspect related to the technology disclosed herein includes an input unit that accepts user input, a processing unit that performs a specific processing using a generative model that generates a predetermined inference result according to the input data, and an output unit that outputs a result of the specific processing to a predetermined device, wherein the input unit accepts dialogue data related to a user to be analyzed as the user input, the processing unit obtains a result of the specific processing related to the user to be analyzed using the output of the generative model when information obtained from the user input is used as the input data, and the output unit visualizes and outputs the result of the specific processing.

[0006] In a data processing device of a second aspect of the present invention, the processing unit acquires a summary of the dialogue data as a first process of the identification process, acquires analysis results related to the analysis parameters as a second process of the identification process using a prompt including an output format of the analysis parameters defined in the dialogue data as input to the generative model, and acquires presentations of specified cases for the dialogue data as a third process of the identification process, and the output unit displays the summary and the presentations of the cases in text form as the visualization, and displays the analysis results including graphs using the analysis parameters.

[0007] In a data processing device of a third aspect of the present invention, the users are a first user who provides user input and a second user who is the subject of analysis, and the dialogue data is a video or audio of the dialogue between the first user and the second user. In the second process, the analysis results are obtained for at least the second user among the users, and in the third process, a presentation of a case example in the case where the first user presents the result to the second user is obtained.

[0008] In a data processing device of a fourth aspect of the present invention, in the second process, the processing unit sets a range of the user's emotional value in an output format, and the output unit outputs a predetermined graph of the emotional value obtained as a result of the analysis.

[0009] In the data processing device of the fifth aspect of the present invention, the output unit visualizes and displays the analysis results for each predetermined section divided into the dialogue data.

[0010] A data processing method of a sixth aspect related to the technology of the present disclosure is a data processing method in which at least one processor of a computer constituting a data processing device executes the steps of accepting user input, performing specific processing using a generative model that generates a predetermined inference result according to the input data, and outputting a result of the specific processing to a predetermined device, wherein the accepting user input step accepts dialogue data related to the user to be analyzed as the user input, the performing specific processing step obtains a result of the specific processing related to the user to be analyzed using the output of the generative model when information obtained from the user input is used as the input data, and the outputting step visualizes and outputs the result of the specific processing.

[0011] A seventh aspect of the program related to the technology disclosed herein is a program that causes at least one processor of a computer constituting a data processing device to execute a process of accepting user input, a process of performing specific processing using a generative model that generates a predetermined inference result according to the input data, and a process of outputting a result of the specific processing to a predetermined device, wherein the process of accepting user input accepts, as the user input, dialogue data related to the user to be analyzed, the process of performing the specific processing obtains a result of the specific processing related to the user to be analyzed using the output of the generative model when information obtained from the user input is used as the input data, and the process of outputting visualizes and outputs the result of the specific processing. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2] FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a smart device. [Figure 3A]1 shows an example of a dashboard interface for specific actions regarding a user to be analyzed. [Figure 3B] 1 shows an example of a dashboard interface for specific actions regarding a user to be analyzed. [Figure 3C] 1 shows an example of a dashboard interface for specific actions regarding a user to be analyzed. [Figure 4] 2 shows a schematic functional configuration of a specific processing unit of the data processing device. [Diagram 5] 13 illustrates an example of an operational flow of a specific process performed by a data processing device. [Figure 6A] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 6B] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0016] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0021] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server. An example of the smart device 14 is a smartphone. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure. Note that various functional configurations of the data processing device according to the technology of the present disclosure may be installed in devices other than the data processing device 12 described below, such as the smart device 14. In addition, the smart device 14 is not limited to a smartphone, and may be a tablet terminal or a wearable terminal. In addition, the smart device 14 may be replaced with a personal computer.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0023] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives a user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive a user input by the touch of the pointer. The microphone 38B detects the user's voice to receive a user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. The received user input may be stored in the storage 50, and the user input may be transmitted to the data processing device 12 under the control of the control unit 46A as necessary for data processing. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the person 20 by outputting the data in a form of expression that the person 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58 and an emotion identification model 60 as examples of a generation model. The data generation model 58 and the emotion identification model 60 are used by the identification processing unit 290. Note that in this embodiment, a case where the data generation model 58 is mainly used as the generation model will be described as an example.

[0030] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as text data, voice data, analysis data, and drawing data. The data format can be specified by the prompt, and the data generation model 58 outputs the inference result in the type of data format according to the specified data format. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. As with the data generation model 58 described above as an example, the generation model is capable of generating inference results such as sentences and analysis data according to the input data.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 62. The reception output program 62 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 62 from the storage 50, and executes the read reception output program 62 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0032] Next, a description will be given of the processing of the identification processing unit 290 when the data processing device 12 in the data processing system 10 performs identification processing on the user to be analyzed. The data processing device 12 performs identification processing (first to third processes) on the user to be analyzed. The aspects of the identification processing (first to third processes) will be described later.

[0033] 3A to 3C show an example of an interface of a dashboard for specific processing related to a user to be analyzed. As shown in FIG. 3A to FIG. 3C, a dashboard (3d) is displayed as an interface for analysis in the output device 40 of the smart device 14. The dashboard (3d) is an interface for analysis displayed on the smart device 14. A user who holds the smart device 14 can perform operations required for analysis and view analysis results (results of specific processing) via the dashboard (3d). When input of dialogue data is accepted as user input from the dashboard (3d), dialogue data which is the user input is transmitted from the smart device 14 to the data processing device 12 under the control of the control unit 46A. The data processing device 12 obtains the results of various specific processing related to the user to be analyzed from the accepted user input and returns it to the smart device 14. Note that the dashboard may accept additional data for conditioning to be added as a prompt instruction as well as dialogue data, in the form of text, image, or the like, as user input, and transmit it to the data processing device 12.

[0034] FIG. 3A shows a state in which dialogue data is input into an input box (3i), and a video (3v), a transcript (3t), and a summary (3s) of the dialogue data are generated and displayed. FIG. 3B shows a state in which a graph (3g) of emotion analysis results is generated from the dialogue data and displayed. FIG. 3C shows a state in which a presented example (3e) is generated from the dialogue data and displayed. Note that the results of various specific processes displayed on the dashboard (3d) may be displayed on one screen by scrolling, or a switching button may be provided to switch the display. Note that in the following description, for the sake of convenience, the symbols in FIGS. 3A to 3C will be omitted.

[0035] 4, the specific processing unit 290 includes an input unit 292, a pre-processing unit 296, a processing unit 296, and an output unit 298. The processing unit 296 includes a first processing unit, a second processing unit, and a third processing unit.

[0036] In the present embodiment, as an example, users are classified into a first user who performs user input to the smart device 14 and a second user who is the subject of analysis. In the following, an example will be described in which the first user performs user input and the result of the specific processing in which the second user is analyzed is acquired. For example, when the data processing system 10 of the present embodiment is used for the purpose of improving communication in a company, the first user is a manager who is a superior, and the second user is a subordinate. In addition, when used in a scene such as a corporate interview, the first user may be an agent, and it is also assumed that the first user is an agent capable of automatically responding to the interview. When the first user is an agent, for example, it is assumed that the first user is used in a first interview using AI. In addition, when used to support the training of corporate managers and interviewers, the first user becomes an agent who responds to simulated subordinates or applicants, and the second user can be a manager, interviewer, or a candidate. In addition, the data processing system 10 of the present embodiment is assumed to be used in various 1-on-1 communication situations such as counseling, online diagnosis, customer support, educational sites, and personal coaching. By utilizing the data processing system 10 of this embodiment, a relationship of trust in communication can be built, and communication between users is expected to improve. In addition, by utilizing the data processing system 10 in a company, communication throughout the organization is expected to improve.

[0037] The input unit 292 accepts user input from the smart device 14. Specifically, dialogue data related to the user to be analyzed is acquired as the user input. The dialogue data may be video data or audio data of a dialogue between a first user and a second user, and may be acquired by inputting from an input box on the dashboard shown in FIG. 3A, for example. The dialogue data may be dialogue data in which the second user answers questions prepared in advance.

[0038] The pre-processing unit 296 transcribes the dialogue data. As a result, the transcription data of the dialogue data is extracted as information obtained from the user input (dialogue data). The pre-processing unit 296 outputs the extracted transcription data of the dialogue data to the processing unit 296 as the user input. The transcription extraction process may be performed by dividing the voice data at intervals of several tens of seconds and inputting the divided data to an arbitrary API for transcription, such as Whisper by OpenAI. In addition, when playing a video on the dashboard, the video may be divided into units of several minutes to several tens of minutes.

[0039] The processing unit 296 performs the first to third processes as the specific process using the data generation model 58. First, the processing unit 296 judges whether or not a predetermined trigger condition is satisfied. Specifically, the trigger condition may be that the processing of the pre-processing unit 296 is completed and the transcription data of the dialogue data is accepted. Alternatively, the trigger condition may be that the dialogue data is input, and the specific process may be performed by accepting inputs one by one in the order in which the transcription data is processed. Also, an analysis start button or the like may be provided on the dashboard, and the trigger condition may be that the analysis start button is pressed after the dialogue data is input. Also, the trigger condition may be a start signal input by voice from the first user (for example, a phrase such as "Please start analysis").

[0040] When the trigger condition is satisfied, the processing unit 296 performs the first to third processes as the specific processes. The processing unit 296 acquires the results of the specific processes, namely, the first process is a summary, the second process is a sentiment analysis, and the third process is a presentation of examples. Each of these will be explained below.

[0041] As a first process of the specific process, the processing unit 296 inputs text (instruction statement) representing an instruction for obtaining a summary of the dialogue data and the transcription data of the dialogue data to the data generation model 58, and acquires a summary as a processing result based on the output of the data generation model 58. Specifically, in the first process, a prompt (first prompt) is generated by adding the transcription data of the dialogue data together with an instruction statement such as "Please summarize the following sentence", and inputs it to the generation model. Note that a restriction on the number of characters in the summary or a designation of topic extraction may be added to the instruction statement. The topic extraction is, for example, a designation to focus on extracting and summarizing topics related to a specific task. This allows the content on which the first user places emphasis in the analysis to be extracted as a summary.

[0042] As the second process of the specific process, the processing unit 296 inputs an instruction sentence for obtaining an emotion analysis result of the second user in the dialogue data and the transcription data of the dialogue data to the data generation model 58, and obtains an emotion analysis result of the second user based on the output of the data generation model 58. In the example shown in FIG. 3B, the emotion analysis result is obtained by dividing the dialogue data into a first half and a second half section. The first half is judged to be "negative" and the second half is judged to be "positive". In addition, the emotion analysis result graphs for each section in the first half and the second half are visualized and shown together with the review, with the emotion values ​​of "fun, surprise, fear, anger, sadness, and expectation". In this way, in the second process, the analysis result graph is visualized for each section into which the dialogue data is divided. Note that the division of the sections is arbitrary, and may be divided into a first half, a middle part, a second half, a first part to an nth part, or by topic depending on the length. The input mode for obtaining such an emotion analysis result will be described below.

[0043] Specifically, in the second process, for example, an output format template and a prompt (second prompt) with transcription data of the dialogue data are generated together with instructions for specifying the sentiment analysis and output format, and input to the data generation model 58. An example of the instructions can be the text "You are one of the world's leading psychoanalysts. Please perform a psychological analysis based on the following text. The response output format shall follow the format below." The following is an example of an output format template, showing a template for determining whether a person is positive or negative, analyzing the sentiment value in the range of 1 to 10, and outputting a review. [result] positive or negative [detail] joy:0~10 surprise:0~10 fear:0~10 anger:0~10 sadness:0~10 anticipation:0~10 [review]

[0044] The above output format is an example, and the type and range of the emotion value to be included in the graph can be set arbitrarily. For example, parameters such as trust (0-10) and disgust (0-10) may be added to the emotion value. In addition, the analysis parameters may be set not only to the emotion value of the user, but also to the work itself, such as motivation and interest in the work, and an arbitrary value range suitable for the service may be set as the output format. When applying to interviews and counseling scenes, the user's personality diagnosis may be the subject of analysis, and an analysis such as introversion / extroversion values ​​and compatibility values ​​for each personality type may be set as the output format. The above-mentioned negative / positive judgment, and the value range of the emotion and personality related to the user are examples of the output format of the analysis parameters of the present disclosure. In addition, the graph in FIG. 3B is an example of a radar chart, but is not limited to this, and may be set to any other format, such as a bar graph, a pie chart, a heat map, or other arbitrary graph.

[0045] As a third process of the specific process, the processing unit 296 inputs an instruction sentence for obtaining a presentation of a case from the first user to the second user and the transcription data of the dialogue data to the data generation model 58, and acquires the presentation of the case based on the output of the data generation model 58. In the example shown in FIG. 3C, "feedback example", "next action example", "method to increase motivation", and "method to improve skills" are acquired as the presentation of the case. Specifically, in the third process, for example, a prompt (third prompt) is generated in which the transcription data of the dialogue data is added together with an instruction sentence such as "You are a boss. Please give advice from the following conversation with your subordinate, such as '〇〇', '△△'..." (the type of case is in parentheses), and the prompt is input to the data generation model 58. Note that the type of case may be set to any content depending on the service.

[0046] The processing unit 296 may receive a user state including the user's behavior as a user input and perform a specific process. The processing unit 296 may also acquire the behavior of the second user in advance and generate an instruction sentence based on the behavior. For example, it is assumed that the overtime behavior of the second user is acquired, and text indicating a condition such as "This user's average overtime hours are xx hours. Please analyze the emotion value of xx taking the overtime hours into consideration" is added as additional data to the instruction sentence of the prompt.

[0047] Furthermore, the emotion analysis of the user by the second process may be not limited to the second user to be analyzed, but may also be performed on the first user. For example, the emotions of both users may be analyzed, and the process of the emotion change between the users in the dialogue may be observable. By analyzing the emotions of the first user as well, it becomes possible to provide feedback taking into consideration the state of the first user himself / herself as the observer. For example, by analyzing the emotion changes of the first user, such as a manager or interviewer, it becomes possible to provide feedback on whether the dialogue in the interview or the interview was appropriate (or was appropriate), and on the tendency of the first user's impression of the second user in the dialogue.

[0048] The output unit 298 visualizes and outputs the result of the specific process to the smart device 14. At this time, the smart device 14 outputs the result of the specific process to the display and displays it as an analysis result on the dashboard exemplified in FIG. 3A to FIG. 3C. For the first and third processes, the output unit 298 displays the summary and the presentation of the case in text form. For the second process, the output unit 298 outputs the summary so as to include a graph of the emotion value obtained as the analysis result for each section such as the first half and the second half exemplified above. In addition, the output unit 298 may output the summary as audio or send a message including a link to the summary and the dashboard of the analysis result to a message application installed on the smart device 14 as an output mode other than display.

[0049] Here, the terms "user input" and "input data" related to the technology of the present disclosure will be summarized. The user input is data acquired from the smart device 14 and is input for the purpose of analyzing the user. Here, the subject who inputs the user input is the first user, and the user input refers to data input for analyzing the second user. That is, the user input is mainly dialogue data, and additional data for conditioning may be included according to any input operation of the user. The dialogue data is, as described above, video data or audio data of the dialogue. Next, the input data refers to a prompt that is input to the data generation model 58 and generated to obtain a desired output result. That is, the input data corresponds to the first prompt, the second prompt, and the third prompt described above. In addition, the transcription data extracted from the dialogue data by the preprocessing unit 296, that is, the "transcription data of the dialogue data" refers semantically to information obtained from the dialogue data. Therefore, the transcription data of the dialogue data is an example of "information obtained from the user input" of the present disclosure. The first prompt is a prompt to which the transcription data of the dialogue data is added together with an instruction sentence for obtaining a summary. An example of the second prompt is a prompt to which an output format template and transcription data of the dialogue data are added together with instructions for sentiment analysis and output format designation. The third prompt is a prompt to which transcription data of the dialogue data are added together with instructions for obtaining the presentation of examples. As described above, various prompts, which are input data of the present disclosure, are generated by adding information obtained from user input (transcription data of the dialogue data) to predetermined instructions to obtain a desired result. As described above, generating a prompt by adding information obtained from user input together with instructions is an example of "when information obtained from user input is used as input data" of the present disclosure. In addition, the second prompt is generated by further adding an output format template of the analysis parameters.

[0050] Fig. 5 shows an example of an operational flow of the operation of the identification processing unit 290 performing identification processing on a user to be analyzed. The operational flow shown in Fig. 5 is automatically executed repeatedly, for example, at regular time intervals. Note that the following first to third processes may be executed in parallel.

[0051] In step S300, the input unit 292 obtains interaction data as a user input.

[0052] In step S302, the preprocessing unit 296 extracts transcription data from the dialogue data.

[0053] In step S304, the processing unit 296 determines whether or not a predetermined trigger condition is satisfied. If the trigger condition is satisfied, the process proceeds to step S306. On the other hand, if the trigger condition is not satisfied, the identification process ends. If the trigger condition is not satisfied, it is assumed that the process of acquiring dialogue data in step S300 or extracting transcription data in step S302 is not completed due to some factor. If the trigger condition is satisfied, the first to third processes are performed as the identification process.

[0054] In step S306, as a first process, processing unit 296 generates a first prompt to which transcription data of the dialogue data is added together with an instruction sentence for obtaining a summary of the dialogue data.

[0055] In step S308, as a first process, the processing unit 296 inputs the generated first prompt to a generative model, and obtains a summary result based on the output of the generative model.

[0056] In step S310, as a second process, the processing unit 296 generates a second prompt to which an output format template and transcription data of the dialogue data are added, together with instructions for sentiment analysis and designation of an output format.

[0057] In step S312, as a second process, the processing unit 296 inputs the generated second prompt into a generative model, and acquires an analysis result of the emotion of the second user based on the output of the generative model.

[0058] In step S314, as a third process, the processing unit 296 generates a third prompt to which the transcription data of the dialogue data is added together with an instruction text for obtaining a presentation of an example from the first user to the second user.

[0059] In step S316, as a third process, the processing unit 296 inputs the generated third prompt into a generative model, and obtains presentation of a case based on the output of the generative model.

[0060] In step S318, the output unit 298 outputs the results of the specific processing from the first process to the third process to the smart device 14, and ends the specific processing. Through this processing, the displays exemplified in Fig. 3A to Fig. 3C are displayed on the display 40A of the smart device 14 as the output of the specific processing. Note that the processing of the output unit 298 may be sequential processing, and the first process to the third process may be displayed in the order in which they were processed.

[0061] As described above, the data processing device 12 can obtain analysis results that can be used to improve communication.

[0062] The data processing device 12 that executes the above-mentioned specific processing may be built into a robot manufactured for communication support. In this case, dialogue data may be transmitted from the smart device 14 to the robot to execute the processing. Also, the robot itself may be considered as an agent capable of spontaneous dialogue, and as an agent system, it may be considered as a first user that mediates or engages in dialogue and obtains dialogue data.

[0063] In addition, if the smart device 14 is a wearable terminal, it may be assumed that the wearable terminal is worn by the first user or the second user to be involved in a real-time dialogue, and that dialogue data is acquired and analyzed in real time during the dialogue.

[0064] Also, in a dashboard operable on the smart device 14, it may be possible to select to acquire and analyze dialogue data in the middle of a dialogue in real time. By real-time sequential processing, dialogue data in the middle of a dialogue may be used as an input, and results of specific processing such as a summary in the middle of the dialogue, sentiment analysis, and case presentation may be acquired and output to the smart device 14. As for the division of dialogue data in the middle of a dialogue, section divisions may be set, and each section may be analyzed in real time. The section divisions may be set, for example, to be divided by (1) time divisions, (2) user operation divisions, and (3) predetermined keyword divisions in the dialogue. As for (2) operation divisions, the start and end of a section may be specified by an operation from the dashboard user. As for (3) keyword divisions in the middle of a dialogue, a start keyword and an end keyword of the section may be set, and the start and end of the section may be extracted from the dialogue data. For example, it may be extracted that keywords such as "Let's start with XX (start keyword)" and "Let's finish asking about XX (end keyword)" are included in the utterance of the first user during the dialogue, and the start and end of the section may be determined. In this way, by feeding back the analysis results of the user in real time during the dialogue, the analysis results can be immediately reflected in the communication during the dialogue. In addition, by feeding back the analysis results of the dialogue in real time to the first user, it is possible to optimize the communication during the dialogue and correct its course.

[0065] (Modification) In the above embodiment, the data generation model 58 is used for emotion analysis in the second process in the processing unit 296, but the present invention is not limited to this, and the emotion identification model 60 may be used to analyze the user's emotion. In this case, the processing unit 296 may estimate the user's emotion using the data generation model 58 for the first process and the third process, and the emotion identification model 60 for the second process, and obtain the analysis result of the user's emotion.

[0066] The emotion identification model 60 is, for example, a model that can determine the emotion of a user according to a specific mapping. Specifically, the emotion identification model 60 may determine the emotion of a user according to an emotion map (see FIG. 6A), which is a specific mapping. For example, when the data processing system 10 is utilized in a real-time dialogue, the emotion identification model 60 can be used to intuitively understand the user to be analyzed. For example, by inputting real-time dialogue data into the emotion identification model 60 and analyzing the emotion of the user, the emotion map can be displayed in real time on the dashboard as the emotion analysis result of the second process. This allows the change in the user's emotion to be observed immediately during the dialogue, which can promote smoother communication.

[0067] FIG. 6A is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0068] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0069] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0070] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0071] The emotion map defines two emotions that encourage learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, it is when the user experiences negative emotions such as "I never want to feel this way again" or "I never want to be scolded again." The other is the positive emotion around "desire" on the reaction side. In other words, it is when the user has positive feelings such as "I want more" or "I want to know more."

[0072] The emotion identification model 60 inputs user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. In addition, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 6B. FIG. 6B shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0073] In addition, in the second process, the emotion value estimated by the emotion identification model 60 may be assigned to the prompt condition and input to the data generation model 58 to generate a review. For example, an instruction sentence such as "This is the user's emotion value for the following sentence. Please perform a psychological analysis based on the emotion value and the sentence" may be generated, and [review] may be specified as the output format. Also, sections of the dialogue data may be divided according to changes and patterns in the emotion value of the emotion map. In this way, the user can be analyzed by focusing on the user's emotions during the dialogue. This concludes the explanation of the modified example.

[0074] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0075] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0076] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0077] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0078] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0079] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0080] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0081] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The above specific processes are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0082] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0083] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0084] 10 Data Processing System 12 Data Processing Device 14 Smart Devices 58 Data Generation Model 60 Emotion identification model 290 Specific Processing Department 292 Input section 294 Pretreatment section 296 Processing section 298 Output section

Claims

1. an input unit for accepting user input; A processing unit that performs a specific process using a generative model that generates a predetermined inference result according to input data; an output unit that outputs a result of the specific processing to a predetermined device, The input unit accepts, as the user input, dialogue data regarding a user to be analyzed; the processing unit acquires a result of the identification process for the user to be analyzed by using an output of the generative model when information obtained from the user input is used as the input data; and The output unit visualizes and outputs a result of the specific processing.

2. A data processing device comprising:

2. The processing unit acquires a summary of the dialogue data as a first process of the identification process, As a second process of the identification process, a prompt including an output format of the analysis parameter defined in the dialogue data is input to the generative model to obtain an analysis result regarding the analysis parameter; As a third process of the identification process, a presentation of a predetermined example for the dialogue data is obtained; The data processing apparatus according to claim 1 , wherein the output unit displays, as the visualization, the summary and the presentation of the cases in text form, and displays the analysis results so as to include a graph using the analysis parameters.

3. The users are a first user who performs the user input and a second user who is the analysis target, and the dialogue data is a video or audio of a dialogue between the first user and the second user; In the second process, the analysis result is obtained for at least the second user among the users; The data processing apparatus according to claim 2 , wherein the third process acquires a presentation of a case example from the first user when the case example is to be presented to the second user.

4. In the second process, the processing unit sets a range of the emotion value of the user in an output format; The data processing device according to claim 3 , wherein the output section outputs a predetermined graph of the emotion value obtained as the analysis result.

5. The data processing apparatus according to claim 2 , wherein the output unit visualizes and displays the analysis results for each of predetermined sections divided into the dialogue data.

6. At least one processor of a computer constituting a data processing device accepting user input; performing a specific process using a generative model that generates a predetermined inference result according to input data; and outputting a result of the specific processing to a predetermined device, The step of accepting the user input accepts, as the user input, dialogue data relating to a user to be analyzed; the step of performing the identification process includes acquiring a result of the identification process related to the user to be analyzed by using an output of the generative model when information obtained from the user input is used as the input data; The outputting step includes visualizing and outputting the result of the specific processing. Data processing methods.

7. At least one processor of a computer constituting a data processing device A process for accepting user input; A process of performing a specific process using a generative model that generates a predetermined inference result according to input data; and a process of outputting a result of the specific process to a predetermined device, The process of accepting the user input includes accepting, as the user input, dialogue data regarding a user to be analyzed; The process of performing the identification process includes obtaining a result of the identification process for the user to be analyzed by using an output of the generative model when information obtained from the user input is used as the input data, and The output process visualizes and outputs the result of the specific process. program.

Images