Program, method, information processing device, and system

Abstract

This program causes a computer including a processor and a storage unit to process information regarding a dialog among a plurality of users. The processor executes: an analysis data acquisition step for acquiring analysis data obtained by performing analysis on the dialog; and an input data creation step for creating input data to be input to the generative AI on the basis of the analysis data acquired in the analysis data acquisition step.

Description

Program, method, information processing device, and system

[0001] The present disclosure relates to a program, a method, an information processing device, and a system.

[0002] There are known techniques for analyzing call information. Patent Document 1 discloses a technique for analyzing call information.

[0003] Re-table 2020 / 184631 publication

[0004] There is a problem that analysis data relating to a dialogue between multiple users is difficult for users to understand simply by being presented with the analysis data. Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology for creating input data such as prompts to be input to a generation AI such as a large-scale language model to obtain answer content (feedback such as comments) from analysis data relating to the dialogue in a manner that is easy for users to understand.

[0005] A program that causes a computer having a processor and a memory unit to process information regarding a dialogue between multiple users, the program executing an analysis data acquisition step in which the processor acquires analysis data obtained by analyzing the dialogue, and an input data creation step in which input data to be input to a generation AI is created based on the analysis data acquired in the analysis data acquisition step.

[0006] According to the present disclosure, it is possible to create input data such as prompts for inputting analytical data related to a dialogue into a generation AI such as a large-scale language model to obtain answer content (feedback such as comments) in a manner that is easy for users to understand.

[0007] 1 is a block diagram showing the functional configuration of the system 1. FIG. 1 is a block diagram showing the functional configuration of the server 10. FIG. 1 is a block diagram showing the functional configuration of the first user terminal 20. FIG. 2 is a block diagram showing the functional configuration of the second user terminal 30. FIG. 2 is a diagram showing the data structure of a user table 1012. FIG. 3 is a diagram showing the data structure of a group table 1013. FIG. 4 is a diagram showing the data structure of a dialogue table 1014. FIG. 5 is a diagram showing the data structure of a label table 1015. FIG. 6 is a diagram showing the data structure of a speech segment table 1016. FIG. 7 is a diagram showing the data structure of a comment table 1021. A flowchart showing the operation of comment processing. An example screen showing the operation of comment processing. FIG. 8 is a block diagram showing the basic hardware configuration of a computer 90.

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0009] <Configuration of System 1> System 1 in the present disclosure is an information processing system that provides an information processing service for efficiently managing inquiries from customers via telephone, etc. System 1 includes information processing devices, such as a server 10, a first user terminal 20, a second user terminal 30, a voice server (PBX) 50, and a generation AI 80, all connected via a network N. FIG. 1 is a block diagram showing the functional configuration of system 1. FIG. 2 is a block diagram showing the functional configuration of the server 10. FIG. 3 is a block diagram showing the functional configuration of the first user terminal 20. FIG. 4 is a block diagram showing the functional configuration of the second user terminal 30.

[0010] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by this hardware configuration will be described later. For each of the server 10, the first user terminal 20, the second user terminal 30, the voice server (PBX) 50, and the generation AI 80, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0011] <Configuration of Server 10> The server 10 is an information processing device that provides an information processing service that executes predetermined information processing in response to inquiries from customers via telephone or the like. The server 10 in the present disclosure is an information processing device that provides an online interactive service (online interactive service) between a first user who is an operator and a second user who is a customer. Note that the server 10 in the present disclosure may also be capable of providing an online interactive service between three or more users, including multiple operators and multiple customers. Note that the customers do not necessarily have to be users of the information processing service according to the present disclosure. The server 10 includes a storage unit 101 and a control unit 104.

[0012] <Configuration of Storage Unit 101 of Server 10 > The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a group table 1013 , a dialogue table 1014 , a label table 1015 , a speech segment table 1016 , and a comment table 1021 .

[0013] The application programs 1011 are programs for causing the control unit 104 of the server 10 to function as each functional unit. The application programs 1011 include applications such as a web browser application.

[0014] The user table 1012 is a table that stores and manages user information. When a user registers to use a service, the user's information is stored in a new record in the user table 1012. This allows the user to use the service according to the present disclosure. In the present disclosure, the user table 1012 is a table that uses the user ID as the primary key and has columns for the user ID, group ID, and user name. FIG. 5 is a diagram showing the data structure of the user table 1012.

[0015] The user ID is an item that stores user identification information for identifying a user. The user identification information is an item in which a unique value is set for each user. The group ID is an item that stores group identification information for identifying a group. One or more group identification information is stored in association with each user, thereby expressing that the user belongs to one or more groups. The user name is an item that stores the user's name. Instead of a name, the user name may be set to any character string, such as a nickname.

[0016] The group table 1013 is a table that stores and manages information (group information) about groups to which users belong. Groups include any group, such as a company, corporation, corporate group, club, or various organizations. Groups may also be defined for more detailed subgroups, such as company departments (sales department, general affairs department, customer support department). The group table 1013 is a table that uses the group ID as the primary key and has columns for group ID, group name, and group attributes. FIG. 6 is a diagram showing the data structure of the group table 1013.

[0017] The group ID is an item that stores group identification information for identifying a group. The group identification information is an item in which a unique value is set for each piece of group information. The group name is an item that stores the name of the group. Any character string can be set as the group name. The group attribute is an item that stores information about the attributes of the group, such as the group type (company, corporate group, other organization, etc.) and industry (real estate, finance, etc.).

[0018] The dialogue table 1014 is a table for storing and managing information (dialogue information) related to dialogues between users and customers. The dialogue table 1014 is a table with a dialogue ID as a primary key and has columns for dialogue ID, user ID, customer ID, dialogue category, sending / receiving type, audio data, and video data. Figure 7 shows the data structure of the dialogue table 1014.

[0019] The conversation ID is a field that stores conversation identification information for identifying a conversation. The conversation identification information is a field that has a unique value set for each conversation information. The user ID is a field that stores user identification information for identifying a user in a conversation between a user and a customer. Multiple user IDs may be associated with each conversation information. The customer ID is a field that stores user identification information for identifying a customer in a conversation between a user and a customer. Multiple user IDs may be associated with each conversation information. The conversation category is a field that stores the type (category) of the conversation between the user and the customer. The conversation data is classified by the conversation category. The conversation category stores values ​​such as telephone user, telemarketing, customer support, and technical support depending on the purpose of the conversation between the user and the customer. The call type is a field that stores information for distinguishing whether the conversation between the user and the customer was initiated by the user (outbound) or received by the user (inbound). In addition, when a conversation involves three or more users, the call type "room" is stored. The audio data field stores audio data collected by a microphone. It may also store reference information (path) for audio data files located elsewhere. The audio data may be in any format, such as AAC, ATRAC, MP3, or MP4. The audio data may be in a format in which identifiers are set that allow the user's voice and the customer's voice to be independently identified. In this case, the control unit 104 of the server 10 can perform independent analysis processing on the user's voice and the customer's voice. Furthermore, the user ID and customer ID can be identified based on the user's and customer's voice data. In the present disclosure, video data containing audio information may be used instead of audio data. Furthermore, audio data in the present disclosure also includes audio data included in video data. Data in other data formats associated with various data may also be saved. For example, data such as contract documents, minutes, presentation files, or emails may be included.Video data is an item that stores video data captured by a camera or the like. It may also store reference information (path) for video data files located elsewhere. The video data may be in any format, such as MP4, MOV, WMV, AVI, or AVCHD. The video data may be in a format in which identifiers are set that allow the user's video and the customer's video to be independently identified. In this case, the control unit 104 of the server 10 can perform independent analysis processing on the user's video and the customer's video. Furthermore, the user ID and customer ID can be identified based on the user's and customer's video data.

[0020] The label table 1015 is a table for storing and managing information about labels (label information). The label table 1015 has columns for conversation IDs and label data. FIG. 8 shows the data structure of the label table 1015.

[0021] The dialogue ID is an item that stores dialogue identification information for identifying a dialogue. The label data is an item that stores label information for managing dialogues. The label information is additional information for managing dialogue information, such as a classification name, label, classification label, or tag. The label data may be a character string indicating the name of the label information, or may be a label ID for referencing the name of label information stored in another table. The label data includes classification information according to the emotional state of a speaker in a specific dialogue. The classification data includes classification information for classifying the speaker's response in a specific dialogue as good or bad.

[0022] The voice segment table 1016 is a table for storing and managing information (voice segment information) relating to multiple voice segments included in dialogue information. The voice segment table 1016 is a table having a section ID as a primary key and columns for section ID, dialogue ID, speaker ID, start date and time, end date and time, section voice data, section video data, and section read-aloud text. Fig. 9 is a diagram showing the data structure of the voice segment table 1016.

[0023] The section ID is an item that stores section identification information for identifying a voice section. The section identification information is an item that is set to a unique value for each voice section information. The dialogue ID is an item that stores dialogue identification information for identifying a dialogue with which the voice section information is associated. The speaker ID is an item that stores speaker identification information for identifying a speaker with which the voice section information is associated. Specifically, the speaker ID is an item that stores multiple user IDs and customer IDs that participated in the dialogue. The start date and time is an item that stores the start date and time of a voice section or a video section. The end date and time is an item that stores the end date and time of a voice section or a video section. The section audio data is an item that stores audio data included in a voice section. Reference information (path) for an audio data file located in another location may be stored. Furthermore, a reference to audio data for the period from the start date and time to the end date and time of the audio data in the dialogue table 1014 may be stored based on the start date and time and the end date and time. Furthermore, the section audio data may include audio data included in the section video data. The audio data format may be any data format such as AAC, ATRAC, MP3, MP4, etc., or may include multiple types of data formats. The section video data is an item that stores video data included in the audio section. It may also store reference information (path) to a video data file located in another location. Furthermore, a reference to video data for a period from the start date / time to the end date / time of the video data in the dialogue table 1014 may be stored based on the start date / time and the end date / time. The video data format may be any data format such as MP4, MOV, WMV, AVI, AVCHD, etc., or may include multiple types of data formats. The section reading text is an item that stores text information about the content spoken by a speaker in the section audio data included in the audio section. Specifically, the section reading text may be generated manually or by using a learning model such as any machine learning or deep learning based on the section audio data and section video data.

[0024] The comment table 1021 is a table for storing and managing information related to answers (answer information). The comment table 1021 is a table having columns for instruction statements, analysis data, input data, and comment data. FIG. 10 is a diagram showing the data structure of the comment table 1021.

[0025] The instruction sentence is an item that stores a character string related to an instruction sentence for generating input data. Specifically, the instruction sentence is input and edited in response to an input operation by the user, or an instruction sentence input by the user selecting a predetermined input candidate is stored. The analysis data is an item that stores information (analysis information) analyzed for dialogue information, speech interval information, etc. The analysis data specifically includes the following information: - Speech features related to speech uttered by a speaker Specifically, the speech features include the ratio of speech by the user to speech by the call destination (talk:listen ratio), the number of overlaps between the user's speech and the call destination's speech (number of overlaps), the number of silences (number of silences), the frequency of the user's speech or the call destination's speech (user's fundamental frequency, call destination's fundamental frequency), and the intonation of the user's speech or the call destination's speech (user's intonation strength, call destination's intonation strength). The analysis data includes the pitch (fundamental frequency), voice intensity (volume), spectral characteristics (including frequency domain characteristics of the speech, voiceprint, timbre, etc.), speech speed, duration of individual syllables, words, phrases, etc., voice rhythm, and voice quality (clear, hoarse, etc.) of the user and the call recipient. - Linguistic features related to the spoken content: Specifically, the linguistic features include the number of occurrences of specific keywords included in the conversation, their frequency, indicators of word diversity, the length of the utterance, indicators of the frequency of use of parts of speech such as nouns, verbs, and adjectives, the use of emotional words, and information on the distribution of topics. - Number of calls and call duration related to the conversation: The number of calls includes the number of calls made within a specific period (e.g., day, week, month). The call duration is an indicator of the duration of each call. The analysis data includes statistical values ​​such as the average, median, maximum, and minimum based on the analysis data including the above features for multiple conversations for each user or group. The analysis data includes comparison results such as rankings and rankings of the analysis data including the above-mentioned features for each user or group. Statistical values ​​and comparison results based on the analysis data may be calculated based on predetermined rules. The input data is an item that stores input data called prompts to be input to the generation AI 50.Comment data is an item that stores data (message documents) of comment messages that are mainly notified to users and are created based on answer data (response) obtained in response to inputting input data into generation AI 50.

[0026] <Configuration of control unit 104 of server 10> The control unit 104 of the server 10 includes a user registration control unit 1041 and a presentation unit 1042. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

[0027] The user registration control unit 1041 performs a process of storing information about users who wish to use the service disclosed herein in the user table 1012. Information stored in the user table 1012 is generated when a user opens a webpage operated by the service provider from an arbitrary information processing terminal, enters information into a predetermined input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service. Prior to the user registration control unit 1041 registering the user information in the user table 1012, the service provider may conduct a predetermined screening process to restrict whether the user can use the service. The user ID may be any string or number that can identify the user, any string or number desired by the user, or an arbitrary string or number may be automatically set by the user registration control unit 1041.

[0028] The presentation unit 1042 executes a presentation process, the details of which will be described later.

[0029] <Configuration of First User Terminal 20> The first user terminal 20 is an information processing device operated by a user who uses a service. The first user terminal 20 may be, for example, a desktop personal computer (PC) or a laptop PC, or may be a mobile terminal such as a smartphone or a tablet. It may also be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch-type terminal. The first user terminal 20 includes a storage unit 201, a control unit 204, an input device 206, and an output device 208.

[0030] <Configuration of Storage Unit 201 of First User Terminal 20> The storage unit 201 of the first user terminal 20 includes a first user ID 2011 and an application program 2012.

[0031] The first user ID 2011 stores the user identification information of the operator. The operator transmits the first user ID 2011 from the first user terminal 20 to the voice server (PBX) 60. The voice server (PBX) 60 identifies the operator based on the first user ID 2011 and provides the operator with the service according to the present disclosure. The first user ID 2011 includes information such as a session ID temporarily assigned by the voice server (PBX) 60 to identify the operator using the first user terminal 20.

[0032] The application program 2012 may be pre-stored in the storage unit 201, or may be downloaded from a web server operated by a service provider via a communication IF. The application program 2012 includes an application such as a web browser application. The application program 2012 includes an interpreter-type programming language such as JavaScript (registered trademark) that is executed on the web browser application stored in the first user terminal 20.

[0033] <Configuration of control unit 204 of first user terminal 20> The control unit 204 of the first user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 executes an application program 2012 stored in the storage unit 201, thereby realizing each functional unit.

[0034] <Configuration of Input Device 206 of First User Terminal 20 > The input device 206 of the first user terminal 20 includes a camera 2061 , a microphone 2062 , a position information sensor 2063 , a motion sensor 2064 , and a keyboard 2065 .

[0035] <Configuration of Output Device 208 of First User Terminal 20 > The output device 208 of the first user terminal 20 includes a display 2081 and a speaker 2082 .

[0036] <Configuration of Second User Terminal 30> The second user terminal 30 is an information processing device operated by a customer who uses a service. The second user terminal 30 may be, for example, a mobile terminal such as a smartphone or tablet, or may be a stationary personal computer (PC) or laptop PC. It may also be a wearable terminal such as a head-mounted display (HMD) or a wristwatch-type terminal. The second user terminal 30 includes a storage unit 301, a control unit 304, an input device 306, and an output device 308.

[0037] <Configuration of Storage Unit 301 of Second User Terminal 30 > The storage unit 301 of the second user terminal 30 includes an application program 3012 and a telephone number 3013 .

[0038] The application program 3012 may be pre-stored in the storage unit 301, or may be downloaded from a web server operated by a service provider via a communication IF. The application program 3012 includes an application such as a web browser application. The application program 3012 includes an interpreter-type programming language such as JavaScript (registered trademark) that is executed on the web browser application stored in the second user terminal 30.

[0039] <Configuration of the control unit 304 of the second user terminal 30> The control unit 304 of the second user terminal 30 includes an input control unit 3041 and an output control unit 3042. The control unit 304 executes an application program 3012 stored in the storage unit 301, thereby realizing each functional unit.

[0040] <Configuration of Input Device 306 of Second User Terminal 30 > The input device 306 of the second user terminal 30 includes a camera 3061 , a microphone 3062 , a position information sensor 3063 , a motion sensor 3064 , and a touch device 3065 .

[0041] <Configuration of Output Device 308 of Second User Terminal 30 > The output device 308 of the second user terminal 30 includes a display 3081 , a speaker 3082 , and a transmission unit 6041 .

[0042] The transmitting unit 6041 is a control unit that executes a process of transmitting evaluation data received from a user in the external server 60 to the server 10 .

[0043] <Configuration of Voice Server (PBX) 50> The voice server (PBX) 50 is an information processing device that functions as a switch that connects the network N and the telephone network T to each other, thereby enabling conversation between the first user terminal 20 and the second user terminal 30. The voice server (PBX) 50 includes a storage unit 501.

[0044] <Configuration of Storage Unit 501 of Voice Server (PBX) 50 > The storage unit 501 of the voice server (PBX) 50 includes an application program 5011 .

[0045] The application programs 5011 are programs for causing the control unit 504 of the voice server (PBX) 50 to function as each functional unit. The application programs 5011 include applications such as a web browser application.

[0046] <Configuration of Generation AI 80> The generation AI 80 is a type of artificial intelligence model (deep learning model) that outputs output data such as a character string or an image based on input data such as a character string or an image. In this disclosure, a large language model (LLM) that outputs output data related to a character string based on input data related to a character string will be mainly described as an example. Examples of LLMs include OpenAI ChatGPT, Microsoft Bing Chat, and Google Bard.

[0047] <Operation of System 1> The following describes each process of the system 1. Fig. 11 is a flowchart showing the comment processing operation. Fig. 12 is an example of a screen showing the comment processing operation.

[0048] <Dialogue Processing> The following describes processes that enable a first user and a second user to interact through an incoming call process in which a first user (operator) receives a call from a second user (customer) and an outgoing call process in which the first user (operator) makes an outgoing call to the second user (customer). Other methods for enabling a first user and a second user to interact are not limited to these. For example, a process in which a first user interacts with a second user in a virtual interaction space called a room will be described as room interaction processing. The invention disclosed herein is applicable to incoming call processing, outgoing call processing, and any method for enabling a first user and a second user to interact.

[0049] <Room Interaction Processing> One method involves creating a virtual interaction space called a room on the server 10 for interaction between a first user and a second user. The first and second users access the room via a web browser or application program stored in the first user terminal 20 and the second user terminal 30, respectively, to enable interaction. In this case, the voice server (PBX) 60 is not required. Specifically, the first user, who will be the host of the interaction, operates the input device 206 of the first user terminal 20 to send a request to the server 10 to host the interaction. Upon receiving the request, the control unit 104 of the server 10 issues room identification information such as a unique room ID and transmits a response to the first user terminal 20. The first user then transmits the received room identification information to the second user, with whom the interaction is intended, via any communication means, such as email or chat. The first user can enter the room by operating the input device 206 of the first user terminal 20, using a web browser or the like to access a URL that provides a room-related service on the server 10, and entering the room identification information. Similarly, a second user can enter a room by operating the input device 306 of the second user terminal 30, accessing a URL that provides a room-related service on the server 10 using a web browser or the like, and entering the room identification information. This allows the first user and the second user to converse via the first user terminal 20 and the second user terminal 30, respectively, in a virtual conversation space called a room associated with each other by the room identification information. By entering the room identification information, one or more other users can enter a single room in addition to the first user and the second user. This allows three or more users to converse via their respective user terminals in a virtual conversation space called a room associated with each other by the room identification information.

[0050] Furthermore, the interactive processing does not necessarily have to be performed by having all participants join the room. For example, in a conference held in a conference room or the like and attended by multiple participants, the interactive processing may be performed by having multiple participants enter the room via a single information terminal. Furthermore, the interactive processing does not necessarily have to be performed online, and may be performed using an information terminal that acquires video and audio of the contents of a conference held in a conference room or the like and attended by multiple participants. For example, the interactive processing may be performed in an application that facilitates the conference.

[0051] <Video Interaction> The system 1 according to the present disclosure may provide an online interaction service (video interaction service) that includes video data. For example, the control unit 204 of the first user terminal 20 and the control unit 304 of the second user terminal 30 transmit video data captured by the camera 2061 of the first user terminal 20 and the camera 3061 of the second user terminal 30, respectively, to the server 10. Based on the received video data, the server 10 transmits the video data captured by the camera 2061 of the first user terminal 20 to the second user terminal 30 and the video data captured by the camera 3061 of the second user terminal 30 to the first user terminal 20. The control unit 204 of the first user terminal 20 displays the received video data captured by the camera 3061 of the second user terminal 30 on the display 2081. The control unit 304 of the second user terminal 30 displays the received video data captured by the camera 2061 of the first user terminal 20 on the display 3081. The server 10 may transmit video data of some or all of the multiple users participating in the online dialogue to the first user terminal 20 and the second user terminal 30. In this case, the control unit 204 of the first user terminal 20 displays the received video data of some or all of the multiple users participating in the online dialogue on a single screen on the display 2081 of the first user terminal 20. This allows the dialogue status of the multiple users participating in the online dialogue to be confirmed. A similar process may also be performed in the second user terminal 30.

[0052] In the outgoing call processing and room interaction processing, when an interaction between the user and the customer is initiated, an interaction storage process is executed in the same manner as in the incoming call processing. The interaction storage process is the same as step S104 in the incoming call processing, and therefore will not be described here.

[0053] The room interaction process may be performed by an online conference service or the like operated by a business operator different from the information processing service according to the present disclosure. Examples of online conference services include Zoom, Google Meet, Microsoft Teams, and the like.

[0054] <Incoming call processing> Incoming call processing is processing in which the user receives an incoming call (call) from a customer.

[0055] <Overview of Incoming Call Processing> Incoming call processing is a series of processes that, when a customer calls the user while the user has an application running on the first user terminal 20, identify a call handling rule to be applied to the customer, execute an incoming call determination process based on the identified call handling rule, and execute a process to connect to the user based on the determination result. Note that, in the present disclosure, an incoming call processing by telephone is described as an example, but the present disclosure is also applicable to incoming call processing using any online interactive service, etc.

[0056] <Details of Incoming Call Processing> An incoming call processing of the system 1 when the user receives an incoming call (call) from a customer will be described.

[0057] When the user receives a call from a customer, the following process is performed in the system 1.

[0058] In step S101, the user operates the first user terminal 20 to launch a web browser and access the website of the CRM service provided by the CRM system 50. At this time, the user is assumed to be logged in to the CRM system 50 using his or her own account in the web browser and is on standby. Note that the user only needs to be logged in to the CRM system 50, and may be performing other tasks related to the CRM service.

[0059] In step S102, the customer operates the second user terminal 30, inputs a predetermined telephone number assigned to the voice server (PBX) 60, and makes a call to the voice server (PBX) 60. The voice server (PBX) 60 receives the call from the second user terminal 30 as an incoming call event. The voice server (PBX) 60 transmits the incoming call event to the server 10. Specifically, the voice server (PBX) 60 transmits an incoming call request including the customer's telephone number 3011 to the server 10.

[0060] In step S103, the first user terminal 20 accepts a response operation by the user. The response operation is realized, for example, by lifting the receiver (not shown) of the first user terminal 20, or by the user operating the mouse 2066 to press a button labeled "Answer Call" on the display 2081 of the first user terminal 20. Upon accepting the response operation, the first user terminal 20 transmits a response request to the voice server (PBX) 60 via the CRM system 50 and the server 10. The voice server (PBX) 60 receives the transmitted response request and establishes voice communication. This enables the first user terminal 20 to interact with the second user terminal 30. The display 2081 of the first user terminal 20 displays information indicating that an interaction is taking place. For example, the display 2081 of the first user terminal 20 may display the words "Interactive."

[0061] <Dialogue Storage Process> Dialogue storage process is executed in step S104. The dialogue storage process is a process for storing data relating to a dialogue between a user and a customer.

[0062] <Outline of Dialogue Storage Processing> The dialogue storage processing is a series of processes for storing data relating to a dialogue in the dialogue table 1014 when a dialogue is started between a user and a customer.

[0063] <Details of Dialogue Storage Process> In step S104, the control unit 104 of the server 10 executes a voice acquisition step of acquiring voice data related to the dialogue. Specifically, when a dialogue between the user and the customer begins, the voice server (PBX) 60 records the voice data related to the dialogue between the user and the customer and transmits it to the server 10. Upon receiving the voice data, the control unit 104 of the server 10 creates a new record in the dialogue table 1014 and stores the data related to the dialogue between the user and the customer. Specifically, the control unit 104 of the server 10 stores the user ID, customer ID, dialogue category, call reception / transmission type, and the content of the voice data in the new record in the dialogue table 1014.

[0064] The control unit 104 of the server 10 acquires the first user ID 2011 of the user from the first user terminal 20 during outgoing call processing or incoming call processing, and stores it in the user ID field of a new record in the dialogue table 1014. The control unit 104 of the server 10 queries the CRM system 50 based on the telephone number during outgoing call processing or incoming call processing. The CRM system 50 acquires the customer ID by searching the customer table 5012 by telephone number, and sends it to the server 10. The control unit 104 of the server 10 stores the acquired customer ID in the customer ID field of the new record in the dialogue table 1014. The control unit 104 of the server 10 stores the value of the dialogue category set in advance for each user or customer in the dialogue category field of the new record in the dialogue table 1014. Note that the dialogue category may be stored by the user selecting and inputting a value for each dialogue. The control unit 104 of the server 10 identifies whether the ongoing conversation was initiated by the user or the customer, and stores either the value of outbound (initiated by the user) or inbound (initiated by the customer) in the incoming / outgoing call type field of the new record in the conversation table 1014.

[0065] The control unit 104 of the server 10 stores the voice data received from the voice server (PBX) 60 in the voice data field of a new record in the dialogue table 1014. Note that the voice data may be stored as a voice data file in another location, and reference information (path) for the voice data file may be stored after the dialogue ends. The control unit 104 of the server 10 may also be configured to store the voice data after the dialogue ends.

[0066] Furthermore, in the video dialogue service, the control unit 104 of the server 10 stores video data received from the first user terminal 20 and the second user terminal 30 in the video data field of a new record in the dialogue table 1014. Note that the video data may be stored as a video data file in another location, and reference information (path) for the video data file may be stored after the dialogue ends. The control unit 104 of the server 10 may also be configured to store the video data after the dialogue ends.

[0067] The control unit 104 of the server 10 executes a voice extraction step of extracting multiple pieces of section voice data for each speech section from the voice data acquired in the voice acquisition step. The voice extraction step includes a step of identifying a speaker for each of the multiple pieces of section voice data. Specifically, the control unit 104 of the server 10 acquires (accepts) the dialogue ID, voice data, and video data stored in the dialogue table 1014. The control unit 104 of the server 10 detects sections (voice sections) in which speech is continuously present from the acquired (accepted) voice data and video data, and extracts the voice data and video data for each speech section as section voice data and section video data, respectively. For example, the section voice data and section video data may be extracted by dividing the voice data and video data by silent sections in which no speech is present. Alternatively, the section voice data and section video data may be extracted by dividing the voice data and video data by sentence units, such as segments, sentences, and paragraphs, for the content of the speech contained in the voice data and video data. The section voice data and section video data are associated with the speaker's user ID, the start date and time of the speech section, and the end date and time of the speech section for each speech section.

[0068] The control unit 104 of the server 10 executes a text generation step of generating a plurality of section reading texts, which are text information of the content spoken by the speaker, for each of the plurality of section audio data extracted in the audio extraction step. Specifically, the control unit 104 of the server 10 performs text recognition on the speech content of the extracted section audio data and section video data, thereby converting the section audio data and section video data into section reading texts, which are characters (text), and transcribing them. Note that the specific method of text recognition is not particularly limited. For example, conversion may be performed using signal processing technology, machine learning using AI (artificial intelligence), deep learning, or the like.

[0069] The control unit 104 of the server 10 stores the dialogue ID to be processed, the speaker's user ID (first user ID 2011 or second user ID 3011), start date and time, end date and time, section audio data, section video data, and section reading text in the dialogue ID, speaker ID, start date and time, end date and time, section audio data, section video data, and section reading text fields of a new record in the audio section table 1016, respectively.

[0070] The voice segment table 1016 stores the section reading text for each speech segment of the voice data as continuous time-series data, associated with the start date and time and the speaker. By checking the section reading text stored in the voice segment table 1016, the user can check the dialogue content as text information without checking the content of the voice data.

[0071] In addition, during the text recognition process, information that is meaningless in understanding the conversation between the user and the customer, such as fillers contained in the text, may be excluded from the text in advance, and the speech recognition information may be stored in the speech segment table 1016.

[0072] <Call Processing> The call processing is processing in which a user (first user) calls (places a call) to a customer (second user).

[0073] <Outline of Call Processing> The call processing is a series of processes in which the user selects a customer to whom they wish to make a call from among multiple customers displayed on the screen of the first user terminal 20 and performs a call operation to make a call to the customer. In the present disclosure, a case in which a second user is selected as a customer will be described as an example.

[0074] <Details of Call Processing> The call processing of the system 1 when a user calls a customer will be described.

[0075] When a user makes a call to a customer, the following process is executed in the system 1.

[0076] The user operates the first user terminal 20 to launch a web browser and access the website of the CRM service provided by the CRM system 50. The user can display a list of their own customers on the display 2081 of the first user terminal 20 by opening a customer management screen provided by the CRM service. Specifically, the first user terminal 20 sends the CRM ID 2013 and a request to display a list of customers to the CRM system 50. Upon receiving the request, the CRM system 50 searches the customer table 5012 and sends information about the user's customers, such as the customer ID, name, telephone number, customer attributes, customer organization name, and customer organization attributes, to the first user terminal 20. The first user terminal 20 displays the received information about the customers on the display 2081 of the first user terminal 20.

[0077] The user presses and selects a customer (second user) to whom they wish to make a call from the list of customers displayed on the display 2081 of the first user terminal 20. With the customer selected, the user presses the "Call" button or the phone number button displayed on the display 2081 of the first user terminal 20 to send a request including the phone number to the CRM system 50. The CRM system 50, which receives the request, sends the request including the phone number to the server 10. The server 10, which receives the request, sends a call request to the voice server (PBX) 60. When the voice server (PBX) 60 receives the call request, it makes a call (call) to the second user terminal 30 based on the received phone number.

[0078] In response to this, the first user terminal 20 controls the speaker 2082 or the like to make a sound indicating that an outgoing call (call) is being made by the voice server (PBX) 60. The display 2081 of the first user terminal 20 also displays information indicating that an outgoing call (call) is being made to the customer by the voice server (PBX) 60. For example, the display 2081 of the first user terminal 20 may display the words "Calling."

[0079] The customer lifts the receiver (not shown) of the second user terminal 30 or presses an "answer" button or the like that is displayed on the input device 306 of the second user terminal 30 when a call arrives, thereby placing the second user terminal 30 in a conversation-enabled state. Accordingly, the voice server (PBX) 60 transmits information indicating that the second user terminal 30 has responded (hereinafter referred to as a "response event") to the first user terminal 20 via the server 10, the CRM system 50, or the like. This allows the user and the customer to engage in a conversation using the first user terminal 20 and the second user terminal 30, respectively, and allows a conversation between the user and the customer. Specifically, the user's voice collected by the microphone 2062 of the first user terminal 20 is output from the speaker 3082 of the second user terminal 30. Similarly, the customer's voice collected by the microphone 3062 of the second user terminal 30 is output from the speaker 2082 of the first user terminal 20.

[0080] When the display 2081 of the first user terminal 20 becomes ready for interaction, the display 2081 receives the response event and displays information indicating that an interaction is taking place. For example, the display 2081 of the first user terminal 20 may display the words "Responding."

[0081] <Presentation Process> The presentation process is a process of generating dialogue summary information summarizing the characteristics of a user's dialogue response and speech features including advice for improving the dialogue response, based on past dialogue information of the user or a group to which the user belongs. Users such as operators can use the content of the comment information to improve their own dialogue response by checking it. Furthermore, users in a position to manage a group consisting of multiple operators, such as managers, can use the content of the comment information to improve the dialogue response of the group they manage.

[0082] <Outline of presentation process> The presentation process is a series of processes that identify the target user of the presentation process, acquire the dialogue information of the user, create analysis data based on the dialogue information, create input data based on the analysis data, create comment information based on the answer results obtained by sending the input data to the generation AI, and present the created comment information.

[0083] <Details of the Presentation Process> The following describes the details of the presentation process. In the present disclosure, a configuration in which a first user executes the presentation process is disclosed as an example; however, the presentation process may be executable by any user. Furthermore, the presentation process may be executable only by a user engaged in management tasks, such as an administrator. The execution authority of the presentation process may be set to any user, or the executable processes may be switched depending on the execution authority of each user. In addition, in the present disclosure, a configuration in which the presentation process is executed based on an operation by a first user is disclosed as an example; however, this is not limited to this. For example, in step S101 described below, a user ID to be the target of the presentation process may be identified, and the identified target users and target groups may be associated with and stored in advance with users to whom comment information is to be distributed. In this case, the presentation unit 1042 of the server 10 may be configured to periodically (daily, weekly, monthly) execute the presentation process and periodically distribute comment messages based on comment information related to the target users and target groups to the users to whom the comment information is to be distributed. When presenting comment information to a user, the target period, target range, etc. may be specified.

[0084] In step S101, the presentation unit 1042 of the server 10 identifies a user ID (target user ID) that is to be the target of the presentation process. The first user operates the input device 206 of the first user terminal 20 to input the URL of a page for executing the presentation process (presentation process page) into a web browser or the like, and open the presentation process page. The control unit 204 of the first user terminal 20 sends a request to open the presentation process page to the server 10. Based on the received request, the control unit 104 of the server 10 generates a presentation process page and sends it to the first user terminal 20. The control unit 204 of the first user terminal 20 displays the received presentation process page on the display 2081 of the first user terminal 20.

[0085] The first user operates the input device 206 of the first user terminal 20 to input the user ID, user name, etc. of the user who is the target of the presentation process into an input field for inputting a target user ID included in the presentation process page. Note that the presentation process page may display a list of user identification information such as user IDs and user names stored in the user table 1012 to the first user, and accept input of the target user ID in response to a selection operation on the listed user identification information. The control unit 204 of the first user terminal 20 transmits the input target user ID to the server 10. The presentation unit 1042 of the server 10 receives and identifies the target user ID.

[0086] The presentation processing page may also be capable of accepting input of multiple user IDs, etc. For example, the presentation processing page displays a list of group identification information, such as group IDs and group names, stored in the group table 1013 to the first user, and accepts input of a group ID (group ID of a target group) in response to a selection operation on the displayed group identification information. The control unit 204 of the first user terminal 20 transmits the input group ID to the server 10. The presentation unit 1042 of the server 10 searches the group ID field in the user table 1012 based on the received group ID, and identifies the user IDs of one or more users belonging to the selected group.

[0087] In step S102, the control unit 104 of the server 10 acquires dialogue information based on one or more target user IDs (hereinafter referred to as target user IDs) identified in step S101. Specifically, the presentation unit 1042 of the server 10 searches the user ID item in the dialogue table 1014 based on the identified target user ID, and acquires one or more pieces of dialogue information. Specifically, the dialogue information includes a dialogue ID, a user ID, a customer ID, a dialogue category, a sending / receiving type, audio data, and video data. The presentation unit 1042 of the server 10 searches the dialogue ID item in the label table 1015 based on the dialogue ID included in the acquired dialogue information, and acquires one or more pieces of label information. The presentation unit 1042 of the server 10 searches the dialogue ID item in the speech segment table 1016 based on the dialogue ID included in the acquired dialogue information, and acquires one or more pieces of voice segment information. The voice segment information includes a segment ID, a dialogue ID, a speaker ID, a start date and time, an end date and time, section audio data, section video data, and section readout text. The dialogue information in the present disclosure may include label information and speech section information associated with predetermined dialogue information based on a dialogue ID, as well as information about any dialogue.

[0088] In step S103, the presentation unit 1042 of the server 10 executes an analysis data acquisition step of acquiring analysis data obtained by analyzing the dialogue. Specifically, the presentation unit 1042 of the server 10 analyzes the voice data, video data, etc. included in the dialogue information acquired in step S102, and the section voice data, section video data, etc. included in the voice section information, to create analysis data including the following voice features and language features. In addition, the presentation unit 1042 of the server 10 analyzes the number of records, voice data, video data, etc. of the dialogue information to create analysis data including indicators related to the dialogue, such as the number of calls related to the dialogue and the call duration. Note that the creation of analysis data is not limited to this step, and analysis data created in advance may also be included in this step.

[0089] Speech features related to speech uttered by speakers Speech features include the ratio of speech by the operator to speech by the customer (talk:listen ratio), the number of overlaps between speech by the operator and speech by the customer (number of overlaps), the number of silences (number of silences), the frequency of speech by the operator or the customer (fundamental frequency of the operator, fundamental frequency of the customer), intonation of speech by the operator or the customer (strength of intonation of the operator, strength of intonation of the customer), etc. Speech features include pitch (fundamental frequency), voice intensity (volume), spectral characteristics (including frequency domain characteristics of speech, voiceprint, timbre, etc.), speech speed of speech, length of individual syllables, words, phrases, etc., speech rhythm, voice quality (clear voice, hoarse voice, etc.) of speech of the operator and the customer. Speech features include score information (speech score) indicating the quality of the speech features calculated based on the above speech features.

[0090] Linguistic features related to spoken content Linguistic features include the number of times certain keywords appear in the dialogue, their frequency of appearance, an index related to word diversity, the length of the utterance, an index showing the frequency of use of parts of speech such as nouns, verbs, and adjectives, the use of emotional words, and information on the distribution of topics. Linguistic features include score information (language score) that indicates the quality of the linguistic features calculated based on the above linguistic features.

[0091] Dialogue-related indicators such as the number of calls and call duration The number of calls includes the number of calls made during a specific period (day, week, month, etc.). Call duration is an indicator that indicates how long each call lasted. Dialogue-related indicators include score information (indicator score) that indicates the quality of the dialogue-related indicators calculated based on the dialogue-related indicators.

[0092] In addition, the score may include dialogue score information (dialogue score) that comprehensively indicates the quality of the dialogue response obtained by combining the voice score, language score, and index score.

[0093] The analysis data may be statistical values ​​such as the average, median, maximum, and minimum values ​​of analysis data (hereinafter referred to as features, etc.) including speech features, language features, and indices in multiple conversations for each user or group. Specifically, if multiple users are identified in step S101, statistical values ​​of indices related to speech features, language features, and conversations for the multiple users may be used as the analysis data.

[0094] The analysis data includes comparison results such as rankings and rankings of analysis data including features, etc., for each user or group. Specifically, if user A is ranked first in speech score, second in language score, fourth in index score, and second in dialogue score, etc., the comparison result for user A is expressed as (1, 2, 4, 2). In this case, the comparison results for user B, user C, and user D can be expressed as (2, 1, 3, 4), (4, 3, 1, 2), and (3, 4, 2, 1), respectively. The comparison result thus includes information comparing the quality of analysis data between multiple users. The analysis data may also include comparisons over a predetermined period. For example, monthly comparison values ​​such as average values ​​can be included to serve as indicators of the degree of improvement, etc.

[0095] The analysis data may include information on the texts read aloud in multiple dialogues for each user or group. Specifically, the texts read aloud in the speech segment table 1016 may be referenced, and the texts read aloud in the dialogues may be included in the analysis data.

[0096] In step S103, the analysis data acquisition step executes a step of acquiring analysis data obtained by analyzing the dialogue performed by a predetermined operator. Specifically, when the user ID of the user associated with the predetermined operator is identified as the target user ID in step S101, the presentation unit 1042 of the server 10 creates and acquires analysis data of the predetermined operator.

[0097] In step S103, the analysis data acquisition step executes a step of acquiring analysis data related to each of the multiple operators obtained by analyzing the multiple conversations performed by the multiple operators. Specifically, when the user IDs of the users related to the multiple operators are identified as the target user IDs in step S101, the presentation unit 1042 of the server 10 creates and acquires analysis data for the multiple operators.

[0098] In step S103, the analysis data acquisition step executes a step of acquiring analysis data for a predetermined period. Specifically, the presentation unit 1042 of the server 10 may create and acquire analysis data based on dialogue information within a predetermined period from the date and time when the presentation process is executed or an arbitrary date and time, excluding dialogue information outside the predetermined period. For example, the presentation unit 1042 of the server 10 may create and acquire analysis data based on dialogue information within the most recent month. This is because providing comment information based on recent dialogue information is considered to be beneficial for improving users' dialogue responses.

[0099] The presentation unit 1042 of the server 10 stores the created analysis data in the analysis data field of the new record (target record) in the comment table 1021 .

[0100] A character string related to a command statement for generating input data, which will be described later, is stored in the command statement field of the target record of the comment table 1021. An example of the command statement is shown below.・"Based on the analysis data, explain the characteristics of the target user's dialogue responses." ・"Based on the analysis data, explain the changing characteristics of the target user's dialogue responses." ・"Based on the analysis data, explain the target user's goal achievement status for dialogue responses." ・"Based on the analysis data, explain areas for improvement in the target user's dialogue responses." ・"Based on the analysis data, suggest other users who can serve as references for the target user's dialogue responses." ・"Based on the analysis data, explain the characteristics of the target group's dialogue responses." ・"Based on the analysis data, explain areas for improvement in the target group's dialogue responses." ・"Based on the analysis data, identify the top users (users with good scores), improvers (users whose scores have improved), bottom users (users whose scores have worsened) among the users in the target group." ・"Based on the analysis data, output areas for improvement, items that have changed, goal achievement possibility, and comparison results."

[0101] The directives in the present disclosure include directives for causing the generation AI 80 to output analysis results for the analysis data. The directives include directives in a format called a zero-shot prompt, which directly and explicitly instruct the generation AI 80 on the task to be executed. The directives also include directives in a format called a short-shot prompt, which instruct the generation AI 80 on the task to be executed using a small number of input / output examples. For example, in a directive in a format called a short-shot prompt, the directive includes input / output examples consisting of pairs of input data and output data, where the output data is "text indicating the analysis results, analysis content, etc." for the analysis data, which is the input data,

[0102] Furthermore, the following instruction may be stored for the analysis data of multiple organizations, groups, etc. in a specific company: "Based on the analysis data, please identify the top group (groups with excellent scores), improvement group (groups with improved scores), bottom group (groups with poor scores), and deterioration group (groups with worsening scores)."

[0103] In addition, the instructions for generating input data may include instructions that support suggestions for good parts of speaking style and parts to be improved in the dialogue, based on the spoken text (analysis data) related to the dialogue. - "Please explain the good parts and parts to be improved in the dialogue response of the target user based on the spoken text." - "Please explain the good parts and parts to be improved in the dialogue response of the target group based on the spoken text."

[0104] The instruction statement may be configured such that one predetermined instruction statement is preset as a default value and stored. The instruction statement may be configured such that a predetermined instruction statement is selected from a plurality of instruction statements and stored. For example, the presentation processing page in step S101 may be configured to accept input of an instruction statement. Specifically, the presentation processing page may be configured to present a plurality of instruction statements to the user, and the predetermined instruction statement selected by the user's input operation may be stored. For example, the user may select a predetermined instruction statement depending on the content of the comment they wish to obtain in the presentation processing. Note that the instruction statement field of the target record in the comment table 1021 may be configured to store character strings relating to a plurality of instruction statements. As a result, the instruction statement and the analysis data are stored in association with each other in the target record in the comment table 1021.

[0105] In step S104, the presentation unit 1042 of the server 10 executes an input data creation step of creating input data to be input to the generation AI based on the analysis data acquired in the analysis data acquisition step. The input data creation step executes a step of creating input data based on at least one of the following: an instruction statement for outputting improvements in the dialogue based on the analysis data; an instruction statement for outputting items that have changed in the dialogue based on the analysis data; an instruction statement for outputting the goal achievement status of an operator or a group to which multiple operators belong based on the analysis data; and an instruction statement for outputting comparison results for multiple operators or multiple groups based on the analysis data. Specifically, the presentation unit 1042 of the server 10 creates input data to be input to the generation AI, called a prompt, based on the instruction statement and analysis data stored in the comment table 1021.

[0106] An example of input data is shown below: [Input data] Please explain the characteristics of target user A's dialogue response based on the analysis data. # Analysis data: Dialogue score: 70 points ・Audio features: Audio score: 60 points Talk:Listen ratio: 0.6 (Ratio of the time the user speaks to the time the listener speaks) Number of overlaps: 10 (Number of times the user and listener's speech overlapped) Number of silences: 15 (Number of times silence occurred during the conversation) Fundamental frequency: 110 (Fundamental frequency of the user's speech) Intonation strength: 0.5 (Strength of the user's speech intonation) ・Language features: Language score: 30 points Number of times a keyword appears: 20 (Number of times a specific keyword appears during the conversation) Word diversity: 0.75 (Indicator of the diversity of words used) Sentence length: 50 (Average length of a user's utterance) Frequency of noun use: 0.3 (Frequency of noun use) Frequency of verb use: 0.2 (Frequency of verb use) Frequency of adjective use: 0.1 (Frequency of adjective use) Use of emotional words: 5 (Number of times words expressing emotions were used) Topic distribution: {Topic A: 0.4, Topic B: 0.3, Topic C: 0.3} (Proportion of speech for each topic) ・Dialogue-related indicators: (Group average) Index score: 80 points Number of calls: 100 (Number of calls in a specific period (e.g., one week)) Call duration: 300 minutes (Total call duration within the same period) # Output result:

[0107] In step S104, the input data creation step executes a step of creating input data based on at least one of information indicating one or more operators or one or more groups whose dialogue is judged to be excellent based on a score for judging the quality of the dialogue for each operator or group to which multiple operators belong, and information indicating one or more operators or one or more groups whose dialogue is judged to be poor based on a score for judging the quality of the dialogue for each operator or group to which multiple operators belong.

[0108] An example of input data is shown below. The input data may also include analytical data for each user included in the group (users A to C). [Input Data] Based on the analytical data, please compare and explain the characteristics of each user included in target group A. # Target group A: Consists of user A, user B, user C, and user D # Analysis data: ・Comparison results (ranking information) Comparison results for user A: (voice score: 1st, language score: 2nd, index score: 4th, dialogue score: 2nd) Comparison results for user B: (voice score: 2nd, language score: 1st, index score: 3rd, dialogue score: 4th) Comparison results for user C: (voice score: 4th, language score: 3rd, index score: 1st, dialogue score: 2nd) Comparison results for user D: (voice score: 3rd, language score: 4th, index score: 2nd, dialogue score: 1st) Dialogue score: 70 points (group average) ・Voice features: (group average) Voice score: 60 points Talk:Listen ratio: 0.6 (ratio of the time the user speaks to the time the listener speaks) Number of overlaps: 10 (number of times the user and listener's speech overlapped) Number of silences: 15 (number of times silence occurred during a conversation) Fundamental frequency: 110 (fundamental frequency of user's speech) Intonation strength: 0.5 (strength of user's speech intonation) - Linguistic features: (group average) Language score: 30 points Number of times a keyword appears: 20 (number of times a specific keyword appears in the conversation) Word diversity: 0.75 (index showing the diversity of words used) Length of speech: 50 (average length of user's speech) Frequency of noun use: 0.3 (frequency of noun use) Frequency of verb use: 0.2 (frequency of verb use) Frequency of adjective use: 0.1 (frequency of adjective use) Use of emotional words: 5 (number of times words expressing emotions are used) Topic distribution: {Topic A: 0.4, Topic B: 0.3, Topic C: 0.3} (proportion of utterances for each topic) - Dialogue-related indicators: (group average) Index score: 80 points Number of calls: 100 (Number of calls in a specific period (e.g., one week)) Call duration: 300 minutes (Total call duration in the same period)

[0109] The presentation unit 1042 of the server 10 stores the created input data in the input data field of the target record in the comment table 1021.

[0110] In step S105, the presentation unit 1042 of the server 10 executes an answer receiving step of receiving the answer content obtained by sending the input data created in the input data creation step to the generation AI. Specifically, the presentation unit 1042 of the server 10 sends the input data created in step S104 to the generation AI 80 as input data (prompt). The generation AI 80 outputs answer data to the server 10 as a response to the input data. The presentation unit 1042 of the server 10 receives and accepts the answer data for the input data.

[0111] In step S106, the presentation unit 1042 of the server 10 executes a comment presentation step of presenting a comment message including the answer content received in the answer receiving step to a predetermined operator. Specifically, the presentation unit 1042 of the server 10 creates comment data based on the answer content received in step S105. The presentation unit 1042 of the server 10 creates comment data by combining at least one of the answer content, the target user, information identifying each user belonging to the target group, and the analysis period. Note that the answer content itself may be used as comment data. Note that in the processing of this flowchart, each step may be repeatedly executed to obtain comment data.

[0112] An example of comment data is shown below. [Comment data] The characteristics of the dialogue response for user A (name, affiliation, etc.) during the period (Y-M-D to Y-M-D) are as follows. # Characteristics of dialogue response: (Response content from generation AI 80)

[0113] An example of comment data is shown below. [Comment data] The characteristics of each user during the period of Group A (Y-M-D to Y-M-D) are as follows: User A (name, affiliation, etc.) User B (name, affiliation, etc.) User C (name, affiliation, etc.) User D (name, affiliation, etc.) # Characteristics of each user: (Answer content from generation AI 80)

[0114] An example of comment data is shown below. [Comment data] The good points and areas for improvement in the speaking style of User A (name, affiliation, etc.) during the period (Y-M-D to Y-M-D) are as follows: # Good points and areas for improvement in speaking style (Answer content from generation AI 80)

[0115] The presentation unit 1042 of the server 10 stores the created comment data in the comment data item of the target record in the comment table 1021.

[0116] In step S106, the presentation unit 1042 of the server 10 executes a comment presentation step of presenting a comment message including the answer content received in the answer reception step to a predetermined user. FIG. 12 is an example of a comment screen D1 illustrating the comment processing operation. The comment screen D1 includes comment information D11 and analysis data D12. The comment information includes an instruction statement D111 and an answer content D112 from the generation AI 80. The analysis data D12 includes content in which the speech features, language features, and dialogue-related index data included in the analysis data described above are visually visualized using graphs or the like. Specifically, the presentation unit 1042 of the server 10 transmits the created comment information to the first user terminal 20. For example, the presentation unit 1042 of the server 10 may transmit a message (comment message) including the comment information to the first user's email address, chat account, etc. The display 2081 of the first user terminal 20 presents the received comment message to the first user. The control unit 204 of the first user terminal 20 displays the comment data in the comment information D11 on the comment screen D1. The control unit 204 of the first user terminal 20 displays the response content from the generation AI 80 in the response content D112 on the comment screen D1. The control unit 204 of the first user terminal 20 may display the instruction text in the instruction text D111 on the comment screen D1. In addition, the control unit 204 of the first user terminal 20 may display the analysis data created in step S103 in the analysis data D12 on the comment screen D1.

[0117] In step S106, the comment presentation step executes a step of presenting a comment message at predetermined intervals. Specifically, the present disclosure discloses, as an example, a configuration in which the presentation process is executed based on an operation by the first user, but is not limited to this. The presentation unit 1042 of the server 10 may be configured to periodically (daily, weekly, monthly) execute the presentation process and periodically distribute comment messages based on comment information regarding target users and target groups to predetermined users, such as an administrator engaged in management work that manages multiple operators.

[0118] In step S106, a comment presentation step is executed to present a comment message including the analysis data acquired in the analysis data acquisition step and the answer content received in the answer reception step. Specifically, the presentation unit 1042 of the server 10 may include the analysis data created in step S103 in the comment information. The presentation unit 1042 of the server 10 transmits the comment message including the analysis data to the first user terminal 20. The control unit 204 of the first user terminal 20 displays the comment information together with the analysis data in the analysis data D12 on the comment screen D1. This allows the first user to check the content of the analysis data that is the source of the comment information along with the comment information. The first user can easily and more deeply understand the content of the analysis data by referring to the content of the comment message.

[0119] 13 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0120] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0121] The main storage device 902 is used to temporarily store programs and data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0122] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0123] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard. The network is composed of the Internet, a LAN, various mobile communication systems constructed using wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a network that is directly connected using a USB (Universal Serial Bus) cable, etc.

[0124] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the concept of the computer 90 includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0125] <Basic Functional Configuration of Computer 90> A description will be given of the functional configuration of the computer realized by the basic hardware configuration (FIG. 13) of the computer 90. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

[0126] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0127] The control unit is realized by the processor 901 reading various programs stored in the auxiliary storage device 903, loading them into the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0128] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. The processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. The control unit can cause the processor 901 to add, update, and delete data stored in the storage unit in accordance with the various programs.

[0129] The term "database" refers to a relational database, which manages data sets called tables and masters in a tabular format structurally defined by rows and columns, by associating them with each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be established and associated. Typically, each table and each master is assigned a column that serves as a primary key to uniquely identify a record, but assigning a primary key to a column is not required. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, the information processing device and information processing system according to the present disclosure can be considered to have been manufactured.

[0130] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0131] The communication unit is realized by the communication IF 991. The communication unit realizes the function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. Furthermore, the communication unit can transmit information output from the control unit to other computers 90.

[0132] <Additional Notes> The matters described in the above embodiments will be added below.

[0133] (Supplementary Note 1) A program that causes a computer having a processor and a storage unit to process information related to a dialogue between multiple users, the program executing an analysis data acquisition step (S103) in which the processor acquires analysis data obtained by analyzing the dialogue, and an input data creation step (S104) in which input data to be input to a generation AI based on the analysis data acquired in the analysis data acquisition step. This makes it possible to create input data such as prompts to be input to a generation AI such as a large-scale language model to obtain answer contents (comments) in a manner that is easy for users to understand, from the analysis data related to the dialogue between multiple users.

[0134] (Supplementary Note 2) The program according to Supplementary Note 1, wherein the analysis data includes information about a predetermined dialogue, such as speech features related to the speech uttered by the speaker, language features related to the content of the utterance, and at least one of the number of calls and call duration related to the dialogue. This makes it possible to create input data such as prompts to be input to a generation AI for a large-scale language model or the like to obtain answer content (comments) in a manner that is easy for users to understand, from numerical data such as speech features, language features, number of calls, and call information related to a dialogue between multiple users.

[0135] (Supplementary Note 3) The program according to Supplementary Note 1 or 2, wherein the analysis data includes statistical values ​​of feature quantities in multiple conversations between multiple users who have had conversations, or comparison results of feature quantities between multiple users who have had conversations. This makes it possible to evaluate the conversations of a user or a group to which multiple users belong, based on statistical values ​​such as the average value or median of feature quantities for each user, or comparison results such as rankings comparing feature quantities between users.

[0136] (Supplementary Note 4) The program according to any one of Supplementary Notes 1 to 3, wherein the input data creation step (S104) is a step of creating input data based on at least one of an instruction statement for outputting improvements in the dialogue based on the analysis data, an instruction statement for outputting items that have changed in the dialogue based on the analysis data, an instruction statement for outputting a goal achievement status of an operator or a group to which multiple operators belong based on the analysis data, and an instruction statement for outputting comparison results for multiple operators or multiple groups based on the analysis data. This makes it possible, in the case where the user is an operator or the like who handles customers through dialogue, to create input data such as prompts to be input to a generation AI such as a large-scale language model to obtain response content (comments) suitable for the operator to improve the dialogue content based on the analysis data related to the dialogue.

[0137] (Supplementary Note 5) The program according to any one of Supplementary Notes 1 to 4, wherein the input data creation step (S104) is a step of creating input data based on at least one of information indicating one or more operators or one or more groups whose dialogue is determined to be excellent based on a score for determining the quality of the dialogue for each operator or group to which the multiple operators belong, and information indicating one or more operators or one or more groups whose dialogue is determined to be poor based on a score for determining the quality of the dialogue for each operator or group to which the multiple operators belong. This makes it possible to create input data such as prompts to be input into a generation AI for a large-scale language model or the like to obtain, as answer content (comments), an operator whose dialogue is excellent, a group to which the operator belongs, or an operator whose dialogue is poor, a group to which the operator belongs, etc., in the case where a user is an operator or the like who handles customers through dialogue.

[0138] (Supplementary Note 6) The program according to any one of Supplementary Notes 1 to 5, wherein the analysis data acquisition step (S103) is a step of acquiring analysis data obtained by analyzing a dialogue performed by a predetermined operator, and the processor executes an answer reception step (S105) of receiving an answer content obtained by sending the input data created in the input data creation step to the generation AI, and a comment presentation step (S106) of presenting a comment message including the answer content received in the answer reception step to the predetermined operator. This allows a user such as an operator to obtain an answer content (comment) regarding the dialogue from the generation AI in a manner that is easy for the user to understand.

[0139] (Supplementary Note 7) The program according to any one of Supplementary Notes 1 to 5, wherein the analysis data acquisition step (S103) is a step of acquiring analysis data for each of a plurality of operators obtained by analyzing a plurality of conversations conducted by a plurality of operators, and the processor executes an answer receiving step (S105) of receiving an answer content obtained by sending the input data created in the input data creation step to the generation AI, and a comment presenting step (S106) of presenting a comment message including the answer content received in the answer receiving step to a predetermined user. This allows a manager or other person in charge of managing operators to obtain answer content (comments) regarding conversations between a plurality of operators they manage from the generation AI in a manner that is easy for users to understand.

[0140] (Supplementary Note 8) The program according to Supplementary Note 6 or 7, wherein the analysis data acquisition step (S103) is a step of acquiring analysis data for a predetermined period, and the comment presentation step (S106) is a step of presenting a comment message every predetermined period. This allows the generation AI to obtain answer contents (comments) regarding the dialogue every predetermined period in a manner that is easy for the user to understand.

[0141] (Supplementary Note 9) The program according to any one of Supplementary Notes 6 to 8, wherein the comment presentation step (S106) is a step of presenting a comment message including the content of the answer received in the answer reception step together with the analysis data acquired in the analysis data acquisition step. This allows the analysis data related to the dialogue to be confirmed along with the content of the answer. The analysis data can be confirmed more effectively.

[0142] (Supplementary Note 10) A method executed by a computer having a processor and a memory, wherein the processor executes all of the steps executed in any of the inventions according to Supplementary Note 1 to Supplementary Note 9. This makes it possible to create input data such as prompts to be input to a generation AI such as a large-scale language model for obtaining answer contents (comments) in a form that is easy for users to understand, from analysis data related to a dialogue between multiple users.

[0143] (Supplementary Note 11) An information processing device comprising a control unit and a storage unit, wherein the control unit executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 9. This makes it possible to create input data such as prompts to be input to a generation AI such as a large-scale language model for obtaining answer contents (comments) in a form that is easy for users to understand, from analysis data related to a dialogue between multiple users.

[0144] (Supplementary Note 12) A system comprising means for executing all steps executed in any of the inventions according to Supplementary Note 1 to Supplementary Note 9. This makes it possible to create input data such as prompts to be input to a generation AI such as a large-scale language model for obtaining answer contents (comments) in a form that is easy for users to understand, from analysis data relating to dialogues between multiple users.

[0145] 1 System, 10 Server, 101 Memory unit, 104 Control unit, 106 Input device, 108 Output device, 20 First user terminal, 201 Memory unit, 204 Control unit, 206 Input device, 208 Output device, 30 Second user terminal, 301 Memory unit, 304 Control unit, 306 Input device, 308 Output device, 50 Voice server (PBX), 501 Memory unit, 504 Control unit, 506 Input device, 508 Output device, 80 Generation AI, 801 Memory unit, 804 Control unit, 806 Input device, 808 Output device

Claims

1. A program that causes a computer having a processor and a storage unit to process information regarding interactions between a plurality of users, the processor: an analysis data acquisition step of acquiring analysis data obtained by analyzing the dialogue; an instruction input step of inputting an instruction sentence that outputs an explanation based on the analysis data and the analysis data acquired in the analysis data acquisition step into a large-scale language model; an answer acquisition step of acquiring an answer content including an explanation based on the analysis data from the large-scale language model; Run The analysis data includes numerical data regarding a plurality of conversations of a plurality of users who have had conversations. program.

2. The analysis data includes information about a predetermined dialogue, such as a speech feature related to the voice spoken by the speaker, a language feature related to the content of the speech, and at least one of the number of calls and the duration of the call. The program according to claim 1.

3. The instruction input step includes: an instruction for outputting an explanation of the characteristics of the dialogue response based on the analysis data; An instruction statement for outputting improvements to the dialogue based on the analysis data; a directive for outputting items that have changed in the dialogue based on the analysis data; a command statement for outputting a target achievement status of an operator or a group to which a plurality of operators belong based on the analysis data; an instruction for outputting comparison results for each of a plurality of operators or a plurality of groups based on the analysis data; inputting the instruction sentence according to at least one of the following into the large-scale language model; The program according to claim 1.

4. The instruction input step includes: Information indicating one or more operators or one or more groups whose dialogue is judged to be excellent based on a score for judging the quality of dialogue for each operator or group to which multiple operators belong; Information indicating one or more operators or one or more groups whose dialogue is determined to be poor based on a score for determining the quality of dialogue for each operator or group to which multiple operators belong; and inputting information relating to at least one of the above into the large-scale language model; The program according to claim 1.

5. the analysis data acquisition step is a step of acquiring the analysis data obtained by analyzing the dialogue performed by a predetermined operator, the processor: a comment presentation step of presenting a comment message including the answer content acquired in the answer acquisition step to the predetermined operator; To execute The program according to claim 1.

6. the analysis data acquisition step is a step of acquiring the analysis data related to each of the plurality of operators by analyzing a plurality of conversations performed by the plurality of operators, the processor: a comment presenting step of presenting a comment message including the answer content acquired in the answer acquiring step to a predetermined user; To execute The program according to claim 1.

7. the analysis data acquisition step is a step of acquiring the analysis data for a predetermined period of time, The comment presentation step is a step of presenting the comment message every predetermined period. The program according to claim 5.

8. the comment presentation step is a step of presenting the comment message including the analysis data acquired in the analysis data acquisition step and the answer content acquired in the answer acquisition step. The program according to claim 5.

9. A method implemented on a computer having a processor and a memory, wherein the processor performs all of the steps performed in the invention according to any one of claims 1 to 8.

10. 9. An information processing device comprising a control unit and a storage unit, wherein the control unit executes all of the steps executed in the invention according to any one of claims 1 to 8.

11. A system having means for executing all steps performed in an invention according to any one of claims 1 to 8.