Program, information processing system and information processing method

Abstract

To provide a program for inferring a response memo more appropriate for a call between a user and a customer, an information processing system, and an information processing method.SOLUTION: A method includes: a step of receiving voice data relating to a call; an acquisition step of acquiring a call attribute of the call; a selection step of selecting a learning model based on the call attribute; and an inference step of inferring a response memo for the call by applying the voice data to the selected learning model. The call attribute includes at least one of user attribute, customer attribute, call category, and incoming / outgoing call type.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing system, and an information processing method. [Background technology]

[0002] Conventionally, in an information processing system that provides a call service, a support system for an operator has been known. Patent Document 1 discloses a system for providing call voice data with metadata that receives and stores call voice data from mobile terminals and reproduces and provides the desired call voice data. Patent Document 2 discloses a telephone response evaluation system that analyzes the voice information of an operator when answering a call and evaluates the operator's telephone response. Patent Document 3 discloses a response presentation device for assisting call center operators. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-108541 [Patent Document 2] Patent Publication No. 2014-086942 [Patent Document 3] Patent Publication No. 2020-098322 Summary of the Invention [Problem to be solved by the invention]

[0004] However, when adding a response memo to a call based on the call voice related to the call between the user and the customer, it has not been possible to infer a more appropriate response memo.

[0005] Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology that infers response notes that are more appropriate for a call between a user and a customer. [Means for solving the problem]

[0006] A program having a processor and a memory unit for causing a computer to manage data related to calls made between a user and a customer, the program causing the processor to execute the steps of accepting voice data related to the call, acquiring call attributes of the call, selecting a learning model based on the call attributes, and inferring a response memo for the call by applying the voice data to the selected learning model. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to infer a response memo that is more suitable for a call between a user and a customer. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing the overall configuration of an information processing system 1. FIG. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the server 10. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a user terminal 20. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a CRM system 30. [Figure 5] 3 is a block diagram showing the functional configuration of a customer terminal 50. FIG. [Figure 6] FIG. 10 is a diagram showing the data structure of a user table 1012. [Figure 7] FIG. 10 is a diagram showing the data structure of an organization table 1013. [Figure 8] FIG. 10 is a diagram showing the data structure of a call table 1014. [Figure 9] FIG. 10 is a diagram showing the data structure of a voice recognition table 1015. [Figure 10] FIG. 10 is a diagram showing the data structure of a summary table 1016. [Figure 11]FIG. 10 is a diagram showing the data structure of a response memo table 1017. [Figure 12] FIG. 10 is a diagram showing the data structure of a customer table 3012. [Figure 13] FIG. 10 is a diagram showing the data structure of a response history table 3013. [Figure 14] 10 is a flowchart showing the operation of the summarization process (first embodiment). [Figure 15] 10 is a flowchart showing the operation of the summarization process (second embodiment). [Figure 16] 10 is a flowchart showing the operation of a response memo adding process. [Figure 17] 10 is a flowchart showing the operation of a response memo suggestion process. [Figure 18] 10 is a flowchart showing the operation of a score calculation process. [Figure 19] 10 is a flowchart showing the operation of a CRM storage process. [Figure 20] 10 is a flowchart showing the operation of a call display process. [Figure 21] FIG. 10 is a diagram showing an outline of binding processing in the summarization processing (first embodiment). [Figure 22] FIG. 10 is a diagram showing an outline of additional extraction processing in the summarization processing (second embodiment). [Figure 23] FIG. 10 is a diagram showing an outline of a summary display process. [Figure 24] FIG. 10 is a diagram showing an example of a screen of a CRM service in the call display process. [Figure 25] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0010] <Outline of Information Processing System 1> 1 is a diagram showing the overall configuration of an information processing system 1. The information processing system 1 in the present disclosure is an information processing system that provides a call service according to the present disclosure. The information processing system 1 is an information processing system that provides a service related to calls made between a user and a customer, and also stores and manages data related to the calls.

[0011] <Basic configuration of information processing system 1> An information processing system 1 according to the present disclosure is shown in Fig. 1. The information processing system 1 includes a server 10, a plurality of user terminals 20A, 20B, and 20C, a CRM system 30, a voice server (PBX) 40, and customer terminals 50A, 50B, and 50C connected to the voice server (PBX) 40 via a telephone network T, all of which are connected via a network N. Fig. 2 is a block diagram showing the functional configuration of the server 10. Fig. 3 is a block diagram showing the functional configuration of the user terminal 20. Fig. 4 is a block diagram showing the functional configuration of the CRM system 30. Fig. 5 is a block diagram showing the functional configuration of the customer terminal 50.

[0012] The server 10 is an information processing device that provides a service of storing and managing data (call data) related to calls made between users and customers.

[0013] The user terminal 20 is an information processing device operated by a user who uses a service. The user terminal 20 may be, for example, a desktop personal computer (PC) or a laptop PC, or may be a mobile terminal such as a smartphone or tablet. It may also be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch terminal.

[0014] The CRM system 30 is an information processing device managed and operated by a business providing CRM (Customer Relationship Management) services (CRM business operator). CRM services include SalesForce, HubSpot, Zoho CRM, and kintone.

[0015] The voice server (PBX) 40 is an information processing device that functions as a switchboard that connects the network N and the telephone network T to each other, thereby enabling calls between the user terminal 20 and the customer terminal 50.

[0016] The customer terminal 50 is an information processing device operated by the customer when making a call with a user. The customer terminal 50 may be, for example, a mobile terminal such as a smartphone or tablet, a stationary personal computer (PC) or a laptop PC, or a wearable terminal such as a head mounted display (HMD) or a wristwatch terminal.

[0017] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, user terminal 20, CRM system 30, voice server (PBX) 40, and customer terminal 50, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0018] The configuration and operation of each device will be explained below.

[0019] <Functional configuration of Server 10> 2 shows the functional configuration realized by the hardware configuration of the server 10. The server 10 includes a storage unit 101 and a control unit 104.

[0020] <Configuration of the storage unit of the server 10> The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , an organization table 1013 , a call table 1014 , a voice recognition table 1015 , a summary table 1016 , and a response memo table 1017 . Fig. 6 is a diagram showing the data structure of the user table 1012. Fig. 7 is a diagram showing the data structure of the organization table 1013. Fig. 8 is a diagram showing the data structure of the call table 1014. Fig. 9 is a diagram showing the data structure of the voice recognition table 1015. Fig. 10 is a diagram showing the data structure of the summary table 1016. Fig. 11 is a diagram showing the data structure of the response memo table 1017.

[0021] User table 1012 is a table that stores and manages information about member users (hereinafter referred to as users) who use the service. When a user registers to use the service, the user's information is stored in a new record in user table 1012. This allows the user to use the service according to the present disclosure. User table 1012 is a table that uses the user ID as the primary key and has columns for user ID, CRM ID, organization ID, user name, collaboration mode, user attributes, and evaluation index.

[0022] The user ID is an item for storing user identification information for identifying a user. The CRMID is an item that stores identification information for identifying a user in the CRM system 30. A user can receive CRM services by logging in to the CRM system 30 using the CRMID. In other words, the user ID in the server 10 and the CRMID in the CRM system 30 are linked. The organization ID is an item for storing the organization ID of the organization to which the user belongs. The user name is an item for storing the user's name. The linkage mode is an item for storing setting items (linkage settings) when data related to a call made between a user and a customer is stored in an external CRM system. In the present disclosure, the collaboration mode is stored for each user, but it may also be stored for each organization or department in the organization table. In this case, the collaboration mode applied to each user is applied to each user by referencing the collaboration mode for each organization or department stored in the organization table. In this case, a uniform collaboration mode can be applied to all users belonging to the organization or department. The user attributes are items that store information about the user's attributes, such as the user's age, sex, hometown, dialect, and occupation (sales, customer support, etc.). The evaluation index is an item that stores quantitative evaluation indexes for the user's call handling skills. Specifically, the evaluation indexes are numerical values ​​calculated by applying a predetermined algorithm to each index of analysis data (talk:listen ratio, number of silences, number of overlaps, number of rallies, fundamental frequency, intonation strength, speaking speed, number of fillers, talk script agreement, etc.) obtained by analyzing the user's past voice data, which will be described later. For example, in the field of inside sales, the evaluation index corresponds to a quantitative representation of each user's customer service skills, and users with higher evaluation indexes are expected to have better sales results.

[0023] The organization table 1013 is a table that defines information about organizations to which a user belongs. Organizations include any organization or group, such as a company, a corporation, a corporate group, a club, or various associations. Organizations may also be defined for more detailed subgroups, such as company departments (sales department, general affairs department, customer support department). The organization table 1013 is a table that uses the organization ID as a primary key and has columns for the organization ID, organization name, and organization attributes.

[0024] The organization ID is an item for storing organization identification information for identifying an organization. The organization name is an item for storing the name of an organization, and includes any organization or group name such as a company name, a corporate name, a corporate group name, a club name, or a group name. The organizational attribute is an item for storing information about the attributes of an organization, such as the type of organization (company, corporate group, other organization, etc.) and the type of industry (real estate, finance, etc.).

[0025] The call table 1014 is a table that stores and manages call data related to calls made between users and customers. The call table 1014 is a table that uses the call ID as a primary key and has columns for the call ID, user ID, customer ID, call category, incoming / outgoing call type, voice data, whether voice recognition is enabled, whether summary is enabled, and analysis data.

[0026] The call ID is an item for storing call data identification information for identifying call data. The user ID is an item for storing the user ID (user identification information) of the user in a call between the user and the customer. The customer ID is an item for storing the customer ID (customer identification information) of a customer during a call between a user and a customer. The call category is an item that stores the type (category) of a call made between a user and a customer. Call data is classified by call category. The call category stores values ​​such as telephone operator, telemarketing, customer support, and technical support depending on the purpose of the call made between the user and the customer. The call type is an item that stores information for distinguishing whether a call between a user and a customer was made by the user (outbound) or received by the user (inbound). The audio data field stores audio data from calls between users and customers. Various audio data formats, such as mp4 and wav, can be used for the audio data. It may also store reference information (paths) for audio data files located elsewhere. The voice data may be in a format in which identifiers are set so that the user's voice and the customer's voice can be independently identified. In this case, the control unit 104 of the server 10 can perform independent analysis processing on the user's voice and the customer's voice. In the present disclosure, moving image data including audio information may be used instead of audio data. Also, the concept of audio data in the present disclosure includes audio data included in video data. The "Speech Recognition Present" field stores information for determining whether speech recognition processing has been performed on the speech data of a call between a user and a customer. If speech recognition processing has been performed on the speech data, information indicating that speech recognition processing has been performed is stored. If speech recognition processing has not been performed on the speech data, blank, null, or other information indicating that speech recognition processing has not been performed is stored. The "summarization" field stores information for determining whether summarization has been performed on the voice data of a call between a user and a customer. If summarization has been performed, information indicating that summarization has been performed is stored. If summarization has not been performed, a blank, null, or other information indicating that summarization has not been performed is stored. The analysis data is an item that stores analysis data obtained by analyzing the voice data of calls between users and customers. The analysis data includes the talk:listen ratio, number of silences, number of overlaps, number of rallies, fundamental frequency, intonation strength, speaking speed, number of fillers, and talk script agreement.

[0027] The voice recognition table 1015 is a table that stores voice recognition information including the time of speech, speaker, and text obtained by performing voice recognition processing on the voice data of a call between a user and a customer. The voice recognition table 1015 is a table that has columns for call ID, text, time of speech, and speaker.

[0028] The call ID is an item for storing the call ID (call data identification information) of the call data from which the voice recognition information is generated. The text field stores text information of the text recognition results for each section (speech section) of the voice data of a call between a user and a customer. Specifically, the content of the sentences spoken by the speaker for each speech section of the voice data is stored as text data. The speech time is an item for storing the start time of the speech section in the speech data (section speech data). Note that the speech time may be any time related to each speech section, such as the start time of each speech section, the end time of each speech section, or any time between each speech section. The speaker stores information for identifying the speaker of the section voice data. Specifically, it is information for identifying the user or customer. Note that user identification information, customer identification information, etc. for identifying the speaker, such as a user ID or a customer ID, may also be stored.

[0029] Summary table 1016 is a table that stores summary information consisting of speech time, speaker, and text obtained by performing a summary process on the speech recognition information of a call between a user and a customer. The summary information is information that is related to a call between a user and a customer and consists of text that characterizes the call. By checking the summary information, the user can quickly understand the content of the call between the user and the customer. Summary table 1016 is a table that has columns for call ID, text, speech time, and speaker.

[0030] The call ID is an item for storing the call ID (call data identification information) of the call data from which summary information is generated. The text is an item for storing the text of the speech recognition information extracted as summary information. The utterance time is an item for storing the utterance time of the speech recognition information extracted as summary information. The speaker is an item for storing the speaker of the speech recognition information extracted as summary information.

[0031] The response memo table 1017 is a table that stores and manages information about response memos that are linked to and stored in connection with call data related to calls made between a user and a customer. By setting (attaching) response memos to call data, a user can organize and manage a large amount of call data. In addition, the server 10 can change processing conditions using the response memos attached to the call data when performing various control processes. The response memo table 1017 is a table that uses the memo ID as a primary key and has columns for the memo ID, call ID, attacher ID, memo content, and memo date and time.

[0032] The memo ID is an item for storing response memo identification information for identifying a response memo. The call ID is an item for storing the call ID (call data identification information) of the call data to which the response memo is attached. The assignor ID is an item for storing the user ID of the user who assigned the response memo to the call data. The memo content is an item for storing the content of the response memo added to the call data. The memo content is usually character string (text) information. The memo date and time is an item for storing the date and time when the user added a response memo to the call data.

[0033] <Configuration of the control unit of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041, a setting unit 1042, a recognition unit 1043, an analysis unit 1044, an importance calculation unit 1045, a summarization unit 1046, a learning unit 1047, a response memo suggestion unit 1048, a response memo attachment unit 1049, a score calculation unit 1050, a CRM storage control unit 1051, and a display control unit 1052. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

[0034] The user registration control unit 1041 performs processing to store information about users who wish to use the service according to the present disclosure in the user table 1012. Information such as the user ID, user name, and user attributes stored in the user table 1012 is obtained when a user opens a web page operated by a service provider from any information processing terminal, enters the user ID, user name, and user attributes in a predetermined input form, and transmits the information to the server 10. The user registration control unit 1041 of the server 10 stores the received user ID, user name, and user attributes in a new record in the user table 1012, thereby completing the user registration. This allows the user stored in the user table 1012 to use the service. Before the user registration control unit 1041 registers the user information in the user table 1012, the service provider may conduct a predetermined examination to restrict whether or not the user is permitted to use the service. The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or may be automatically set by the user registration control unit 1041 of the server 10. When registering a user, the user registration control unit 1041 may associate information such as the organization ID, organization name, and organization attributes of the organization to which the user belongs with the user and store the information in the organization table 1013. The information on the organization to which the user belongs may be entered by the user himself or may be registered by an administrator of the organization to which the user belongs, an operator of the service according to the present disclosure, or the like.

[0035] The setting unit 1042 executes a cooperation mode setting process, the details of which will be described later. The recognition unit 1043 executes a voice recognition process, the details of which will be described later. The analysis unit 1044 executes a voice analysis process, the details of which will be described later. The importance calculation unit 1045 executes importance calculation processing, the details of which will be described later. The summarizing unit 1046 executes summarization processing, the details of which will be described later. The learning unit 1047 executes a learning process, the details of which will be described later. The response memo suggestion unit 1048 executes a response memo suggestion process, the details of which will be described later. The response memo attachment unit 1049 executes a response memo attachment process, the details of which will be described later. The score calculation unit 1050 executes a score calculation process, the details of which will be described later. The CRM storage control unit 1051 executes CRM storage processing, the details of which will be described later. The display control unit 1052 executes a call display process, the details of which will be described later.

[0036] <Functional Configuration of User Terminal 20> 3 shows the functional configuration realized by the hardware configuration of the user terminal 20. The user terminal 20 includes a storage unit 201, a control unit 204, an input device 206 connected to the user terminal 20, and an output device 208. The input device 206 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, a keyboard 2065, and a mouse 2066. The output device 208 includes a display 2081 and a speaker 2082.

[0037] <Configuration of the storage unit of the user terminal 20> The storage unit 201 of the user terminal 20 stores a user ID 2011 for identifying a user who uses the user terminal 20, an application program 2012, and a CRM ID 2013. The user ID is the user's account ID for the server 10. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the user with the service according to the present disclosure. The user ID includes information such as a session ID temporarily assigned by the server 10 to identify the user using the user terminal 20. The CRMID is the user's account ID for the CRM system 30. The user transmits the CRMID 2013 from the user terminal 20 to the CRM system 30. The CRM system 30 identifies the user based on the CRMID 2013 and provides the CRM service to the user. The CRMID 2013 includes information such as a session ID temporarily assigned by the CRM system 30 to identify the user using the user terminal 20. The application program 2012 may be stored in advance in the storage unit 201, or may be configured to be downloaded from a web server operated by a service provider via a communication IF. The application program 2012 includes an interpreter-type programming language such as JavaScript (registered trademark) that is executed on a web browser application stored in the user terminal 20.

[0038] <Configuration of the control unit of the user terminal 20> The control unit 204 of the user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 executes an application program 2012 stored in the storage unit 201, thereby realizing the functional units of the input control unit 2041 and the output control unit 2042. The input control unit 2041 of the user terminal 20 acquires information output from input devices such as the camera 2061, microphone 2062, position information sensor 2063, motion sensor 2064, keyboard 2065, and mouse 2066 connected to the user terminal 20 and executes various processes. The input control unit 2041 of the user terminal 20 executes a process of transmitting the information acquired from the input device 206 to the server 10 together with the user ID 2011. Similarly, the input control unit 2041 of the user terminal 20 executes a process of transmitting the information acquired from the input device 206 to the CRM system 30 together with the CRMID 2013. The output control unit 2042 of the user terminal 20 receives operations by the user on the input device 206 and information from the server 10 and the CRM system 30, and executes control processes for the display content of the display 2081 and the voice output content of the speaker 2082 connected to the user terminal 20.

[0039] <Functional Configuration of CRM System 30> The functional configuration realized by the hardware configuration of the CRM system 30 is shown in FIG. 4. The CRM system 30 includes a storage unit 301 and a control unit 304. The user has separately concluded a contract with a CRM operator, and can receive CRM services by accessing (logging in) a website operated by the CRM operator via a web browser or the like using the CRMID 2013 set for each user.

[0040] <Configuration of the Storage Unit of CRM System 30> The storage unit 301 of the CRM system 30 includes a customer table 3012 and a response history table 3013. FIG. 12 is a diagram showing the data structure of the customer table 3012. FIG. 13 is a diagram showing the data structure of the response history table 3013.

[0041] The customer table 3012 is a table for storing and managing customer information. The customer table 3012 is a table having columns of customer ID, user ID, name, telephone number, customer attribute, customer organization name, and customer organization attribute with the customer ID as the primary key.

[0042] The customer ID is an item for storing customer identification information for identifying a customer. The user ID is an item that stores the user ID (user identification information) of the user associated with the customer. The user can display a list of customers associated with their own user ID and can make calls to customers. In the present disclosure, a customer is linked to a user, but a customer may also be linked to an organization (organization ID in organization table 1013). In this case, a user belonging to an organization can display a list of customers linked to his / her organization ID and make calls to the customers. The name is an item for storing the name of the customer. The telephone number is an item for storing the telephone number of the customer. A user can access a website provided by the CRM system, select the customer to whom they wish to make a call, and perform a predetermined operation such as "call" to make a call to the customer's phone number from the user terminal 20. The customer attributes are items that store information about the attributes of the customer, such as the customer's age, sex, hometown, dialect, and occupation (sales, customer support, etc.). The customer organization name is an item for storing the name of the organization to which the customer belongs. The organization name includes any organization or group name such as a company name, a corporate name, a corporate group name, a club name, or a group name. The customer organization attribute is an item for storing information about the attributes of the organization, such as the type of customer organization (company, corporate group, other organization, etc.) and the type of industry (real estate, finance, etc.). The customer attributes, customer organization name, and customer organization attributes may be configured to be stored by being input by the user, or may be input by the customer by accessing a predetermined website.

[0043] The response history table 3013 is a table for storing and managing records (response history information) of customer responses (response history). When the customer response is a sales activity, records of past sales activities (date and time, content of the sales activity, etc.) are stored. The response history table 3013 is a table with the response history ID as the primary key, and has columns for the response history ID, call ID, URL, customer ID, user ID (caller), dial date and time, call start date and time, call end date and time, and comment.

[0044] In the present disclosure, each time a call is made between a user and a customer, a new record is created and response history information is stored. The response history ID is an item for storing response history identification information for identifying a response history. The call ID is an item for storing the call ID (call data identification information) of the call data related to the response history. As a result, the response history in the CRM system is linked to the call data in the server 10. The URL is URL (Uniform Resource Locator) information that includes a character string that uniquely identifies the call ID. The URL may directly include the call ID, or may include a character string that can be decoded to obtain the call ID, or another specific character string that can be used to obtain the call ID by referencing a table (not shown). By operating the browser on the user terminal 20 and opening the URL, the user can access the server 10 and check an analysis result screen that visualizes the analysis data of the call data related to the response history. The customer ID is an item for storing the customer ID (customer identification information) of the customer related to the response history. The user ID is an item for storing the user ID (user identification information) of the user related to the response history. The dialing date and time is an item for storing the date and time when the user made a call to a customer in relation to the response history. The call start date and time is an item that stores the start date and time of a call between a user and a customer in the response history. The call end date and time is an item that stores the end date and time of a call between a user and a customer in the response history. Comments are items that store text information such as memos and comments regarding the response history. The user can edit the reasons for comments, such as matters noticed or postponed regarding the response history. Also, the control unit 104 of server 10 can freely edit comments by sending a predetermined request to the API (Application Programming Interface) of the CRM system 30.

[0045] <Configuration of the control unit of the CRM system 30> The control unit 304 of the CRM system 30 includes a user registration control unit 3041. The control unit 304 realizes each functional unit by executing the application program 3011 stored in the storage unit 301.

[0046] The CRM system 30 provides functions called API (Application Programming Interface), SDK (Software Development Kit), and code snippets (hereinafter referred to as "beacons"). The user can perform linking settings such as account information for the server 10 and the CRM system 30 according to the present disclosure in advance, so that the control unit 104 of the server 10 and the control unit 304 of the CRM system 30 can communicate with each other and realize arbitrary information processing.

[0047] <Overview of the voice server (PBX) 40> When there is an outgoing call from the user to the customer, the voice server (PBX) 40 makes an outgoing call (rings) to the customer terminal 50. When there is an incoming call from the customer to the user, the voice server (PBX) 40 sends a message indicating that (hereinafter referred to as "incoming call notification message") to the user terminal 20. Also, the voice server (PBX) 40 can send an incoming call notification message to the beacons, SDK, API, etc. provided by the server 10.

[0048] <Functional configuration of the customer terminal 50> 5 shows the functional configuration realized by the hardware configuration of the customer terminal 50. The customer terminal 50 includes a memory unit 501, a control unit 504, a touch panel 506, a touch-sensitive device 5061, a display 5062, a microphone 5081, a speaker 5082, a position information sensor 5083, a camera 5084, and a motion sensor 5085.

[0049] <Configuration of the storage unit of the customer terminal 50> The storage unit 501 of the customer terminal 50 stores a telephone number 5011 and an application program 5012 of the customer who uses the customer terminal 50 . The application program 5012 may be stored in advance in the storage unit 501, or may be downloaded from a web server operated by the service provider via a communication IF. The application program 5012 includes an interpreter-type programming language such as JavaScript (registered trademark) that is executed on a web browser application stored in the customer terminal 50.

[0050] <Configuration of the control unit of the customer terminal 50> The control unit 504 of the customer terminal 50 includes an input control unit 5041 and an output control unit 5042. The control unit 504 executes an application program 5012 stored in the storage unit 501, thereby realizing the functional units of the input control unit 5041 and the output control unit 5042. The input control unit 5041 of the customer terminal 50 acquires information output from input devices such as the user's operations on the touch-sensitive device 5061 of the touch panel 506, voice input to the microphone 5081, and the position information sensor 5083, camera 5084, and motion sensor 5085, and performs various processes. The output control unit 5042 of the customer terminal 50 receives information from the server 10 and operations performed by the user on the input device, and executes control processing for the content displayed on the display 5062, the content of audio output from the speaker 5082, and the like.

[0051] <Operation of information processing system 1> Each process of the information processing system 1 will be described below. FIG. 14 is a flowchart showing the operation of the summarization process (first embodiment). FIG. 15 is a flowchart showing the operation of the summarization process (second embodiment). FIG. 16 is a flowchart showing the operation of the response memo attaching process. FIG. 17 is a flowchart showing the operation of the response memo suggestion process. FIG. 18 is a flowchart showing the operation of the score calculation process. FIG. 19 is a flowchart showing the operation of the CRM storage process. FIG. 20 is a flowchart showing the operation of the call display process. FIG. 21 is a diagram showing an outline of the binding process in the summarization process (first embodiment). FIG. 22 is a diagram showing an outline of the additional extraction process in the summarization process (second embodiment). FIG. 23 is a diagram showing an outline of the summary display process. FIG. 24 is a diagram showing an example of a screen of a CRM service in the call display process.

[0052] <Term definition> In describing each process of the information processing system 1, terms will be defined as follows. The call data is data related to calls made between a user and a customer, and includes data stored in each item of the call table 1014, data stored in each item of the voice recognition table 1015 linked by a call ID, data stored in each item of the summary table 1016 linked by a call ID, etc. The call data includes data stored in each item of the response memo table 1017 and the response history table 3013 linked by a call ID, etc. Call attributes are data related to the attributes of calls made between a user and a customer, and include user attributes, the organization name or organization attributes of the organization to which the user belongs, customer attributes, the organization name or organization attributes of the organization to which the customer belongs, call category, caller / receiver type, etc. In other words, call data is characterized by attribute values ​​such as the user attributes of the user making the call, the customer attributes of the customer making the call, the call category of the call, and the caller / receiver type.

[0053] <Call processing> The outgoing call process is a process in which a user makes an outgoing call (call) to a customer.

[0054] <Outline of outgoing call processing> The call processing is a series of processes in which the user selects a customer to whom he / she wishes to make a call from among a plurality of customers displayed on the screen of the user terminal 20, and performs a call operation to make a call to the customer.

[0055] <Details of outgoing call processing> The call processing of the information processing system 1 when a user makes a call to a customer will be described.

[0056] When a user makes a call to a customer, the information processing system 1 executes the following process.

[0057] The user operates the user terminal 20 to launch a web browser and access the website of the CRM service provided by the CRM system 30. The user can display a list of their own customers on the display 2081 of the user terminal 20 by opening a customer management screen provided by the CRM service. Specifically, the user terminal 20 transmits a request to display a list of CRMIDs 2013 and customers to the CRM system 30. Upon receiving the request, the CRM system 30 searches the customer table 3012 and transmits information about the user's customers, such as customer IDs, names, telephone numbers, customer attributes, customer organization names, and customer organization attributes, to the user terminal 20. The user terminal 20 displays the received information about the customers on the display 2081 of the user terminal 20.

[0058] The user presses and selects a customer to whom they wish to make a call from the list of customers displayed on the display 2081 of the user terminal 20. With the customer selected, the user presses the "Call" button or the phone number button displayed on the display 2081 of the user terminal 20 to send a request including the phone number to the CRM system 30. The CRM system 30, which receives the request, sends the request including the phone number to the server 10. The server 10, which receives the request, sends a call request to the voice server (PBX) 40. When the voice server (PBX) 40 receives the call request, it makes a call (calls) to the customer terminal 50 based on the received phone number.

[0059] In response to this, the user terminal 20 controls the speaker 2082 etc. to make a sound indicating that a call is being made (a call) by the voice server (PBX) 40. The display 2081 of the user terminal 20 also displays information indicating that a call is being made (a call) to the customer by the voice server (PBX) 40. For example, the display 2081 of the user terminal 20 may display the words "Calling."

[0060] The customer terminal 50 becomes ready for a call when the customer lifts the receiver (not shown) of the customer terminal 50 or presses an "answer" button or the like that is displayed when a call arrives on the touch panel 506 of the customer terminal 50. In response to this, the voice server (PBX) 40 transmits information indicating that the customer terminal 50 has responded (hereinafter referred to as a "response event") to the user terminal 20 via the server 10, the CRM system 30, etc. As a result, the user and the customer are able to communicate using the user terminal 20 and the customer terminal 50, respectively, and can communicate with each other. Specifically, the user's voice collected by the microphone 2062 of the user terminal 20 is output from the speaker 5082 of the customer terminal 50. Similarly, the customer's voice collected by the microphone 5081 of the customer terminal 50 is output from the speaker 2082 of the user terminal 20.

[0061] When the display 2081 of the user terminal 20 becomes available for communication, it receives the response event and displays information indicating that the call is in progress. For example, the display 2081 of the user terminal 20 may display the words "Answering."

[0062] When the call is ready, the call storage process and CRM storage process, which will be described later, are executed.

[0063] <Incoming call processing> The incoming call process is a process in which the user receives an incoming call (a call) from a customer.

[0064] <Outline of incoming call processing> The incoming call processing is a series of processes in which, when a user has an application running on the user terminal 20, the user receives a call when a customer makes a call to the user.

[0065] <Incoming call processing details> The following describes the incoming call processing of the information processing system 1 when the user receives an incoming call (call) from a customer.

[0066] When the user receives a call from a customer, the information processing system 1 executes the following process.

[0067] The user operates the user terminal 20 to launch a web browser and access the website of the CRM service provided by the CRM system 30. At this time, the user is assumed to be logged in to the CRM system 30 using his or her own account in the web browser and is on standby. Note that the user only needs to be logged in to the CRM system 30, and may also be performing other tasks related to the CRM service.

[0068] The customer operates the customer terminal 50, inputs a predetermined telephone number assigned to the voice server (PBX) 40, and makes a call to the voice server (PBX) 40. The voice server (PBX) 40 receives the call made by the customer terminal 50 as an incoming call event.

[0069] The voice server (PBX) 40 transmits an incoming call event to the server 10. Specifically, the voice server (PBX) 40 transmits an incoming call request including the customer's telephone number 5011 to the server 10. The server 10 transmits the incoming call request to the user terminal 20 via the CRM system 30. In response to this, the user terminal 20 controls the speaker 2082 etc. to make a sound indicating that an incoming call is being received by the voice server (PBX) 40. The display 2081 of the user terminal 20 displays information indicating that an incoming call is being received from the customer by the voice server (PBX) 40. For example, the display 2081 of the user terminal 20 may display the words "Incoming call".

[0070] The user terminal 20 accepts a response operation by the user. The response operation is realized, for example, by the user lifting a receiver (not shown) on the user terminal 20, or by the user operating the mouse 2066 to press a button labeled "Answer the call" on the display 2081 of the user terminal 20. When the user terminal 20 receives the response operation, it transmits a response request to the voice server (PBX) 40 via the CRM system 30 and the server 10. The voice server (PBX) 40 receives the transmitted response request and establishes voice communication. This enables the user terminal 20 to communicate with the customer terminal 50. The display 2081 of the user terminal 20 displays information indicating that a call is in progress. For example, the display 2081 of the user terminal 20 may display the words "Call in progress."

[0071] When the call is ready, the call storage process and CRM storage process, which will be described later, are executed.

[0072] <Call memory processing> The call storage process is a process for storing data relating to calls made between a user and a customer.

[0073] <Outline of call memory processing> The call storage process is a series of processes for storing data relating to a call in the call table 1014 when a call is initiated between a user and a customer.

[0074] <Details of call memory processing> When a call is initiated between a user and a customer, the voice server (PBX) 40 records voice data relating to the call between the user and the customer and transmits it to the server 10. When the control unit 104 of the server 10 receives the voice data, it creates a new record in the call table 1014 and stores data relating to the call between the user and the customer. Specifically, the control unit 104 of the server 10 stores the user ID, customer ID, call category, incoming / outgoing call type, and the content of the voice data in the call table 1014.

[0075] The control unit 104 of the server 10 acquires the user ID 2011 of the user from the user terminal 20 during the outgoing call processing or incoming call processing, and stores it in the user ID field of the new record. The control unit 104 of the server 10 queries the CRM system 30 based on the telephone number during outgoing or incoming call processing. The CRM system 30 searches the customer table 3012 by telephone number to obtain the customer ID and transmits it to the server 10. The control unit 104 of the server 10 stores the obtained customer ID in the customer ID field of the new record. The control unit 104 of the server 10 stores the call category value set in advance for each user or customer in the call category field of the new record. Note that the call category may be stored by the user selecting or inputting a value for each call. The control unit 104 of the server 10 identifies whether the call being made is made by the user or the customer, and stores either the value of outbound (made by the user) or inbound (made by the customer) in the call type field of the new record. The control unit 104 of the server 10 stores the voice data received from the voice server (PBX) 40 in the voice data field of the new record. Note that the voice data may be stored as a voice data file in another location, and reference information (path) for the voice data file may be stored after the call ends. The control unit 104 of the server 10 may also be configured to store the data after the call ends.

[0076] <Speech recognition processing> The voice recognition process is a process of converting voice data of a call between a user and a customer into text information by performing text recognition and storing the text information.

[0077] <Outline of speech recognition processing> The speech recognition process is a series of processes that divides the speech data stored in the speech table 1014 into sections where speech exists (utterance sections) and extracts section speech data. Text recognition is performed on each section speech data to generate speech recognition information and store it in the speech recognition table 1015.

[0078] <Details of speech recognition processing> The recognition unit 1043 of the server 10 searches the call table 1014 for records for which call storage processing has been performed but voice recognition processing has not been performed. Specifically, the recognition unit 1043 of the server 10 searches the call table 1014 for records for which the voice recognition presence / absence field is blank, null, or other information indicating that voice recognition processing has not been performed. Note that the recognition unit 1043 of the server 10 may determine that a record for which voice data is stored in the call table 1014 but a call ID does not exist in the voice recognition table 1015 is a record for which voice recognition processing has not been performed.

[0079] The recognition unit 1043 of the server 10 acquires (accepts) the call ID and voice data of a record for which voice recognition processing has not been performed. The recognition unit 1043 of the server 10 detects sections (utterance sections) in which voice exists from the acquired (accepted) voice data, and extracts voice data for each utterance section as section voice data. The section voice data is linked to the speaker and the time of speech for each utterance section. The recognition unit 1043 of the server 10 performs text recognition on the extracted section voice data to convert the section voice data into characters (text) (transcribe it into text). Note that the specific method of text recognition is not particularly limited. For example, conversion may be performed using signal processing technology, machine learning using AI (artificial intelligence), or deep learning.

[0080] The recognition unit 1043 of the server 10 associates a series of data, in which the text for each speech section is associated with the start time of each speech section and the speaker (user or customer), with the call ID to be processed, and stores the data in the speech recognition table 1015. The recognition unit 1043 of the server 10 stores information indicating that the speech recognition process has been completed in the speech recognition yes / no field of the call table 1014.

[0081] The text for each speech section of the voice data is linked to the time of speech and the speaker and stored as continuous time-series data in the voice recognition table 1015. By checking the data stored in the voice recognition table 1015, the user can check the contents of the call as text information without listening to the voice data.

[0082] The speech time may be any time related to each speech section, such as the start time of each speech section, the end time of each speech section, or any time between each speech section.

[0083] <Variations of speech recognition processing> In addition, during the voice recognition process, it is also possible to remove from the text any information that is meaningless in understanding the conversation between the user and the customer, such as fillers contained in the text, and store the voice recognition information in the voice recognition table 1015.

[0084] <Audio analysis processing> The voice analysis process is a process of analyzing voice data of a call between a user and a customer, generating analysis data, and storing the analysis data.

[0085] <Outline of voice analysis processing> The voice analysis process is a series of processes that executes the voice analysis process on the voice data stored in the call table 1014 to generate analysis data and store it in the call table 1014.

[0086] <Details of voice analysis processing> The analysis unit 1044 of the server 10 searches for records in the call table 1014 that have undergone call storage processing but have not undergone voice analysis processing. Specifically, the analysis unit 1044 of the server 10 searches for records in which voice data is stored but no analysis data is stored from the call table 1014. No analysis data is stored means that the analysis data item is blank, null, or other information indicating that no data is stored is stored. Alternatively, information indicating that the record is a target for voice analysis processing may be stored in a column (not shown), and the record in which this information is stored may be the target for voice analysis processing.

[0087] The server 10 acquires the call ID and voice data of a record that has not undergone voice analysis processing. The analysis unit 1044 of the server 10 analyzes the acquired voice data and calculates the talk:listen ratio, number of silences, number of overlaps, number of rallies, fundamental frequency, intonation strength, speech rate, speech rate ratio, number of fillers, talk script agreement, etc. Note that the specific analysis method is not particularly limited. For example, the analysis may be performed using signal processing technology, machine learning using AI (artificial intelligence), or deep learning.

[0088] The Talk:Listen ratio is the ratio of the user's speaking time to the speaking time of the other party. The number of silences is the number of times silences occurred during a call. The number of overlaps is the number of times overlaps occurred during a call. The number of rallies is the number of times the user and the customer interacted during the call (the number of times the conversation changed). The fundamental frequency is defined for each user or customer, and is information corresponding to the fundamental frequency of the user's or customer's voice, that is, the pitch of the voice. The strength of intonation is defined for each user or customer, and is information that quantitatively evaluates the magnitude of the user's or customer's intonation. The speech rate is defined for each user or customer and is the speed at which the user or customer speaks. The speech rate is indicated, for example, by the number of characters (or words) spoken per second. The speech rate ratio is information about the ratio of the speech rates of the user and the customer. Specifically, it is expressed as a numerical value obtained by dividing the speech rate of the user by the speech rate of the customer. For example, the larger the value, the faster the user's speech rate is compared to the customer. The number of fillers is the number of fillers (such as hesitations such as "eh," "um," etc.) detected in a sentence of speech recognition data. The number of fillers may be defined for each user or customer. The talk script agreement is information about the agreement between a talk script set for each user or each organization to which the user belongs and the content of the user's utterance detected from the voice recognition data.

[0089] The analysis unit 1044 of the server 10 stores the analysis data in the analysis data field of the record to be processed in the call table 1014.

[0090] <Summary Processing (First Example)> The summarization process (first embodiment) is a process for generating and storing summary information from speech recognition information. Details of the summarization process (first embodiment) will be explained below with reference to the flowchart in FIG.

[0091] <Outline of Summarization Process (First Example)> The summarization process (first embodiment) is a series of processes that acquires the speech recognition information stored in the speech recognition table 1015, arranges the text included in the speech recognition information in order of the time of speech, executes a binding process that binds the text before and after a speaker change into a text group, calculates the importance of the text and text group after the binding process, extracts the text to be used in the summary document based on the importance, generates the summary document, and stores it in the server 10 and the CRM system 30.

[0092] <Details of Summarization Process (First Example)> The summarizing unit 1046 of the server 10 automatically executes the summarizing process (first embodiment) periodically. In step S101, the summarizing unit 1046 of the server 10 periodically starts summarizing processing (first embodiment). Note that the summarizing unit 1046 of the server 10 may start summarizing processing (first embodiment) upon detecting the end of a call between a user and a customer.

[0093] In step S102, the summarizing unit 1046 of the server 10 refers to the call table 1014 and searches for a record in which information indicating that the speech recognition process has been completed is stored in the speech recognition status field. When the summarizing unit 1046 of the server 10 finds a record in which information indicating that the speech recognition process has been completed is stored, the summarizing unit 1046 acquires the record and proceeds to step S103 (Yes in step S102). After the call between the user and the customer ends, until the voice recognition process is completed, the voice recognition status field of the corresponding record in the call table 1014 does not store information indicating that the voice recognition process has been completed, so the summarization unit of the server 10 waits in step S102 (No in step S102).

[0094] In step S103, the summarization unit 1046 of the server 10 searches for records in which the summary presence / absence field of the records acquired in step S102 is blank, null, or other information indicating that summarization processing (first embodiment) has not been performed. Note that the summarization unit 1046 of the server 10 may determine that a record in which voice data is stored in the call table 1014 but a call ID does not exist in the summary table 1016 is a record in which summarization processing (first embodiment) has not been performed. When the summarizing unit 1046 of the server 10 finds a record for which the summarization process (first embodiment) has not been performed, it acquires the record and proceeds to step S104 (Yes in step S102).

[0095] In step S104, the summarization unit 1046 of the server 10 searches the speech recognition table 1015 based on the call ID of a record for which summarization processing (first embodiment) has not been performed, and obtains speech recognition information consisting of records of text, speaking time, and speaker.

[0096] In step S105, the summarizing unit 1046 of the server 10 performs binding processing on the acquired speech recognition information. Specifically, when the acquired speech recognition information is arranged in order of utterance time, a text group is generated that combines multiple texts before and after a change in speaker information. A text group is a data structure such as an array, and is information that includes multiple texts by different speakers. The text group may also include other speech recognition information such as utterance time and speaker.

[0097] 21 is a diagram showing an overview of the binding process in the summarization process (first embodiment), in which the text for each speaker is arranged downward in order of the time of speech. Identification numbers such as U1, U2..., C1, C2... are assigned to the text recognized for each speech section for each user and customer. The summarization unit 1046 of the server 10 generates a text group by combining, for example, the texts C4 and U5, in which the speaker information changes from customer to user.

[0098] In the present disclosure, a text group is generated by combining one text each from the user and the customer, but a text group may also be generated by combining two or more texts before and after a change in speaker information. The importance of texts before and after a change in speaker information (e.g., C4, U5, etc.) may be calculated in advance, and a text group may be generated by combining a number of texts according to the importance value. For example, the greater the importance, the more texts may be combined to generate a text group. In a conversation between three or more people, a text group may be generated by collecting multiple pieces of text before and after a change in speaker information.

[0099] In step S106, the importance calculation unit 1045 of the server 10 performs importance calculation processing on the text and the text group. The importance calculation method may be an arbitrary key sentence extraction algorithm. In this disclosure, an importance calculation method using an algorithm called LexRank will be described as an example. LexRank is an algorithm that takes multiple input sentences, divides each input sentence into words using morphological analysis, calculates the similarity (e.g., cosine similarity) for each sentence, calculates a graph structure based on the similarity between the sentences, and calculates the importance of each sentence based on the graph structure. Specifically, the importance calculation unit 1045 of the server 10 calculates the importance of each of the text and the text group by applying LexRank to each of the text and the text group as one input sentence.

[0100] <Importance Calculation Process (First Example)> In the importance calculation process (first embodiment), the importance calculation unit 1045 of the server 10 treats a text obtained by combining multiple texts included in a text group as a single sentence and applies LexRank to it. For example, C4 "Is Mr. Taguchi here?" and U5 "You must be Taguchi." are combined to form "Is Mr. Taguchi here? You must be Taguchi.", and LexRank is applied to calculate the importance.

[0101] <Importance Calculation Process (Second Embodiment)> In the importance calculation process (second embodiment), the importance calculation unit 1045 of the server 10 applies LexRank to each of the multiple texts included in the text group as an independent sentence. The importance calculation unit 1045 of the server 10 executes a process in which the sum of the importance calculated for the independent sentences is determined as the importance of the text group. For example, the importance calculation unit 1045 of the server 10 applies LexRank to each of the sentences C4 and U5 included in the text group, and calculates the importance of each of C4 and U5. Thereafter, the importance calculation unit 1045 of the server 10 executes a process of determining the importance of the text group as the sum of the importance of C4 and the importance of U5. Note that the importance of a text group may be a statistical value (such as the average, median, mode, maximum, or minimum) obtained by performing statistical processing on the importance calculated for each independent sentence. Specifically, the average or maximum value of the importance calculated for each independent sentence is suitable as the importance of a text group.

[0102] In step S107, the summarization unit 1046 of the server 10 extracts text or a group of texts whose importance has been calculated to be equal to or greater than a predetermined value. The summarization unit 1046 of the server 10 stores the extracted text and the text included in the group of texts in the summarization table 1016 together with the utterance time and speaker of each text in the speech recognition table 1015. As a result, summary information is stored in the summarization table 1016. The summarization unit 1046 of the server 10 stores information indicating that the summarization process (first embodiment) has been completed in the "Summarization" field of the conversation table 1014.

[0103] The summarization unit 1046 of the server 10 combines the extracted text and the text included in the text group into one sentence (summary text) along with the speaking time and speaker of each text in the speech recognition table 1015, generates a request including the call ID and summary text, and sends it to the CRM system 30. When the CRM system 30 receives the request, it searches the response history table 3013 using the call ID included in the request, and stores the received summary text in the comment field of the record that has the call ID.

[0104] <Summary Processing (Second Embodiment)> The summarization process (second embodiment) is a second embodiment of the process for generating and storing summary information from speech recognition information. Details of the summarization process (second embodiment) will be explained below with reference to the flowchart in FIG.

[0105] <Outline of Summarization Process (Second Embodiment)> The summarization process (second embodiment) acquires the speech recognition information stored in the speech recognition table 1015, calculates the importance of the text included in the speech recognition information, and extracts text to be used in a summary document based on the importance. When the text included in the speech recognition information is arranged in order of utterance time, the text before and after a change in speaker is extracted as text to be used in a summary document for the extracted text, and the summary document is generated and stored in the server 10 and the CRM system 30.

[0106] <Details of Summarization Process (Second Embodiment)> The processing from step S201 to step S204 is the same as the processing from step S101 to step S104 in the summarization processing (first embodiment), and therefore a description thereof will be omitted.

[0107] In step S205, the importance calculation unit 1045 of the server 10 performs importance calculation processing on the acquired speech recognition information. Any key sentence extraction algorithm may be applied to calculate the importance, but in this disclosure, an importance calculation method using an algorithm called LexRank, similar to the summarization processing (second embodiment), will be described as an example.

[0108] <Importance Calculation Process (Third Example)> The importance calculation unit 1045 of the server 10 treats the text included in the acquired speech recognition information as a single sentence and applies LexRank. Figure 22 is a diagram showing an overview of the additional extraction process in the summarization process (second embodiment), in which the text for each speaker is arranged downward in order of the time of speech. In the example of Figure 22, the importance is calculated for all of the texts U1 to U8 and C1 to C6.

[0109] In step S206, the summarizing unit 1046 of the server 10 extracts text whose importance is calculated to be equal to or greater than a predetermined value. The summarizing unit 1046 of the server 10 additionally extracts text that is spoken by a different speaker than the extracted text and that occurs before or after the extracted text in terms of spoken time. For example, when text U5 is extracted, text C4, which is spoken by a different speaker and before or after the time of speech, is additionally extracted.

[0110] In the present disclosure, one piece of text spoken by a different speaker and before and after the time of speech is additionally extracted, but two or more pieces of text may be additionally extracted. A number of pieces of text may be additionally extracted according to the importance value of the extracted text. For example, the greater the importance, the more pieces of text spoken by different speakers and before and after the time of speech may be additionally extracted. In a conversation between three or more people, multiple pieces of text before and after a change in speaker information may be additionally extracted.

[0111] The process of step S207 is the same as the process of step S107 in the summarization process (first embodiment), and therefore a description thereof will be omitted.

[0112] <Modification of Summarization Process> In the summarization process (first embodiment) and the summarization process (second embodiment), the importance calculation unit 1045 of the server 10 may calculate the importance of the text by excluding information that is meaningless in understanding the call between the user and the customer, such as filler contained in the text. Similarly, during the summarization process (first embodiment) and the summarization process (second embodiment), information that is meaningless in understanding the conversation between the user and the customer, such as filler contained in the text, may be removed from the text in advance, and the summary information may be stored in the summary table 1016.

[0113] <Summary display processing> The summary display process is a process for displaying summary information to the user as a summary document. Details of the summary display process will be explained below using the example screen of FIG.

[0114] <Summary display process overview> The summary display process is a series of processes that retrieves the summary information stored in the summary table 1016, stores the text contained in the summary information in speech bubbles of different colors for each speaker, arranges them in order of speaking time, and displays them as a summary document on the display 2081 of the user terminal 20.

[0115] <Details of summary display process> A user logs in to the server 10 and performs a predetermined operation to send a request to the server 10 to display a list of past call histories between the user or a colleague of the organization to which the user belongs and a customer. Upon receiving the request, the server 10 consults the call table 1014 and transmits records of the past call histories to the user terminal 20. The user terminal 20 displays a list of the received records of the past call histories on the display 2081 of the user terminal 20.

[0116] The user operates the user terminal 20 to select the call history record for which they wish to check summary information, and by pressing a summary display button or the like, sends a request to the server 10 for a summary display including the call ID for the selected call history. When the server 10 receives the request, it searches the summary table 1016 based on the call ID and transmits summary information related to the call ID to the user terminal 20 .

[0117] The user terminal 20 displays the received summary information as a summary document on the display 2081. The summary document displayed on the user terminal 20 will be described with reference to FIG. 23. The display 2081 of the user terminal 20 displays a summary display screen 70. The summary display screen displays the user's utterance content (text) together with the utterance time 701 in a speech bubble 702, and the customer's utterance content (text) together with the utterance time 711 in a speech bubble 712. The speech bubble 702 displaying the user's utterance content faces leftward and is displayed so as to be distinguishable from the speech bubble 712 displaying the customer's utterance content, which faces rightward. Based on the information on the text, speech time, and speaker contained in the acquired summary information, the display control unit 1052 of the user terminal 20 displays a list of texts in order of speech time from the top of the summary display screen 70 in a manner that allows the texts in which the speaker is the user to be distinguished in a bubble 702 that displays the user's speech content, and texts in which the speaker is the customer to be distinguished in a bubble 712 that displays the customer's speech content.

[0118] The color of the speech bubble 702 displaying the user's utterance content is different from the color of the speech bubble 712 displaying the customer's utterance content. This allows the user to visually confirm the speaker of the utterance content when checking the summary document. Furthermore, the colors of the speech bubbles 702 and 712 displaying the utterance content may be displayed by changing at least one of the brightness, hue, saturation, and intensity based on the importance of the text included in the utterance content calculated in the summarization process (first embodiment) and the summarization process (second embodiment). For example, the color of the speech bubbles 702 and 712 containing text of higher importance may be made darker. This allows the user to visually confirm the importance of each utterance content when checking the summary document, and to understand the text content in a short time.

[0119] <Voice recognition display processing> The voice recognition display process is a process for displaying voice recognition information to the user as a voice recognition document.

[0120] <Outline of voice recognition display processing> The speech recognition display process is a series of processes that acquires speech recognition information stored in the speech recognition table 1015, stores the text contained in the speech recognition information in speech bubbles of different colors for each speaker, arranges the text in order of the time of speech, and displays it on the display 2081 of the user terminal 20 as a speech recognition document.

[0121] <Details of voice recognition display processing> The speech recognition display process is the same as the summary display process except that speech recognition information is used instead of summary information, i.e., speech recognition table 1015 is used instead of summary table 1016, and therefore a description thereof will be omitted. In the speech recognition display process, speech recognition information is displayed in a list in speech bubble format in a manner that allows distinction for each speaker on a speech recognition display screen that is substantially the same as that shown in Fig. 23. In the speech recognition display process, the color of the speech bubble displaying the user's utterance is different from the color of the speech bubble displaying the customer's utterance. This allows the user to visually identify the speaker of the utterance when reviewing the speech-recognized document. Furthermore, the color of the speech bubble displaying the utterance may be displayed by changing at least one of the brightness, hue, saturation, and intensity based on the importance of the text contained in the utterance calculated in the summarization process (first embodiment) or the summarization process (second embodiment). For example, the color of a speech bubble containing text of higher importance may be darker. This allows the user to visually confirm the importance of each utterance when reviewing the speech-recognized document, thereby enabling the user to understand the content of the text in a short time. In other words, even when the user reviews all of the speech recognition information of a call between the user and a customer, the user can visually and intuitively grasp only the important utterance content.

[0122] <Response memo assignment process> The response memo attachment process is a process for automatically attaching a response memo to a call made between a user and a customer. Details of the response memo attachment process will be described below with reference to the flowchart in FIG.

[0123] <Outline of response memo assignment process> The response note assignment process is a series of processes that acquires voice data related to a call between a user and a customer, infers response note candidates by applying a learning model to the voice data, and stores the inferred response note candidates by linking them to the call data related to the call.

[0124] <Details of the response note assignment process> In step S301, the response memo attachment unit 1049 of the server 10 detects the end of the call between the user and the customer, and starts the response memo attachment process.

[0125] In step S302, the response memo attachment unit 1049 of the server 10 acquires call attributes of the call to be processed. Specifically, the response memo attachment unit 1049 of the server 10 searches the call table 1014 based on the call ID of the call to be processed, and acquires the call category and the call reception / transmission type. The response memo attachment unit 1049 of the server 10 searches the user table 1012 based on the user ID of the call to be processed and acquires the user attributes. The response memo attachment unit 1049 of the server 10 searches the organization table 1013 from the organization ID stored in the user table 1012 based on the user ID of the call and acquires the organization name and organization attributes of the organization to which the user belongs. The response memo attachment unit 1049 of the server 10 queries the CRM system 30 based on the customer ID of the call to be processed, and acquires the customer attributes, customer organization name, and customer organization attributes from the customer table 3012 of the CRM system 30. The response memo adding unit 1049 of the server 10 does not need to acquire all call attributes, but only needs to acquire at least one call attribute from among a plurality of call attributes as needed.

[0126] In step S303, the response memo adding unit 1049 of the server 10 selects a learning model based on the acquired call attributes. A learning model may be prepared for each call attribute, or may be prepared for each combination of multiple call attributes, for example, for each combination of user attributes and customer attributes. The learning model includes any machine learning or deep learning model, and is trained using a dataset created according to call attributes. Details of the learning process will be described later. In this disclosure, a deep learning model will be described as an example of a learning model. The deep learning model may be any deep learning model that uses any time-series data as input data, such as a recurrent neural network (RNN), a long short-term memory (LSTM), or a gated recurrent unit (GRU). The learning model may be any deep learning model, including, for example, attention and transformer.

[0127] In step S304, the response memo attachment unit 1049 of the server 10 acquires the voice data of the call to be processed and applies the voice data as input data for the selected learning model, thereby outputting (inferring) multiple response memo candidates (a group of response memos) as output data along with a probability distribution. Specifically, for response notes such as "AAA," "BBB," and "CCC," probability distributions such as "0.6," "0.3," and "0.1" are output, respectively. The probability distribution may be normalized using a softmax function or the like, or it does not have to be normalized.

[0128] In step S305, the response memo assigning unit 1049 of the server 10 associates the most likely response memo candidate from the output response memo candidates with the call ID of the call to be processed and stores it in the memo content field of the response memo table 1017. The current date and time may be stored in the assigned date and time field. The assigner ID field may store information indicating that the memo was automatically assigned by a system other than the user, such as the server 10. The response memo assigning unit 1049 of the server 10 may store, among the output response memo candidates, a plurality of response memo candidates with a probability equal to or greater than a predetermined value in the memo content field of the response memo table 1017.

[0129] <Response Memo Addition Process (Variation)> In step S303, the response memo assigning unit 1049 of the server 10 may select multiple learning models based on multiple different call attributes. For example, the response memo assigning unit 1049 of the server 10 may select a first learning model prepared for each user attribute and a second learning model prepared for each customer attribute. In this case, the response memo assigning unit 1049 of the server 10 may select multiple learning models based on any multiple call attributes.

[0130] In step S304, the response memo attachment unit 1049 of the server 10 acquires the voice data of the call to be processed and applies the voice data as input data for the selected multiple learning models, thereby outputting (inferring) multiple response memo candidates as output data along with their probability distributions for each applied learning model. The response memo attachment unit 1049 of the server 10 may calculate the probability distribution for each response memo candidate by applying any operation to the probability distribution of multiple response memo candidates. For example, the probability distribution for each response memo candidate to be output may be calculated by taking the sum or product of the probability distributions of multiple response memo candidates. Specifically, for the response notes "AAA," "BBB," "CCC," etc., if the inferred results of the probability distribution for the first learning model are "0.6," "0.3," and "0.1," and the inferred results of the probability distribution for the second learning model are "0.3," "0.4," and "0.1," then the products of these, "0.18," "0.12," and "0.01," may be used as the probability distributions for the response notes "AAA," "BBB," and "CCC." The probability distribution may or may not be normalized using a softmax function or the like.

[0131] <Response memo suggestion process> The response memo suggestion process is a process for suggesting response memo candidates to the user during a call between the user and a customer. Details of the response memo adding process will be described below with reference to the flowchart in FIG.

[0132] <Outline of response memo suggestion process> The response note suggestion process is a series of processes that acquires voice data related to a call between a user and a customer, infers response note candidates by applying a learning model to the voice data, suggests the inferred response note candidates to the user, and stores the response note candidates selected by the user in association with the call data related to the call.

[0133] <Details of the response memo suggestion process> The processing from step S401 to step S404 is the same as the processing from step S301 to step S304 of the response memo attachment processing, and therefore a description thereof will be omitted.

[0134] In step S405, the response note proposing unit 1048 of the server 10 transmits the output response note candidates and probability distribution to the user terminal 20. The display 2081 of the user terminal 20 displays a list of the received response note candidates so that the user can select one. At this time, the user terminal 20 may use the probability of each response memo candidate as the priority of the response memo candidate, and display a response memo candidate with a higher priority in a position that makes it easier for the user to select it on the display 2081 of the user terminal 20. For example, a response memo candidate with a higher priority may be displayed at the top of the list, or if multiple response memo candidates are displayed in a carousel or by page forwarding, it may be displayed on the first page. That is, the user terminal 20 displays highly probable response memo candidates in a position on the display 2081 of the user terminal 20 that makes it easy for the user to select them. This allows the user to more accurately and easily select a more probable response memo from multiple response memos.

[0135] In step S406, the user selects one or more response memo candidates from the response memo candidates displayed on the display 2081 of the user terminal 20. The user presses the send button displayed on the display 2081 of the user terminal 20 to send the selected response memo candidates and the user ID 2011 to the server 10.

[0136] In step S407, the response memo assigning unit 1049 of the server 10 associates the received response memo candidate and user ID 2011 with the call ID of the call to be processed and stores them in the memo content and assigner ID fields of the response memo table 1017. The current date and time may be stored in the assigned date and time field.

[0137] <Learning process> The learning process is a process for constructing a learning model to be used in the response memo assignment process, response memo suggestion process, and the like.

[0138] <Learning process overview> The learning process is a process for learning the parameters of a learning model for each call attribute related to the call, using voice data related to the call between the user and the customer as input data and the memo contents linked to the voice data as output data (output vector).

[0139] <Learning process details> The learning unit 1047 of the server 10 searches the call table 1014 and acquires the voice data and the memo content linked to the voice data by referring to the response memo table 1017 via the call ID. The learning unit 1047 of the server 10 divides the voice data into call attributes and creates data sets such as training data, test data, and verification data for each call attribute. The learning unit 1047 of the server 10 uses a data set corresponding to each call attribute for the learning model prepared for each call attribute to learn the parameters of the learning model.

[0140] The learning unit 1047 of the server 10 may exclude voice data linked to a predetermined call memo. Specifically, when creating a dataset, the learning unit 1047 of the server 10 excludes voice data related to calls with memo content indicating that the call with the customer was not actually completed, such as "answering machine," "customer (person in charge) absent," or "customer reception blocked." In this way, by excluding data that is undesirable for creating a learning model that infers the memo content from the dataset, a more accurate learning model can be created.

[0141] As an example, the learning unit 1047 of the server 10 may create a data set for each piece of information about one of the user attributes of the user making the call, such as the occupation type, the industry of the organization to which the user belongs, or the name of the organization to which the user belongs. As an example, the learning unit 1047 of the server 10 may create a data set for each piece of information on one of the customer attributes, such as the occupation of the customer making the call, the industry of the organization to which the customer belongs, or the name of the organization to which the customer belongs. As an example, the learning unit 1047 of the server 10 may create a data set for each piece of information about the call category of the calls made, such as telephone operator, telemarketing, customer support, technical support, and the like. As an example, the learning unit 1047 of the server 10 may create a data set for each piece of information relating to either outbound communications sent from a user to a customer or inbound communications received by a user from a customer.

[0142] When training a learning model for each user attribute of a user making a call, the learning unit 1047 of the server 10 may perform the learning process using input data that has been processed to extract only the user's voice (excluding the customer's voice). In this case, the voice data to be applied to the learning model in the inference process in steps S304 and S404 of the response memo assignment process and the response memo suggestion process is also processed using input data that has been processed to extract only the user's voice (excluding the customer's voice). A learning model according to user attributes can be obtained that can more accurately infer conversation notes by learning based only on the content of the user's utterances.

[0143] When training a learning model for each customer attribute of a customer who makes a call, the learning unit 1047 of the server 10 may perform the learning process using input data that has been processed to extract only the customer's voice (excluding the user's voice). In this case, the voice data to be applied to the learning model in the inference process in steps S304 and S404 of the response memo assignment process and the response memo suggestion process is also processed using input data that has been processed to extract only the customer's voice (excluding the user's voice). By training a learning model based on customer attributes solely on the content of customer utterances, it is possible to obtain a learning model that can more accurately infer conversation notes.

[0144] <Modification of the learning process> In the present disclosure, voice data is used as input data for the learning model, but data converted by performing arbitrary information processing on the voice data may also be used as input data. For example, instead of voice data, text data obtained by performing text recognition on the voice data (transcription) may be used as input data. Furthermore, data converted by performing arbitrary information processing on text data may also be used as input data. In the present disclosure, applying voice data to a learning model also includes applying the learning model to data converted by performing arbitrary information processing on certain voice data. In this case, when inferring response memo candidates in a learning model for the response memo assignment process, response memo suggestion process, etc., it is sufficient to apply the learning model by performing similar information processing on the voice data that is the input data. For example, when inferring response memo candidates in a learning model for the response memo assignment process, response memo suggestion process, etc., it is sufficient to use text data obtained by performing text recognition on the voice data (transcription) instead of the voice data as input data.

[0145] <Score calculation process> The score calculation process is a process for calculating a call score for each user. Details of the score calculation process will be described below with reference to the flowchart in FIG.

[0146] <Outline of score calculation process> The score calculation process calculates a call score for each user by performing analytical and statistical processing on the call data for each user, which allows, for example, the customer service skills of each user to be evaluated using a quantitative index.

[0147] <Details of score calculation process> In step S501, the user operates the user terminal 20 to send a request for a user list to the server 10. Based on the received request, the score calculation unit 1050 of the server 10 obtains a list of users who belong to the same organization (have the same organization ID) as the user who sent the request from the user table 1012, and sends the list to the user terminal 20. The display 2081 of the user terminal 20 displays the acquired list of users in a list that the user can select from. The user selects a user for whom they wish to calculate a score from the list of users displayed on the display 2081 of the user terminal 20. The user sends the user ID of the selected user to the server 10 by pressing a send button displayed on the display 2081 of the user terminal 20.

[0148] In step S502, the score calculation unit 1050 of the server 10 searches the call table based on the received user ID, and acquires analysis data for each call of the target user.

[0149] At this time, the score calculation unit 1050 of the server 10 may exclude analysis data linked to a predetermined response memo. Specifically, when acquiring analysis data for each call of the target user, the score calculation unit 1050 of the server 10 acquires the memo content of the response memo table 1017 linked by the call ID of the call. The score calculation unit 1050 of the server 10 excludes analysis data related to calls to which memo content indicating that the call with the customer was not actually completed, such as "answering machine," "customer (person in charge) absent," or "customer reception blocked," is attached. In this way, when calculating the call score of the target user, calls that were not actually completed can be excluded, thereby making it possible to calculate a call score with higher accuracy.

[0150] The score calculation unit 1050 of the server 10 calculates a call score by applying a predetermined algorithm to the analysis data for each call. Specifically, the score calculation unit 1050 of the server 10 calculates the call score by taking a predetermined weighted sum based on the degree of deviation of various index values ​​(number of silences, number of overlaps, number of rallies, etc.) included in the analysis data from reference index values, etc. At this time, any processing such as normalization may be applied to the index values.

[0151] In step S504, the score calculation unit 1050 of the server 10 calculates a statistical value (average, median, mode, maximum, minimum, etc.) as the user evaluation index of the target user by performing statistical processing on the call score calculated based on the analysis data for each call of the target user. Specifically, the average of the call scores calculated based on the analysis data for each call of the target user is suitable as the user evaluation index.

[0152] The score calculation unit 1050 of the server 10 stores the calculated user evaluation index in the evaluation index item of the record of the target user in the user table 1012. The score calculation unit 1050 of the server 10 transmits the calculated user evaluation index to the user terminal 20 of the user who executed the score calculation process. The display 2081 of the user terminal 20 displays the received user evaluation index of the target user to the user.

[0153] <Linkage Mode Setting Process> The linkage mode setting process is a process for setting the storage process of call information to the CRM system 30 in the call storage process.

[0154] <Overview of Linkage Mode Setting Process> A user, or an administrator of an organization or department to which the user belongs, etc. opens a predetermined website provided by the server 10 and displays an editing screen for linkage mode settings. The user can set the storage process of call information to the CRM system 30 in the server 10 by selecting a desired linkage mode on the linkage mode setting screen and performing a save operation or the like. Note that the linkage mode setting may have a predetermined default value without the user setting it. It may also be set when the user first uses the voice call service according to the present disclosure.

[0155] <Details of Linkage Mode Setting Process> A user, or an administrator of an organization or department to which the user belongs, etc. (user etc.) operates their own user terminal 20 and opens a website related to the editing screen for linkage mode settings provided by the server 10 using a web browser or the like. The user etc. operates the user terminal 20 and selects either the first mode or the second mode as the desired linkage mode on the linkage mode setting screen. The user etc. operates the user terminal 20 and transmits their user ID 2011 and the selected linkage mode to the server 10. The setting unit 1042 of the server 10 searches the user table 1012 based on the received user ID 2011 and stores the received linkage mode in the linkage mode item of the record of the user. Note that the linkage mode may be stored for each organization to which the user belongs rather than for each user. That is, it may be stored in the linkage mode item provided in the organization table 1013, and the linkage mode of each user may refer to the linkage mode item of the organization table 1013 linked by the organization ID.

[0156] <CRM Storage Process> The CRM memory process is a process for storing information related to a call made between a user and a customer in the CRM system 30 using the voice call service according to the present disclosure. Hereinafter, the details of the CRM memory process will be described using the flowchart of FIG. 19.

[0157] <Overview of CRM Memory Process> The CRM memory process is a series of processes that, when a call is started between a user and a customer, obtains the linkage mode set for each user or organization, and stores data related to the call in the CRM system 30 according to the set value of the linkage mode. As a result, the data related to the call is stored in the CRM system 30 in association with the customer information of the call target.

[0158] <Details of CRM Memory Process> In step S601, after the outgoing call process or the incoming call process (incoming / outgoing call process) is performed, a call is started between the user and the customer. As a result, the server 10 detects that a call has been started between the user and the customer.

[0159] In step S602, the CRM memory control unit 1051 of the server 10 searches the user table 1012 based on the user ID 2011 received from the user terminal 20, and obtains the linkage mode of the user making the call. When the linkage mode is stored in the organization table 1013 or the like for each organization, the organization table 1013 or the like is searched based on the organization ID of the corresponding user, and the linkage mode associated with the corresponding user is set as the linkage mode of the user.

[0160] In step S603, the CRM memory control unit 1051 of the server 10 determines whether the obtained linkage mode is the first mode or the second mode. If the obtained linkage mode is the first mode, the CRM memory control unit 1051 of the server 10 proceeds to step S604. If the obtained linkage mode is the second mode, the CRM memory control unit 1051 of the server 10 skips step S604 and proceeds to step S605.

[0161] In step S604, the CRM storage control unit 1051 of the server 10 transmits a request to store the first call data and the call data including the call ID in the call table 1014 of the call in association with the customer ID to the CRM system 30. The first call data is data related to the call that can be obtained after the call starts and before the call ends. Specifically, the CRM system 30 associates the user ID of the user making the call, the customer ID of the customer, the outgoing date and time (only in the case of outgoing calls), and the start date and time of the call as first call data, and stores these in the items of user ID, customer ID, dialing date and time, and call start date and time in the response history table 3013. Note that the first call data may be configured to include at least one of the user ID, the customer ID of the customer, the dialing date and time, and the call start date and time.

[0162] In step S605, the CRM storage control unit 1051 of the server 10 detects the end of the call between the user and the customer.

[0163] In step S606, the CRM storage control unit 1051 of the server 10 determines whether the voice analysis process for the call has been completed. Specifically, the CRM storage control unit 1051 of the server 10 refers to the call table 1014 and determines whether the analysis data item in the record of the call contains blanks, nulls, or other information indicating that the voice analysis process has not been completed. If the voice analysis process is not complete, the CRM storage control unit 1051 of the server 10 waits in step S606. If the voice analysis process is complete, the CRM storage control unit 1051 of the server 10 proceeds to step S607.

[0164] In step S607, the CRM storage control unit 1051 of the server 10 determines whether the acquired linkage mode is the first mode or the second mode. If the acquired linkage mode is the first mode, the CRM storage control unit 1051 of the server 10 proceeds to step S609. If the acquired linkage mode is the second mode, the CRM storage control unit 1051 of the server 10 proceeds to step S608.

[0165] In step S608, the CRM storage control unit 1051 of the server 10 transmits a request to store the second call data and the call data including the call ID in the call table 1014 of the call in association with the customer ID to the CRM system 30. The second call data is data related to the call that can be obtained after the call ends. The CRM system 30 associates the call ID with the user ID of the user making the call, the customer ID of the customer, the outgoing date and time (only in the case of outgoing calls), the start date and time of the call, and the end date and time of the call as second call data and stores them in the fields of user ID, customer ID, dialing date and time, call start date and time, and call end date and time in the response history table 3013, respectively. The CRM system 30 associates the call ID with the voice recognition information and summary information related to the call and stores them in the field of comment in the response history table 3013. Specifically, like the voice recognition result and summary result of comment 807 in FIG. 24, the CRM system 30 converts the voice recognition result and summary result into text-format information that combines the utterance time, speaker, and text, and stores the converted text in the field of comment in the response history table 3013. The CRM system 30 associates the voice recognition result and summary result of comment 807 in FIG. 24 with the voice recognition result and summary result, and stores the converted text-format information in the field of comment in the response history table 3013. The CRM system 30 associates the call ID with the URL and stores a URL generated based on the call ID in the field of URL in the response history table 3013. Note that the response history table 3013 may be configured to store only a portion of the information of the second call data.

[0166] In step S609, the CRM storage control unit 1051 of the server 10 sends a request to the CRM system 30 to store the third call data and the call data including the call ID of the call in the call table 1014 in association with the customer ID. The third call data is data related to the call that can be obtained after the call ends, excluding the data included in the first call data. The CRM system 30 associates the call ID with the end date and time of the call and stores the end date and time of the call as second call data in the call end date and time field of the response history table 3013. The CRM system 30 associates the call ID with the voice recognition information related to the call and summary information related to the call and adds it to the comment field of the response history table 3013. Specifically, like the voice recognition result and summary result of comment 807 in Figure 24, the CRM system 30 converts it into text-format information that combines the speech time, speaker, and text and adds it to the comment field of the response history table 3013. At this time, if the collaboration mode is the first mode, the CRM storage control unit 1051 of the server 10 adds to the comment of the record so as not to overwrite the content already written in the comment, since a record has already been created in the response history table 3013 in step S604. Specifically, if the collaboration mode is the first mode, a new record of the call is created in the response history table 3013, and other users may add to the comment. In such cases, the CRM storage control unit 1051 of the server 10 also adds to the comment of the record so as not to overwrite it. The CRM system 30 associates the URL generated based on the call ID with the call ID and stores it in the URL field of the response history table 3013. Note that the response history table 3013 may be configured to store only part of the information of the third call data.

[0167] <Modification> The server 10 can be configured to store customer information itself by acquiring customer information from the customer table 3012 of the CRM system 30 and storing it in a database (not shown). In this case, the CRM storage control unit 1051 of the server 10 may store the call data in association with the customer who was the target of the call by sending a request including a customer ID, the customer's name, the name of the customer organization, etc., instead of the call ID, to the CRM system 30. In this case, the data related to the call is also stored in the CRM system 30 in association with the customer information of the target of the call.

[0168] In the present disclosure, the server 10 also provides services related to calls between the user and the customer, but the call service may be provided by an external service (not shown). In this case, the CRM storage control unit 1051 of the server 10 may detect the start of a call between the user and the customer upon receiving a request for starting a call between the user and the customer provided by the external service, and may start CRM storage processing. Similarly, the end of a call between the user and the customer may be detected upon receiving a request for ending a call between the user and the customer provided by the external service. Alternatively, the end of a call between the user and the customer may be detected based on the voice data exchanged between the user and the customer, for example, when the voice data is lost.

[0169] <Call display processing> The call display process is a process for displaying call data stored in the CRM system 30 to the user. Details of the call display process will be explained below using the flowchart in Figure 20. Figure 24 is a diagram showing an example of a screen output by the CRM system 30 in the call display process.

[0170] <Outline of call display processing> After the call with the customer ends, the user operates the user terminal 20 to display the response history with the customer stored in the CRM system 30. The user operates the user terminal 20 to select the response history from the response history with the customer for which analysis data is to be displayed, and requests the display of the analysis data. The CRM system 30 then obtains the analysis data from the server 10 and displays the customer information and analysis data on the display 2081 of the user terminal 20.

[0171] <Call display process details> In step S701, the user operates the user terminal 20 to open a predetermined website provided by the CRM system 30, and displays the response history display screen. Specifically, the user terminal 20 sends a request for displaying a list of customer information to the CRM system 30. The CRM system 30 searches the customer table 3012 and sends the records to the user terminal 20. The display 2081 of the user terminal 20 displays the received customer information in a list so that the user can select it. The user selects the customer for whom they wish to display the response history from the customer information listed on the display 2081 of the user terminal 20, and sends the customer ID of the selected customer to the CRM system 30 by pressing the "Send" button. Upon receiving the customer ID, the CRM system 30 searches the response history table 3013 and sends the response history record for the selected customer to the user terminal 20. The display 2081 of the user terminal 20 displays the received response history in a list that the user can select.

[0172] In step S702, the user selects the record for which they wish to display the response history from the response history list displayed on the display 2081 of the user terminal 20, and presses the "Send" button to send the response history ID of the selected response history to the CRM system 30. The CRM system 30 generates a display screen for the selected response history information and sends it to the user terminal 20. The display 2081 of the user terminal 20 displays the display screen for the received response history information. The display screen for the response history information displays the URL, customer information, user information, dialing date and time, call start date and time, call end date and time, speech recognition information and summary information stored in the comment by the CRM storage process, and the like, all of which are stored in the response history table 3013. In addition, without performing the processing of steps S701 and S702, for example, after the call between the user and the customer ends, the user terminal 20 may set the response history related to the call as the selected response history and send the response history ID to the CRM system 30.

[0173] In step S703, the user operates the user terminal 20 and presses the "Analysis Results" button 801 displayed on the display screen of the response history information. The user terminal 20 sends a request to the CRM system 30 to display the analysis results including the response history ID related to the call. The CRM system 30 searches the response history table 3013 based on the received response history ID and identifies the call ID. The CRM system 30 sends a request requesting analysis data including the identified call ID to the server 10. The server 10 searches the call table 1014 based on the received call ID and checks whether analysis data exists. If analysis data does not exist, the system waits in step S703. If analysis data exists, the process proceeds to step S704.

[0174] In step S704, the server 10 searches the call table 1014 based on the received call ID and transmits the analysis data to the CRM system 30. The CRM system 30 generates an analysis result screen 808 that visualizes the analysis data based on the received analysis data and transmits it to the user terminal 20. The analysis result screen 808 includes the voice analysis result 802, response evaluation 804, voice evaluation 805, and speaking speed 806 shown in FIG. 24. The analysis result screen 808 may also include a comment 807, which is a text-format voice recognition result and summary result stored in the comment field of the response history table 3013. The user can play the call audio by pressing the play / stop button 803. The CRM system 30 stores information about customers such as a customer table 3012, and the user can selectively switch between the information about the customer stored in the CRM system 30 and an analysis result screen that visualizes the analysis data, or display them side by side, thereby quickly understanding the customer interaction history in more detail. In other words, by operating the CRM system 30, the user can manage customer information and also check the visualization results of the analysis data.

[0175] <Modification> In the present disclosure, the voice server (PBX) 40 and the customer terminal 50 are described as being connected via a telephone network T, but this is not particularly limited. In other words, the voice server (PBX) 40 and the customer terminal 50 may be connected via any communication means including, for example, the Internet.

[0176] In this case, in the outgoing call process, the call may be made (a call made) to the customer terminal 50 based on any information, such as customer identification information or a URL, for identifying the customer, rather than the customer's telephone number. Similarly, in the incoming call process, the incoming call request may include any information, such as customer identification information or a URL, for identifying the customer, and the incoming call (received call) may be identified based on the information, such as the customer identification information or the URL.

[0177] <Basic computer hardware configuration> 25 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main memory device 902, an auxiliary memory device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0178] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0179] The main memory device 902 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0180] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0181] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards. A network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, networks include 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, networks also include those that are directly connected using a USB (Universal Serial Bus) cable, etc.

[0182] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0183] <Basic functional configuration of computer 90> The following describes the functional configuration of a computer realized by the basic hardware configuration (FIG. 25) of the computer 90. The computer includes at least the functional units of a control unit, a storage unit, and a communication unit.

[0184] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0185] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0186] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0187] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs.

[0188] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0189] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0190] (Appendix 1) A program having a processor and a memory unit for causing a computer to manage data related to calls made between a user and a customer, the program causing the processor to execute the steps of accepting voice data related to the call, acquiring call attributes of the call (S302, S402), selecting a learning model based on the call attributes (S303, S403), and inferring a response memo for the call by applying the voice data to the learning model selected in the selection step (S304, S404). This allows the computer that manages data related to calls between the user and the customer to infer more appropriate response notes for the calls between the user and the customer, eliminating the need for the user to go through the tedious process of checking the response notes.

[0191] (Appendix 2) 2. The program of claim 1, wherein the call attributes include at least one of user attributes, customer attributes, call categories, and call types. This allows the user to infer more appropriate response notes for calls between the user and a customer by using a learning model based on call attributes.

[0192] (Appendix 3) The program described in Appendix 2, wherein the user attributes include information regarding one of the attributes of the user making the call, the occupation of the user, the industry of the organization to which the user belongs, and the name of the organization to which the user belongs; the customer attributes include information regarding one of the attributes of the customer making the call, the occupation of the organization to which the user belongs, the industry of the organization to which the user belongs, and the name of the organization to which the customer belongs; the call category includes information regarding the type of call made between the user and the customer or the purpose of the call; and the call type includes information regarding either an outbound call made from the user to the customer or an inbound call received from the customer by the user. This allows the user to infer more appropriate response notes for calls between the user and a customer by using a learning model based on call attributes.

[0193] (Appendix 4) A program described in any one of Appendices 1 to 3, wherein the call attributes include user attributes and customer attributes, and the selection step includes steps (S303, S403) of selecting a learning model based on the user attributes and customer attributes. This allows the user to infer more appropriate response notes by applying a learning model that is more suited to the call for which the response notes are to be inferred, based on both user attributes and customer attributes.

[0194] (Appendix 5) The call attributes include a first attribute and a second attribute, the selection step includes steps (S303, S403) of selecting a first learning model based on the first attribute, and steps (S303, S403) of selecting a second learning model based on the second attribute, and the inference step includes a first inference step (S304, S404) of inferring a first group of response notes consisting of multiple response notes from the voice data by applying voice data related to the call to the first learning model, a second inference step (S304, S404) of inferring a second group of response notes consisting of multiple response notes from the voice data by applying the voice data related to the call to a second learning model, and steps (S304, S404) of inferring the response notes based on the first group of response notes and the second group of response notes. This means that even if there is not enough voice data (datasets used for learning) for each call attribute, more appropriate call notes can be inferred by combining and applying multiple learning models.

[0195] (Appendix 6) 6. The program of claim 5, wherein the first attribute is a user attribute and the second attribute is a customer attribute. This allows users to infer more appropriate response notes by combining and applying learning models for user attributes and customer attributes, even if there is not enough voice data for each user attribute and customer attribute.

[0196] (Appendix 7) The program described in Appendix 6, wherein the first learning model is trained based on the voice data relating to the user's call portion of the voice data, and is not trained based on the voice data relating to the customer's call portion of the voice data, and the second learning model is trained based on the voice data relating to the customer's call portion of the voice data, and is not trained based on the voice data relating to the user's call portion of the voice data. This allows the user to infer more appropriate response notes based on both user attributes and customer attributes.

[0197] (Appendix 8) The inference step is a step (S405) of inferring a group of response notes consisting of multiple response notes, and the program causes a processor to execute a display step (S406) of displaying the multiple response note groups so that they can be selected by the user, and a storage step (S407) of storing one or more response notes selected by the user from the group of response notes in association with voice data. This allows the user to select a response memo to be added to the voice data from the plurality of inferred response memos, thereby enabling the user to add a response memo to the voice data without the need for complicated input work.

[0198] (Appendix 9) The program described in Appendix 8, wherein the inference step includes a step (S404) of calculating a priority for each of the response notes included in the group of response notes, and the display step includes a step (S405) of displaying a first response note included in the group of multiple response notes at a first position, and displaying a second response note included in the group of multiple response notes at a second position that is more difficult for the user to select than the first position, and wherein the first response note has a higher priority than the second response note. This allows a response memo with a higher priority to be displayed in a position that is easier for the user to select, allowing the user to assign a more likely response memo from a plurality of response memos to the voice data without any complicated work.

[0199] (Appendix 10) 10. The program according to any one of appendices 1 to 9, wherein the learning model is learned by excluding, from among the voice data, voice data to which predetermined response notes have been added. This allows us to infer more appropriate response notes by using a learning model that excludes calls where a conversation with a customer is not actually completed, such as "answering machine," "customer (person in charge) absent," and "customer reception blocked."

[0200] (Appendix 11) An information processing system comprising a processor and a memory unit for managing data related to calls made between a user and a customer, the information processing system causing the processor to execute the steps of accepting voice data related to the call, acquiring call attributes of the call (S302, S402), selecting a learning model based on the call attributes (S303, S403), and inferring a response memo for the call by applying the voice data to the learning model selected in the selection step (S304, S404). This allows the computer that manages data related to calls between the user and the customer to infer more appropriate response notes for the calls between the user and the customer, eliminating the need for the user to go through the tedious process of checking the response notes.

[0201] (Appendix 12) An information processing method comprising a processor and a memory unit for causing a computer to manage data relating to calls made between a user and a customer, the information processing method causing the processor to execute the steps of accepting voice data relating to the call, acquiring call attributes of the call (S302, S402), selecting a learning model based on the call attributes (S303, S403), and inferring a response memo for the call by applying the voice data to the learning model selected in the selection step (S304, S404). This allows the computer that manages data related to calls between the user and the customer to infer more appropriate response notes for the calls between the user and the customer, eliminating the need for the user to go through the tedious process of checking the response notes. [Explanation of symbols]

[0202] 1 Information processing system, 10 Server, 101 Memory unit, 103 Control unit, 20A, 20B, 20C User terminal, 201 Memory unit, 204 Control unit, 30 CRM system, 301 Memory unit, 304 Control unit, 50A, 50B, 50C Customer terminal, 501 Memory unit, 504 Control unit

Claims

1. A program for causing a computer to manage data related to calls made between a user and a customer, the program comprising: a processor; and a storage unit. The program causes the processor to: receiving voice data relating to the call; an input step of accepting input of information related to the call from a user; a selection step of selecting a learning model based on the information related to the call received in the input step; an inference step of inferring a response memo for the call by applying the speech data to the learning model selected in the selection step; a storage step of storing the response memo in association with information indicating that the response memo was added by a user or information indicating that the response memo was automatically added by a system other than the user; Execute The learning model is a learning model that has been trained to output a response memo using the voice data as input data. program.

2. The input step is a step of accepting input of information about the customer. The program according to claim 1.

3. The information related to the call includes call attribute information, which is at least one of user attributes, customer attributes, call categories, and call receiving / transmitting types. The program according to claim 1.

4. The user attribute includes information on one attribute of the user making the call, including the occupation of the user, the industry of the organization to which the user belongs, and the name of the organization to which the user belongs; The customer attributes include information on one of the attributes of the customer making the call, the occupation of the customer, the industry of the organization to which the customer belongs, and the name of the organization to which the customer belongs; The call category includes information regarding the type of call or purpose of the call made between the user and the customer; The type of call received or sent includes information on either an outbound call sent from the user to the customer or an inbound call received from the customer by the user. The program according to claim 3.

5. The information related to the call includes call attribute information related to user attributes and customer attributes, The selecting step includes a step of selecting the learning model based on the user attributes and the customer attributes.

5. The program according to claim 1.

6. the information related to the call includes call attribute information relating to a first attribute and a second attribute; The selecting step includes: selecting a first learning model based on a first attribute; selecting a second learning model based on a second attribute; Including, The inferring step a first inference step of inferring a first group of response notes from the voice data by applying the voice data to a first learning model; a second inference step of inferring a second group of response notes from the voice data by applying the voice data related to the call to a second learning model; Inferring the response memo based on the first response memo group and the second response memo group; Including, 6. The program according to claim 1.

7. The first attribute is a user attribute, The second attribute is a customer attribute. The program according to claim 6.

8. The first learning model is trained based on the voice data relating to the user's call portion of the voice data, and is not trained based on the voice data relating to the customer's call portion, The second learning model is trained based on the voice data relating to the customer's call portion of the voice data, and is not trained based on the voice data relating to the user's call portion. The program according to claim 7.

9. the inference step is a step of inferring a group of the response notes, the group being made up of a plurality of the response notes; The program causes the processor to: a display step of displaying the plurality of groups of response notes so that the group of response notes can be selected by a user; a storage step of storing one or more of the response notes selected by a user from the group of response notes in association with the voice data; The program according to any one of claims 1 to 8, which executes the above.

10. the inference step includes a step of calculating a priority for each of the response notes included in the response note group; The display step includes: Displaying a first response memo included in the plurality of response memo groups at a first position; a second response memo included in the plurality of response memo groups is displayed at a second position that is less likely to be selected by the user than the first position; including the steps The first response memo has a higher priority than the second response memo. The program according to claim 9.

11. The learning model is learned by excluding, from the voice data, the voice data to which the predetermined response memo is added. The program according to any one of claims 1 to 10.

12. An information processing system for managing data related to calls made between a user and a customer, the information processing system comprising a processor and a storage unit, the processor: receiving voice data relating to the call; acquiring call attributes of the call; a selection step of selecting a learning model based on the call attributes; an inference step of inferring a response memo for the call by applying the speech data to the learning model selected in the selection step; a storage step of storing the response memo in association with the voice data and information identifying a person who has provided the response memo; Run the information identifying the grantor includes information indicating that the grantor was automatically granted by a computer; The learning model is a learning model that has been trained to output a response memo using the voice data as input data. Information processing system.

13. An information processing method for managing data related to calls made between a user and a customer by a computer, the method comprising: the processor: receiving voice data relating to the call; acquiring call attributes of the call; a selection step of selecting a learning model based on the call attributes; an inference step of inferring a response memo for the call by applying the speech data to the learning model selected in the selection step; a storage step of storing the response memo in association with the voice data and information identifying a person who has provided the response memo; Run the information identifying the grantor includes information indicating that the grantor was automatically granted by a computer; The learning model is a learning model that has been trained to output a response memo using the voice data as input data. Information processing methods.

Images