Emotion scoring device, service system, and emotion scoring program

Abstract

To provide a technique capable of automatically scoring what kind of feeling a user has for an IT service.SOLUTION: An emotion scoring device according to the present invention acquires a user-specific log in which an operation of an information system when a user uses a service is recorded together with an identifier of the user, prepares a learner that has learned a correspondence relationship between the operation of the information system and a classification of an emotion of the user by machine learning, and acquires the classification of the emotion and scores the emotion by inputting the user-specific log to the learner.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for scoring the feelings that users of a service provided by an information system have toward that service. [Background technology]

[0002] In IT services, it is important to visualize the user experience (UX) and reflect it in service improvements. Typically, page views, clicks, and time spent on a service are measured, and then analytical tools are used to quantitatively evaluate the user experience based on these figures, and the results are used to consider directions for improvement. In this case, while it is possible to grasp general trends based on user behavior, it is difficult to predict how users feel about an IT service or how they will act based on those feelings.

[0003] The following Patent Document 1 addresses the issue of "establishing a behavioral index highly correlated with profitability and proposing a method for improving the behavioral index," and describes the following technology (see abstract): "A customer analysis device acquires responses from a user who has received financial services to one or more type 1 questions defined for investigating behavioral motivation for financial services, and calculates a behavioral index based on the one or more responses. Next, the customer analysis device acquires responses from the user who has received financial services to multiple type 2 questions defined for investigating customer sentiment toward the financial service, and calculates multiple types of emotional indexes based on the multiple responses. The customer analysis device calculates a correlation coefficient between the behavioral index and each of the multiple types of emotional indexes, and graphically displays the influence of each of the multiple emotional indexes on the behavioral index based on the correlation coefficients."

[0004] The following Patent Document 2 aims to "provide technology that appropriately supports the analysis of text," and describes the technology in which "an analysis device 10 includes a text acquisition unit 22 that acquires text obtained from human utterances or descriptions related to a living space, an emotional expression extraction unit 23 that extracts emotional expressions that express human emotions and factors of those emotions from the text acquired by the text acquisition unit 22, and an analysis unit 25 that acquires trends in human emotions and factors related to the living space, or relationships between human emotions and factors related to the living space, by analyzing the emotional expressions and factors extracted by the emotional expression extraction unit 23" (see abstract). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2021-140778 [Patent Document 2] Japanese Patent Publication No. 2022-046315 Summary of the Invention [Problem to be solved by the invention]

[0006] Conventional technologies for analyzing customer emotions, such as those described in Patent Document 1, are based on responses to questions (surveys). Surveys do not necessarily accurately reflect user emotions, and conducting a survey requires time and effort. Therefore, it would be desirable to be able to predict user emotions more automatically.

[0007] As in Patent Document 2, the technology for predicting emotions from text cannot be implemented unless the user inputs or provides the text. Since IT services do not necessarily assume that users will provide text describing their emotions toward the service, it is difficult to directly apply the technology in this document to emotion prediction for IT services.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that can automatically score how a user feels about an IT service. [Means for solving the problem]

[0009] The emotion scoring device of the present invention acquires a user-specific log that records the operation of an information system when a user uses a service together with the user's identifier, prepares a learning device that learns through machine learning the correspondence between the operation of the information system and the emotion classification of the user, and inputs the user-specific log into the learning device to acquire the emotion classification and score the emotion. [Effects of the Invention]

[0010] The emotion scoring device according to the present invention can automatically score the emotion a user has toward an IT service. Other objects, configurations, advantages, etc. of the present invention will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a configuration diagram of an emotion scoring device 1. [Figure 2] The structure of user data 221 and an example of the data are shown below. [Figure 3] The configuration and data example of the application log 222 are shown below. [Figure 4] The configuration of the infrastructure log 223 and an example of data are shown below. [Figure 5] The structure of the questionnaire log 224 and an example of data are shown below. [Figure 6] The configuration of the user log 125 and an example of data are shown below. [Figure 7] The structure of feature point data 121 and an example of the data are shown below. [Figure 8] The structure of the training data 122 and an example of the data are shown below. [Figure 9]10 shows the structure of emotion score data 123 and an example of the data. [Figure 10] The structure of upsell data 124 and an example of the data are shown below. [Figure 11] 1 is a flowchart illustrating the overall operation of emotion scoring device 1. [Figure 12] 11 is a flowchart illustrating details of S1101. [Figure 13] 11 is a flowchart illustrating details of step S1102. [Figure 14] 11 is a flowchart illustrating details of step S1103. [Figure 15] 11 is a flowchart illustrating details of step S1104. [Figure 16] 15 is a screen example showing an example of a report output by the arithmetic device 11 in S1504. DETAILED DESCRIPTION OF THE INVENTION

[0012] Figure 1 is a configuration diagram of a service system according to an embodiment of the present invention. The service system is a system that provides IT services to users, and is composed of an emotion scoring device 1 and an information system 2. The emotion scoring device 1 is a device that scores the emotions that users of IT services provided by the information system 2 have toward the services. The IT services referred to here are, for example, services in which the information system 2 provides software functions to users via a network (e.g., online shopping, subscription services, etc.).

[0013] The emotion scoring device 1 includes a calculation device 11, a storage device 12, a display device 13, and a learning device 14. The calculation device 11 scores the emotion a user has toward an IT service through processing described below. The display device 13 displays the results. The learning device 14 will be described later. The storage device 12 stores feature point data 121, training data 122, emotion score data 123, upselling time data 124, and a user-specific log 125. Examples of these data will be described later.

[0014] The information system 2 is connected to the emotion scoring device 1 via a network. The information system 2 includes a storage device 22. The storage device 22 stores user data 221, an application log 222, an infrastructure log 223, and a questionnaire log 224. Examples of these data will be described later.

[0015] 2 shows the structure and example data of user data 221. User data 221 is data that describes the attributes of users who use IT services provided by information system 2. User data 221 describes the user's name and gender, user ID, service contract start date and contract end date, contract plan, etc.

[0016] 3 shows the configuration and example data of the application log 222. The application log 222 is data that records the operation of applications executed by the information system 2 to provide IT services, and includes, for example, the following logs: (a) an access log that records user access to an application; (b) an operation log that records operations performed by a user on an application; and (c) an error log that records errors that occurred during the operation of an application. These logs are recorded together with the date and time the event occurred, the event type (log level), the name of the process on the application that caused the event, and so on. Of these logs, if an event occurred while a user was logged in to an application, the user's ID can also be recorded.

[0017] Figure 4 shows the configuration and example data of the infrastructure log 223. The infrastructure log 223 is data that records the operations of the OS (Operating System), middleware, etc. that the information system 2 executes to provide IT services. The operations of the OS and middleware are not necessarily directly linked to individual applications. Therefore, the infrastructure log 223 does not directly record the IDs of users associated with an event when that event occurs. This point will be discussed later.

[0018] 5 shows the configuration and example data of the questionnaire log 224. The questionnaire log 224 is data that records the results of responses to a questionnaire evaluating IT services provided by the information system 2. The evaluations can be recorded, for example, using a five-level numerical scale, and the evaluation results are recorded for each user ID that responded.

[0019] 6 shows the configuration and example data of the user log 125. The user log 125 is data that aggregates, by user ID, the records recorded in the application log 222, infrastructure log 223, and questionnaire log 224. Here, two users with user IDs "A0001" and "B0002" are shown as examples.

[0020] 6, the first and third lines are logs extracted from the application log 222, the second line is a log extracted from the infrastructure log 223, and the fourth line is a log extracted from the questionnaire log 224. The processing procedure for aggregating each log into the user log 125 will be described later.

[0021] FIG. 7 shows the structure of feature point data 121 and an example of the data. User log 125 records actions taken by users with respect to IT services. It is believed that the user's feelings toward the IT service are reflected in the action record as a result of the action. Therefore, it is believed that the user's feelings toward the IT service can be quantified by extracting feature amounts from the action record and linking them to the feelings that the feature amounts are believed to represent. Feature point data 121 is data that lists the types of such feature amounts. For convenience of description, the types of feature amounts are described using natural language character strings in FIG. 7, but the actual feature point data 121 may be in any format that allows the computing device 11 to understand the types of feature amounts.

[0022] As shown in Figure 7, at least one of the following types of features can be used: (a) the length of time that users use an IT service; (b) the variance of the length of time that users use an IT service; (c) the frequency with which users use an IT service; (d) the variance of the frequency with which users use an IT service; (e) the type of operations that users perform when using an IT service; (f) the number of operations that users perform when using an IT service; (g) the speed of response that users perform when using an IT service; (h) the accuracy of response that users perform when using an IT service; (i) the rate at which errors occur when users use an IT service; (j) the rate at which processing is interrupted when users use an IT service; and (k) the evaluation results of the IT service provided by users in a questionnaire.

[0023] FIG. 8 shows the structure of the training data 122 and an example of the data. The training data 122 is configured to associate specific values ​​of feature quantities with emotion classifications. For example, if a user spends a long time using an IT service and their usage pattern is consistent each time, it is estimated that the user has an emotion classified as "satisfaction, loyalty" toward the IT service. The type of feature quantity is described by the feature point data 121, and specific numerical values ​​of the feature quantity can be set appropriately for each emotion classification. For convenience of description, FIG. 8 describes the feature quantities as character strings rather than specific numerical values. However, for example, the feature quantity "long usage time" is actually described by a mathematical formula indicating that usage time is above a threshold.

[0024] 9 shows the structure and example data of emotion score data 123. Emotion score data 123 is data that records, for each user ID, a scored value of the emotion a user has toward an IT service. The procedure for creating emotion score data 123 will be described later.

[0025] 10 shows the structure and example data of the upsell time data 124. The upsell time data 124 is data that records, for each user ID, a scored value of the emotion that the user had when they took an upsell action (an action to increase service usage, such as changing to a higher-level contract plan). Each record of the upsell time data 124 can be created by transcribing the record of that user in the emotion score data 123 when the user took the upsell action.

[0026] FIG. 11 is a flowchart explaining the overall operation of the emotion scoring device 1. The arithmetic device 11 extracts new upsell users or new subscribers by analyzing the user data 221 (S1101). The arithmetic device 11 creates a user-specific log 125 by analyzing the application log 222 and the infrastructure log 223 (S1102). The arithmetic device 11 creates emotion score data 123 (S1103). The arithmetic device 11 creates a report based on the results of the above processing (S1104). Details of these steps will be explained with reference to the following drawings.

[0027] FIG. 12 is a flowchart illustrating the details of S1101.

[0028] S1201: The arithmetic device 11 acquires the user data 221.

[0029] S1202: The computing device 11 extracts users who have recently taken upselling action from the user contract information read from the user data 221. For example, it is conceivable to extract users whose contract plans have been changed to higher-level plans. If the upselling action is recorded in other data, the relevant user may be extracted from that data, or upselling action other than a change in contract plan may be extracted. The computing device 11 obtains a record in the emotion score data 123 corresponding to the user who has recently taken upselling action (a record at the time of the upselling action if the emotion score data 123 has an update history), and stores the record in the upselling time data 124.

[0030] S1203: The arithmetic device 11 extracts users who have newly signed up for an IT service from the user contract information read from the user data 221. For example, if there is a user ID that is not recorded in the emotion score data 123 but is recorded in the user data 221, the user is considered to be a new subscriber. The arithmetic device 11 adds a record of the user ID of the new subscriber to the emotion score data 123.

[0031] FIG. 13 is a flowchart illustrating the details of S1102.

[0032] S1301: The arithmetic device 11 acquires the application log 222.

[0033] S1302: The arithmetic unit 11 repeatedly executes S1303 to S1307 for each user ID recorded in the user data 221 (S1302), except that the process is skipped for users who have already canceled their subscriptions.

[0034] S1303: The arithmetic unit 11 performs S1304 to S1305 for all records described in the application log 222 that correspond to the current user ID.

[0035] S1304: The processor 11 extracts one record corresponding to the current user ID from the records described in the application log 222, and stores the extracted record in the user-specific log 125.

[0036] S1305: The computing device 11 extracts records from the infrastructure log 223 that are related to the records from the application log 222 extracted in S1304. For example, it is sufficient to extract records from the infrastructure log 223 that include operations at the same time as the records extracted in S1304. As a result, although user IDs are not recorded in the infrastructure log 223, it is possible to associate the operations of the OS and middleware described in the infrastructure log 223 with user operations on the application using the time as a key. The computing device 11 stores the extracted records from the infrastructure log 223 in the user-specific log 125.

[0037] S1305: Supplementary Note: In this step, distributed tracing technology may be used to associate records in the application log 222 with records in the infrastructure log 223. Distributed tracing technology is a technology that visualizes the entirety of each operation performed in a request to a service by recording the operation of each microservice in a service composed of a collection of microservices. Therefore, by referring to each log recorded in the distributed tracing system, it is possible to associate the operations performed in one request among the records in the application log 222 and the records in the infrastructure log 223 with each other. This step may be performed based on this.

[0038] S1306: The calculation device 11 extracts the record corresponding to the current user ID from the questionnaire log 224 and stores it in the user-specific log 125.

[0039] S1307: The processor 11 sorts the user-specific logs 125 by the date and time field.

[0040] FIG. 14 is a flowchart illustrating the details of S1103.

[0041] S1401 to S1402: The calculation device 11 acquires the user log 125 (S1401) and the teacher data 122 (S1402).

[0042] S1403: The arithmetic unit 11 carries out S1404 to S1405 for each user ID recorded in the emotion score data 123.

[0043] S1404: The calculation device 11 executes S1405 for each record recorded in the teacher data 122.

[0044] S1405: The calculation device 11 searches for records described in the user log 125 that match the features described in the training data 122. If a matching record is found, the calculation device 11 adds up the emotion scores of the records in the emotion score data 123 that correspond to the user ID of that record. For example, if a record in the training data 122 illustrated in FIG. 8 that matches "long usage time, consistent usage pattern" is found in the user log 125, the scores of "satisfaction" and "loyalty" of the corresponding user ID are added up. The greater the degree of match, the greater the addition amount.

[0045] S1405: Supplement: This step may be performed using a learning device 14 configured by machine learning. For example, machine learning is performed in advance so that when the user-specific log 125 is input, an additional value for the emotion score data 123 is output. The training data provided at this time is data describing to which emotion category a score should be added when a record of the user-specific log 125 has certain features (i.e., data such as that shown in FIG. 8). In addition to this, the level of score to be added may be provided as training data, or the amount of score to be added may be calculated by the learning device 14 according to the degree of match of the features.

[0046] FIG. 15 is a flowchart illustrating the details of S1104.

[0047] S1501: The arithmetic device 11 extracts features that are common between records described in the upsell-time data 124 (e.g., the distance between feature vectors is equal to or less than a threshold, and so on). The arithmetic device 11 extracts user IDs that have features that are common to the extracted features from the emotion score data 123. This step is intended to understand the emotions of users who perform upsell behavior.

[0048] S1502: The arithmetic device 11 extracts users who have canceled the IT service from the user data 221. The arithmetic device 11 extracts user IDs having common features among the users who have canceled from the records in the emotion score data 123. This step is intended to understand the emotions of users who have taken cancellation action.

[0049] S1503: The calculation device 11 extracts new features not described in the teacher data 122 from the records described in the user-specific log 125 that correspond to upsell actors and churners. The calculation device 11 adds the extracted new features to the teacher data 122. The learning device 14 performs re-learning using the new teacher data 122. This step is intended to extract the behavioral patterns (emotion scores) of upsell actors and churners as features from the user-specific log 125 and to re-learn using these features, thereby more accurately extracting churn candidate and upsell candidate candidates.

[0050] S1504: The calculation device 11 formats and outputs the results of S1501 to S1503 as a report. The result report of S1501 includes the IDs of users who took upselling actions and the feature values ​​common to those users. The result report of S1502 includes the IDs of users who took cancellation actions and the feature values ​​common to those users. The result report of S1503 includes the items of newly extracted feature values ​​and specific numerical ranges.

[0051] 16 is an example screen showing an example of a report output by the calculation device 11 in S1504. The upper left of the screen shows the emotion score data of user A who canceled, and the fact that there are three users who have emotion score data similar to that. The upper right of the screen shows the emotion score data of user X who was upsold, and the fact that there are ten users who have emotion score data similar to that. The type of newly added feature is shown at the bottom of the screen. In addition to these, measures to be taken in the future may be presented, for example, based on the service menu that led to the cancellation or upsell.

[0052] <Modifications of the present invention> In the above embodiment, the learning device 14 can be configured using a known technique such as a neural network. The substance of the arithmetic processing (learning process, inference process, etc.) performed by the learning device 14 can be executed by the arithmetic device 11.

[0053] In the above embodiments, the arithmetic device 11 can be configured by hardware such as a circuit device that implements the functions, or can be configured by a processor such as a CPU (Central Processing Unit) that executes software that implements the functions (emotion scoring program).

[0054] In the above embodiments, the emotion scoring device 1 may acquire the data stored in the storage device 22 of the information system 2 from the information system 2, for example, via a network, or may acquire the same data by storing it in an appropriate storage medium and then reading the data from the storage device. [Explanation of symbols]

[0055] 1: Emotion scoring device 11: Arithmetic device 12: Storage device 121: Feature point data 122: Teacher data 123: Emotion score data 124: Upsell data 125:User-specific logs 13:Display device 2: Information Systems 221:User data 222: App log 223: Infrastructure log 224: Survey log

Claims

1. An emotion scoring device that scores emotions that a user of a service provided by an information system has toward the service, comprising: a storage device for storing a user-specific log that records the operation of the information system when the user uses the service together with the user's identifier; a learner that learns, by machine learning, a correspondence between the operation of the information system and the classification of the emotion; a computing device for scoring the emotion; Equipped with the arithmetic device inputs the user-specific logs into the learning device to acquire the emotion classification; The computing device scores the emotion based on the emotion classification acquired from the learning device. An emotion scoring device characterized by:

2. the computing device acquires an application log that describes the operation of an application implemented in the information system together with an identifier of the user; The computing device acquires an infrastructure log that describes the operation of the OS or middleware used by the information system, The computing device compares the application log with the infrastructure log to extract, from among the operations described in the infrastructure log, those that occurred in connection with the user's use of the service; The computing device creates the user-specific log based on the behavior extracted from the infrastructure log.

2. The emotion scoring device according to claim 1.

3. The service is composed of multiple microservices that operate independently of each other, The information system implements distributed tracing, which visualizes the entire flow of requests to the services by recording the operation of each of the microservices as a log; The computing device extracts, from among the operations described in the infrastructure log, those that occurred as the user used the service by referring to the log recorded by the distributed tracing.

3. The emotion scoring device according to claim 2.

4. The user-specific log describes the time when an operation of the information system occurred as the user used the service, the operation performed by the user in the service, and an error or processing interruption that occurred in the information system when the user used the service, The learning device is the length of time the user uses the service; the variance in the length of time that the users use the service; the frequency with which the user uses the service; Variance in the frequency with which the users use the service; the type of operation the user performs when using the service; the number of operations performed by the user when using the service; the speed of response when the user uses the service; the accuracy of the response when the user uses the service; the error rate when the user uses the service; the rate of interruptions when the user uses the service; and the emotion classification, The computing device inputs the user-specific logs into the learning device to calculate the score of the emotion associated with at least one of the service usage time, the service usage frequency, the operation on the service, and an error or processing interruption in the service.

2. The emotion scoring device according to claim 1.

5. the user-specific log describes the results of a questionnaire answered by the user regarding the evaluation of the service; the learning device learns a correspondence between the results of the questionnaire and the classification of the emotions; The computing device inputs the user log into the learning device to calculate a score of the emotion associated with the result of the questionnaire.

2. The emotion scoring device according to claim 1.

6. the computing device calculates a feature of the score of the user; The computing device classifies the users according to the feature amounts; The computing device outputs a user interface that presents the results of the classification.

2. The emotion scoring device according to claim 1.

7. the storage device stores upsell time data describing the score held by a user who took an upsell action in the service among the plurality of users at that time, the calculation device calculates the feature amount of the score of the user and the feature amount of the score described in the upsell-time data; The computing device extracts the users whose calculated feature amounts are similar to the upsell time data, and outputs a user interface that presents the results.

2. The emotion scoring device according to claim 1.

8. the storage device stores cancellation time data describing the score held by a user who has cancelled the service among the plurality of users at the time of cancellation; the calculation device calculates the feature amount of the score of the user and the feature amount of the score described in the cancellation data, The computing device extracts the users whose calculated feature amounts are similar to the cancellation data, and outputs a user interface that presents the results.

2. The emotion scoring device according to claim 1.

9. the storage device stores upsell time data describing the score held by a user who took an upsell action in the service among the plurality of users at that time, the calculation device calculates a feature amount of the score described in the upselling data, The learning device learns a correspondence between the feature amount of the upselling data and the classification of the emotion when performing an upselling action in the service.

2. The emotion scoring device according to claim 1.

10. the storage device stores cancellation time data describing the score held by a user who has cancelled the service among the plurality of users at the time of cancellation; the calculation device calculates a feature amount of the score described in the cancellation data, The learning device learns a correspondence between the feature amount of the cancellation data and the classification of the emotion at the time of canceling the service.

2. The emotion scoring device according to claim 1.

11. The emotion scoring device according to claim 1, an information system that provides the service; A service system comprising:

12. An emotion scoring program that causes a computer to execute a process of scoring emotions that a user of a service provided by an information system has toward the service, the program comprising: acquiring a user-specific log from a storage device that stores the user-specific log, in which the operation of the information system when the user uses the service is recorded together with the user's identifier; Scoring the emotion using a learning device that has learned a correspondence between the operation of the information system and the emotion classification by machine learning; Execute In the scoring step, the computer inputs the user-specific log into the learning device to obtain the emotion classification; In the step of scoring, the computer is caused to score the emotion based on the classification of the emotion acquired from the learning device. An emotion scoring program characterized by:

Images