Information processing device, information processing method, and information processing program

Abstract

To enable efficient classification of questionnaires.SOLUTION: An information processing device of disclosed herein comprises an input unit for receiving an input of an answer sentence to a questionnaire, and a classification unit configured to output a classification item corresponding to the entered answer sentence based on a classification model generated through machine learning using learning data that marks the classification item corresponding to the answer sentence as a correct answer among a plurality of classification items.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Traditionally, quality evaluation departments of companies that provide various services to users regularly conduct surveys with the aim of maintaining and improving service quality. These surveys can be free-form, allowing users to freely comment on the good points and areas for improvement of the service, or multiple-choice, where users choose from a range of options. The quality evaluation department selects and checks the contents of the surveys collected, then categorizes them into multiple evaluation categories and provides feedback on the results to the people in charge of each service. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Yahoo! Crowdsourcing, [Searched July 1, 2023], Internet<URL:https: / / crowdsourcing.yahoo.co.jp / > Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a questionnaire is conducted by a quality evaluation department, the contents of free-form responses must be manually checked and categorized, unlike multiple-choice responses that can be classified based on rules. This results in a significant amount of work and costs being incurred to manually check and categorize the contents of all the responses collected.

[0005] The present application has been made in view of the above, and aims to provide an information processing device, an information processing method, and an information processing program that can efficiently classify questionnaires. [Means for solving the problem]

[0006] The information processing device according to the present application is characterized by comprising an input unit that accepts input of a questionnaire response sentence, and a classification unit that outputs a classification item corresponding to the input response sentence based on a classification model generated by machine learning using training data in which a classification item corresponding to the response sentence among a plurality of classification items is assigned as a correct answer. [Effects of the Invention]

[0007] According to one aspect of the embodiment, it is possible to provide an effect that questionnaires can be efficiently classified. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of an information processing system including an information processing device according to an embodiment. [Figure 2] Figure 2 is an explanatory diagram that provides an overview of the UX pyramid. [Figure 3] FIG. 3 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Figure 4] FIG. 4 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Figure 5] FIG. 5 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Figure 6] FIG. 6 is an explanatory diagram showing an example of the questionnaire input screen. [Figure 7] FIG. 7 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an information processing device, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to these embodiments. Furthermore, the same components in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0010] 1 is a block diagram showing an example of the configuration of an information processing system 100 including an information processing device 1 and a plurality of terminal devices 2 according to an embodiment.

[0011] The information processing device 1 and the plurality of terminal devices 2 are connected to each other so as to be able to communicate with each other by wire or wirelessly via a network N. Note that the information processing system 100 shown in FIG.

[0012] Each terminal device 2 is an information processing device used by a user. Here, the user includes a service user who receives various services, a person in charge of quality evaluation who tallys up questionnaires answered by the service users, and the like.

[0013] For example, the terminal device 2 is a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a mobile phone, a PDA (Personal Digital Assistant), etc. Note that the terminal device 2 is not limited to the above examples and may be, for example, a smart watch or a wearable device.

[0014] The information processing device 1 according to the embodiment is a device capable of communicating with various devices via a predetermined network N such as the Internet, and performs various information processing and outputs the processing results to a terminal device 2. The information processing device 1 is realized by, for example, a server device or a cloud system.

[0015] Specifically, the information processing device 1 conducts a survey of service users of various services via the terminal device 2 and collects the response results. The survey includes a free-form format in which users can freely comment on the good points and areas for improvement of the service they used, and a multiple-choice format in which users can choose from a range of options.

[0016] The information processing device 1 classifies the collected questionnaire response results into a plurality of evaluation items, and outputs the response results classified into the evaluation items to the terminal device 2 of the person in charge of the quality evaluation department.

[0017] For example, the information processing device 1 classifies the results of multiple-choice answers into evaluation items corresponding to the selected content based on predetermined rules (rule-based).Furthermore, the information processing device 1 classifies free-form comments (answer sentences) into evaluation items corresponding to the answer sentences based on a classification model generated by machine learning (details will be described later).

[0018] It should be noted that the multiple classification items for classifying the questionnaire response results are assumed to be evaluation items included in the UX pyramid, as shown in Figure 2. Figure 2 is an explanatory diagram for explaining an overview of the UX pyramid.

[0019] The UX pyramid is a description of a hierarchy of user needs that closely mirrors Abraham Maslow's hierarchy of needs, proposed by Aaron Walter in 1940.

[0020] Specifically, the UX pyramid is composed of the following levels (stages): Level 1: Functional, Level 2: Reliable, Level 3: Usable, Level 4: Convenient, Level 5: Enjoyable and comfortable, and Level 6: Meaningful. In other words, the UX pyramid states that after the basic needs (such as functionality and ease of use) at the lower levels are met, superior needs (such as pleasure and value at the top of the pyramid) at the higher levels can be achieved.

[0021] For example, levels 1 to 3 of the UX pyramid focus on the user's ability to accomplish the desired task, while levels 4 to 6 focus on the user experience when using a service. These are commonly used methods for measuring the quality of user experience. The quality evaluation department further subdivides these levels into 11 evaluation categories: "defects," "goal achievement," "reliability," "quickness," "clarity of expression," "ease of reading information," "operability," "usefulness," "continued use," "recommended," and "would be in trouble without it," and categorizes the collected responses. By categorizing the survey responses into these subdivided evaluation categories, the quality evaluation department can identify the strengths and challenges of the service, which can be used to improve service quality.

[0022] Returning to FIG. 1, the information processing device 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0023] The communication unit 10 is realized by, for example, a network interface card (NIC). The communication unit 10 is connected to a network N by wire or wirelessly, and transmits and receives information to and from various other devices. For example, the communication unit 10 transmits and receives information to and from each of a plurality of terminal devices 2 via the network N.

[0024] The storage unit 11 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 also has classification results 20, training data 21, and model information 22.

[0025] The classification result 20 is a result of categorizing the responses (multiple choice, free description) to the questionnaire collected via the terminal device 2 into the above-mentioned multiple evaluation items. Specifically, the classification result 20 includes a classification result ("defect", "objective achieved", ...) for each collected questionnaire response (options, response text (comments)).

[0026] The classification results 20 include not only the results of classification by the classification unit 33 (details of which will be described later) of the information processing device 1, but also results of manual classification by a user (a person in charge of the quality evaluation department) via the terminal device 2. Among the classification results 20, the results of classification by the person in charge of the quality evaluation department are managed separately from the results of classification by the classification unit 33 (for example, by assigning a flag or the like).

[0027] The training data 21 is also called training data for training (learning) the classification model described above. Specifically, in the training data 21, for each case (answer sentence), a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer label.

[0028] The model information 22 is data related to the classification model described above. For example, the model information 22 is parameters of each layer in a neural network that functions as a classification model.

[0029] The processing unit 12 is a controller, and is realized by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) using RAM as a work area to execute various programs (examples of information processing programs) stored in a storage device (e.g., the storage unit 11) inside the information processing device 1. The processing unit 12 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0030] The processing unit 12 has a training data generation unit 30, a training data expansion unit 31, a model generation unit 32, and a classification unit 33, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in Fig. 1, and may have any other configuration as long as it performs the information processing described below.

[0031] The training data generation unit 30 is a processing unit that generates the training data 21. Specifically, the training data generation unit 30 assigns a classification item corresponding to each answer sentence as a correct label to each answer sentence, based on the results of manual classification by a person in the quality evaluation department for each answer sentence of the questionnaire included in the classification result 20.

[0032] FIG. 3 is an explanatory diagram illustrating an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram illustrating an example of the process (S1) related to the generation of the training data 21.

[0033] As shown in FIG. 3, the learning data generation unit 30 acquires the results (comment classification results) of manual classification by a quality evaluation department employee for each of the questionnaire response sentences (comments) included in the classification result 20. Next, the learning data generation unit 30 assigns labels "Label 01," "Label 02," ..., associated with each response sentence to identify each of the classification items, such as "Defect," "Objective Achievement," ..., based on the results of manual classification by the quality evaluation department employee. Next, the learning data generation unit 30 stores the generated learning data 21 in the storage unit 11 by assigning the labels "Label 01," "Label 02," ..., to each response sentence (comment) (S1).

[0034] In this way, the training data 21 generated by the training data generation unit 30 makes it possible to easily perform machine learning using a natural language processing library such as fastText (registered trademark).

[0035] Returning to FIG. 1, the learning data expansion unit 31 is a processing unit that performs processing to expand the number of examples (the number of answer sentences to which correct answer labels are assigned) in the learning data 21 using crowdsourcing.

[0036] FIG. 4 is an explanatory diagram illustrating an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram illustrating an example of the process (S2) related to the expansion of the learning data 21.

[0037] As shown in Fig. 4, the learning data expansion unit 31 presents the service user with a questionnaire response and a plurality of classification items via the display screen G1 of the terminal device 2. Next, the learning data expansion unit 31 accepts input of the classification items with which the service user has classified the presented response sentences on the access screen. In the illustrated example, the display screen G1 accepts the selection of a classification item from 0 to 1 for the comment (response sentence) "I can see timely news."

[0038] Next, the learning data expansion unit 31 assigns the classification item selected and accepted for the answer sentence presented via the display screen G1 or the like as a correct label for the answer sentence, and adds the answer sentence to the learning data 21. The learning data expansion unit 31 expands the learning data 21 in this manner.

[0039] If only the results of manual classification by a person in charge of quality evaluation are used, the number of examples contained in the training data 21 may be small, and the classification accuracy of the classification model based on machine learning using the training data 21 may be insufficient. In response to this, in the information processing device 1, the training data expansion unit 31 expands the training data 21, thereby making it possible to improve the classification accuracy of the classification model.

[0040] Returning to FIG. 1 , the model generation unit 32 is a processing unit that generates (learns and trains) a classification model by machine learning based on the learning data 21. Any known technology can be applied to the learning of the classification model performed by the model generation unit 32, and a learning method appropriately selected depending on the acquired information may be used.

[0041] For example, the classification model may be trained using various conventional machine learning techniques (e.g., supervised machine learning techniques such as SVM (Support Vector Machine)). Also, deep learning techniques may be used for training the classification model. For example, various deep learning techniques such as RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network) may be used for training the classification model.

[0042] Specifically, the model generation unit 32 performs machine learning to generate parameters of the classification model using backpropagation so that when an answer sentence is input to the input layer of the classification model in the learning data 21, the classification item assigned to the answer sentence as the correct label is output from the output layer. The model generation unit 32 stores the parameters of the classification model after such machine learning as model information 22 in the storage unit 11.

[0043] The classification unit 33 is a processing unit that receives input of questionnaire response results from the service users via the terminal device 2, classifies the received response results for each of a plurality of classification items, and outputs the results.

[0044] For example, the classification unit 33 classifies the multiple-choice answer results into items corresponding to the selected content based on a rule definition (not shown) previously set in the storage unit 11.

[0045] Furthermore, the classification unit 33 classifies free-form comments (answer sentences) into categories corresponding to the answer sentences based on the classification model generated by the model generation unit 32.

[0046] Figure 5 is an explanatory diagram illustrating an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram illustrating an example of processing related to the generation of a classification model M1 (S3) and classification based on the generated classification model M1 (S4).

[0047] 5, the model generation unit 32 generates a classification model M1 by machine learning based on the training data 21 (S3). The classification unit 33 classifies the comments (answer sentences) actually entered in the questionnaire based on the classification model M1 generated by the model generation unit 32 (S4).

[0048] More specifically, the classification unit 33 accepts input of questionnaire responses from the service user via a questionnaire input screen or the like on the terminal device 2. Fig. 6 is an explanatory diagram showing an example of the questionnaire input screen.

[0049] As shown in FIG. 6, the classification unit 33 presents a questionnaire input screen G2 to the terminal device 2 of the service user, and accepts input of comments (answers) to the questionnaire from the service user.

[0050] 5, the classification unit 33 inputs each comment R11 received via the questionnaire input screen G2 into the classification model M1 and obtains a predicted probability R13 for each label (classification item). The classification unit 33 outputs the label (classification item) with the highest value among the predicted probabilities R13 for each label as the classification result R12.

[0051] For example, for a comment R11 that "it's easy to find news related to a topic," the classification unit 33 outputs the highest label "operability" with a predicted probability R13 of "0.98098516" as the classification result R12. Here, the classification unit 33 may output the predicted probability R13 together with the classification result R12 that indicates the classification item (label).

[0052] The output destination of the classification unit 33 may be a file output to the storage unit 11, or may be the display screen of the terminal device 2 of the person in charge of the quality evaluation department.

[0053] The information processing device 1 or the terminal device 2 according to the above-described embodiment is realized by, for example, a computer 80 configured as shown in Fig. 7. The following description will be given taking the information processing device 1 as an example. Fig. 7 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing device 1 according to the embodiment. The computer 80 has a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0054] The CPU 81 operates based on programs stored in the ROM 83 or the HDD 84 and controls each unit. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 starts up, programs dependent on the hardware of the computer 80, and the like. The HDD 84 stores programs executed by the CPU 81 and data used by such programs. The communication I / F 85 receives data from other devices via the network N (see FIG. 1) and sends it to the CPU 81, and transmits data generated by the CPU 81 to other devices via the network N.

[0055] The CPU 81 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output I / F 86. The CPU 81 acquires data from the input devices via the input / output I / F 86. The CPU 81 also outputs generated data to the output devices via the input / output I / F 86.

[0056] The media I / F 87 reads a program or data stored in a recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads the program or data from the recording medium 88 onto the RAM 82 via the media I / F 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0057] For example, when the computer 80 functions as the information processing device 1 according to the embodiment, the CPU 81 of the computer 80 executes programs loaded onto the RAM 82 to realize the functions of the processing unit 12. In addition, the HDD 84 stores data in the storage unit 11. The CPU 81 of the computer 80 reads and executes these programs from a recording medium 88, but as another example, the CPU 81 may obtain these programs from another device via the network N.

[0058] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0059] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0060] For example, the information processing device 1 described above may be realized by multiple server computers, and depending on the function, the configuration can be flexibly changed, such as by calling an external platform using an API (Application Programming Interface) or network computing.

[0061] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0062] As described above, the information processing device 1 according to the embodiment accepts input of a questionnaire response sentence. The information processing device 1 also outputs a classification item corresponding to the input response sentence based on a classification model M1 generated by machine learning using learning data 21 in which a classification item corresponding to the response sentence is assigned as a correct answer among a plurality of classification items. This allows the information processing device 1 to efficiently classify the questionnaire response sentences without manually checking and classifying them. This prevents a user (e.g., a quality evaluation department) from having to spend a large amount of time manually classifying the questionnaire response sentences into a plurality of classification items, thereby reducing the cost of classification.

[0063] Furthermore, the information processing device 1 generates learning data 21 in which a classification item that is determined to be the correct answer is assigned to each of a plurality of answer sentences, based on a classification result 20 in which a classification item corresponding to the answer sentence is set for each of the plurality of answer sentences. Based on the generated learning data 21, the information processing device 1 performs machine learning on a classification model M1 so that, when an answer sentence included in the learning data 21 is input, the classification model M1 outputs the classification item assigned as the correct answer to the answer sentence. This allows the information processing device 1 to generate the classification model M1 using the classification result 20 set by a user or the like.

[0064] Furthermore, the information processing device 1 accepts input of classification items for the presented answer sentence, assigns the accepted classification items to this answer sentence as a correct answer, and adds it to the learning data 21. This enables the information processing device 1 to expand the examples (answer sentences and classification items that are correct answers for the answer sentences) included in the learning data 21, and prepares learning data 21 for generating a classification model M1 with higher accuracy.

[0065] Furthermore, the multiple classification items in the information processing device 1 are evaluation items included in the UX pyramid, which allows the information processing device 1 to easily classify into the evaluation items in the UX pyramid.

[0066] Furthermore, the information processing device 1 outputs information indicating the accuracy of the classification (prediction accuracy) along with the classification item corresponding to the input answer sentence, thereby enabling the user of the information processing device 1 to easily know the accuracy of the classification predicted by the information processing device 1.

[0067] The above describes the embodiments of the present application in detail based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0068] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, the learning data generation unit can be read as learning data generation means or a learning data generation circuit. [Explanation of symbols]

[0069] 1...Information processing device 2...Terminal device 10. Communications Department 11...Storage section 12...Processing section 20…Classification result 21...Learning data 22...Model information 30...Learning data generation unit 31…Learning Data Expansion Department 32...Model generation section 33...Classification section 80...Computer 81...CPU 82...RAM 83...ROM 84...HDD 85...Communication I / F 86...Input / output interface 87...Media I / F 88...Recording media 100...Information Processing Systems G1…display screen G2: Questionnaire entry screen N...Network M1…Classification model R1...input / output data R11...Comments R12…Classification result R13: Predicted probability

Claims

1. An input unit that accepts input of a questionnaire response for a predetermined service; a classification unit that outputs a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer; Equipped with a learning data generation unit that generates the learning data in which each of the plurality of answer sentences is assigned a classification item that is determined to be a correct answer, based on the classification item that is determined to be a correct answer assigned to each of the plurality of answer sentences, the classification item being based on the classification results of each of the plurality of answer sentences by a predetermined person in charge; a model generation unit that performs machine learning on the classification model based on the generated learning data so that, when an answer sentence included in the learning data is input, the classification model outputs a classification item assigned as a correct answer to the answer sentence; a learning data expansion unit that receives, from a user who uses the predetermined service, an input of a classification item for an answer sentence presented to the user, and assigns the received classification item to the answer sentence as a correct answer and adds the answer sentence to the learning data, 1. An information processing device comprising:

2. The classification unit outputs the classification items based on the classification model generated by the machine learning using the learning data in which the classification item corresponding to the answer sentence is assigned as the correct answer among the classification items set based on the evaluation items indicated by a UX pyramid that indicates user needs in stages.

2. The information processing apparatus according to claim 1, wherein:

3. the classification unit outputs information indicating the accuracy of the classification together with the classification items.

2. The information processing apparatus according to claim 1, wherein:

4. 1. A computer-implemented information processing method, comprising: an input step of accepting input of a questionnaire response for a predetermined service; a classification step of outputting a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer; Including, a learning data generation step of generating the learning data in which each of the plurality of answer sentences is assigned a classification item that is determined to be a correct answer, based on the classification item that is determined to be a correct answer assigned to each of the plurality of answer sentences, the classification item being based on the classification results of each of the plurality of answer sentences by a predetermined person in charge; a model generation process for machine learning the classification model based on the generated learning data so that, when an answer sentence included in the learning data is input, the classification model outputs a classification item assigned as a correct answer to the answer sentence; and a learning data expansion step of accepting, from a user who uses the predetermined service, an input of a classification item for an answer sentence presented to the user, and assigning the accepted classification item to the answer sentence as a correct answer and adding the answer sentence to the learning data. An information processing method comprising:

5. An input procedure for accepting input of a questionnaire response for a predetermined service; a classification procedure for outputting a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer; on the computer, a learning data generation step of generating the learning data in which each of the plurality of answer sentences is assigned a classification item that is determined to be a correct answer, based on the classification item that is determined to be a correct answer assigned to each of the plurality of answer sentences, the classification item being based on the classification results of each of the plurality of answer sentences by a predetermined person in charge; a model generation step of machine learning the classification model based on the generated learning data so that, when an answer sentence included in the learning data is input, the classification model outputs a classification item assigned as a correct answer to the answer sentence; and causing the computer to further execute a learning data expansion procedure of accepting, from a user who uses the predetermined service, an input of a classification item for an answer sentence presented to the user, and assigning the accepted classification item to the answer sentence as a correct answer and adding the answer sentence to the learning data. An information processing program characterized by:

Images