Information processing device, information processing method, and information processing program

Abstract

To enable efficient classification of questionnaires.SOLUTION: An information processing device of disclosed herein comprises an input unit for receiving an input of an answer sentence to a questionnaire, and a classification unit configured to output a classification item corresponding to the entered answer sentence based on a classification model generated through machine learning using learning data that marks the classification item corresponding to the answer sentence as a correct answer among a plurality of classification items.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Traditionally, quality evaluation departments of companies that provide various services to users regularly conduct surveys to maintain and improve service quality. These surveys can be free-form, where users can freely comment on the good points and areas for improvement of the service, or multiple-choice, where users can choose from a range of options. The quality evaluation department selects and checks the contents of the surveys collected, classifies them into multiple evaluation items, and provides feedback on the classification results to the people in charge of each service. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Yahoo! Crowdsourcing, [Searched on July 1, 2023], Internet<URL:https: / / crowdsourcing.yahoo.co.jp / > Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a questionnaire is conducted in a quality evaluation department, the contents of free-form comments must be manually checked and categorized, unlike multiple-choice questions that can be categorized based on rules. This results in a large amount of work to manually check and categorize the contents of all the collected questionnaires, resulting in a problem of high costs.

[0005] The present application has been made in consideration of the above, and has an object to provide an information processing device, an information processing method, and an information processing program capable of efficiently classifying questionnaires. [Means for solving the problem]

[0006] The information processing device according to the present application is characterized by comprising an input unit that accepts input of a questionnaire response sentence, and a classification unit that outputs a classification item corresponding to the input response sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the response sentence among a plurality of classification items is assigned as a correct answer. Effect of the Invention

[0007] According to one aspect of the embodiment, it is possible to efficiently classify questionnaires. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of an information processing system including an information processing device according to an embodiment. [Diagram 2] FIG. 2 is an explanatory diagram outlining the UX pyramid. [Diagram 3] FIG. 3 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Figure 4] FIG. 4 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Diagram 5] FIG. 5 is an explanatory diagram illustrating an example of the operation of the information processing device according to the embodiment. [Figure 6] FIG. 6 is an explanatory diagram showing an example of the questionnaire input screen. [Figure 7] FIG. 7 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the information processing device, the information processing method, and the information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to the embodiments. In addition, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions are omitted.

[0010] 1 is a block diagram showing an example of a configuration of an information processing system including an information processing device according to an embodiment. As shown in FIG. 1, the information processing system 100 includes an information processing device 1 and a plurality of terminal devices 2.

[0011] The information processing device 1 and the multiple terminal devices 2 are connected to each other so as to be able to communicate with each other by wire or wirelessly via a network N. Note that the information processing system 100 shown in FIG.

[0012] Each terminal device 2 is an information processing device used by a user. Here, the user includes a service user who receives various services, a person in charge of quality evaluation who tallys up questionnaires answered by the service users, and the like.

[0013] For example, the terminal device 2 is a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a mobile phone, a PDA (Personal Digital Assistant), etc. The terminal device 2 is not limited to the above examples, and may be, for example, a smart watch or a wearable device.

[0014] The information processing device 1 according to the embodiment is a device capable of communicating with various devices via a predetermined network N such as the Internet, and performs various information processing and outputs the processing results to a terminal device 2. The information processing device 1 is realized by, for example, a server device or a cloud system.

[0015] Specifically, the information processing device 1 conducts a survey of service users of various services via the terminal device 2 and collects the response results. The survey includes a free description format in which the user can freely comment on the good points and areas for improvement of the service used, a multiple-choice format in which the user can select from a range of options, and the like.

[0016] The information processing device 1 classifies the collected questionnaire response results into a plurality of evaluation items, and outputs the response results classified into the evaluation items to the terminal device 2 of the person in charge of the quality evaluation department.

[0017] For example, the information processing device 1 classifies the multiple choice answer results into evaluation items corresponding to the selected contents based on a predetermined rule (rule-based). Also, the information processing device 1 classifies the free-form comments (answer sentences) into evaluation items corresponding to the answer sentences based on a classification model generated by machine learning (details will be described later).

[0018] It should be noted that the multiple classification items for classifying the questionnaire response results are evaluation items included in the UX pyramid, as shown in Figure 2. Figure 2 is an explanatory diagram for explaining the outline of the UX pyramid.

[0019] The UX Pyramid is Aaron Walter's description of a hierarchy of user needs that closely mirrors Abraham Maslow's hierarchy of needs, first proposed in 1940.

[0020] Specifically, the UX pyramid is composed of the following levels (stages): Level 1: Functional, Level 2: Reliable, Level 3: Usable, Level 4: Convenient, Level 5: Enjoyable and comfortable, and Level 6: Meaningful. In other words, the UX pyramid states that after the basic needs (such as functionality and ease of use) at the lower levels are met, superior needs (such as pleasure and value at the top of the pyramid) at the higher levels can be achieved.

[0021] For example, levels 1 to 3 of the UX pyramid focus on the user's ability to accomplish the desired task, while levels 4 to 6 focus on the user experience when using a service, and are generally used as a method of measuring the quality of the user experience. The quality evaluation department further subdivides these levels into 11 evaluation items, namely "defects," "goal achievement," "reliability," "quickness," "clarity of expression," "ease of reading information," "operability," "usefulness," "continuous use," "recommended," and "inconvenience without," and categorizes the collected responses. By categorizing the survey responses into these subdivided evaluation items, the quality evaluation department can clarify the strengths and challenges of the service, which will be useful in improving service quality.

[0022] Returning to FIG. 1, the information processing device 1 includes a communication unit 10, a storage unit 11, and a processing unit 12.

[0023] The communication unit 10 is realized by, for example, a network interface card (NIC). The communication unit 10 is connected to a network N by wire or wirelessly, and transmits and receives information to and from various other devices. For example, the communication unit 10 transmits and receives information to and from each of a plurality of terminal devices 2 via the network N.

[0024] The storage unit 11 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 11 also has a classification result 20, training data 21, and model information 22.

[0025] The classification result 20 is a result of classifying the responses (multiple choice, free description) to the questionnaire collected via the terminal device 2 into the above-mentioned multiple evaluation items. Specifically, the classification result 20 includes a classification result ("defect", "objective achievement", ...) for each of the collected responses (options, answer sentences (comments)) to the questionnaire.

[0026] The classification result 20 includes not only the results of classification by the classification unit 33 (described in detail later) of the information processing device 1, but also results of manual classification by a user (a person in charge of the quality evaluation department) via the terminal device 2. Among the classification results 20, the results of classification by the person in charge of the quality evaluation department are managed separately from the results of classification by the classification unit 33 (for example, by attaching a flag or the like).

[0027] The learning data 21 is data also called training data for training (learning) the above-mentioned classification model. Specifically, in the learning data 21, for each case (answer sentence), a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer label.

[0028] The model information 22 is data related to the above-mentioned classification model. For example, the model information 22 is parameters of each layer in a neural network that functions as a classification model.

[0029] The processing unit 12 is a controller, and is realized by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (an example of an information processing program) stored in a storage device (e.g., the storage unit 11) inside the information processing device 1 using a RAM as a working area. The processing unit 12 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0030] The processing unit 12 has a learning data generating unit 30, a learning data expanding unit 31, a model generating unit 32, and a classification unit 33, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the processing unit 12 is not limited to the configuration shown in Fig. 1, and may be any other configuration as long as it performs the information processing described below.

[0031] The learning data generation unit 30 is a processing unit that generates the learning data 21. Specifically, based on the result of manual classification by a person in charge of quality evaluation of each answer sentence to the questionnaire included in the classification result 20, the learning data generation unit 30 assigns a classification item corresponding to the answer sentence as a correct label to each answer sentence.

[0032] FIG. 3 is an explanatory diagram for explaining an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram for explaining an example of the process (S1) related to the generation of the learning data 21. As shown in FIG.

[0033] As shown in Fig. 3, the learning data generation unit 30 acquires the results (comment classification results) of manual classification by a person in charge of quality evaluation department for each of the questionnaire response sentences (comments) included in the classification result 20. Next, the learning data generation unit 30 assigns labels "Label 01", "Label 02", ..., which are associated with each response sentence to identify each of the classification items of "Defect", "Objective Achievement", ..., based on the results of manual classification by the person in charge of quality evaluation department. Next, the learning data generation unit 30 stores the learning data 21 generated by assigning the labels "Label 01", "Label 02" to each response sentence (comment) in the storage unit 11 (S1).

[0034] In this way, the learning data 21 generated by the learning data generation unit 30 makes it possible to easily perform machine learning using a natural language processing library such as fastText (registered trademark).

[0035] Returning to FIG. 1, the learning data expanding unit 31 is a processing unit that performs processing to expand the number of examples (the number of answer sentences to which correct answer labels are assigned) in the learning data 21 by using crowdsourcing.

[0036] FIG. 4 is an explanatory diagram illustrating an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram illustrating an example of the process (S2) related to expanding the learning data 21.

[0037] As shown in Fig. 4, the learning data expansion unit 31 presents the answer sentence to the questionnaire and a plurality of classification items to the service user via the display screen G1 of the terminal device 2 or the like. Next, the learning data expansion unit 31 accepts input of the classification items with which the service user has classified the presented answer sentence on the access screen. In the illustrated example, the display screen G1 accepts the selection of a classification item from 0 to 1 for the comment (answer sentence) "I can see timely news."

[0038] Next, the learning data expansion unit 31 assigns the classification item for which the selection has been accepted to the answer sentence presented via the display screen G1 or the like as a correct label for the answer sentence, and adds it to the learning data 21. The learning data expansion unit 31 expands the learning data 21 in this manner.

[0039] Manual classification by a person in charge of quality evaluation may result in a small number of cases contained in the learning data 21, resulting in insufficient classification accuracy of the classification model based on machine learning using the learning data 21. In response to this, in the information processing device 1, the learning data expansion unit 31 expands the learning data 21, thereby making it possible to improve the classification accuracy of the classification model.

[0040] Returning to Fig. 1, the model generation unit 32 is a processing unit that generates (learns and trains) a classification model by machine learning based on the learning data 21. Any known technology can be applied to the learning of the classification model performed by the model generation unit 32, and a learning method appropriately selected according to the acquired information may be used.

[0041] For example, the classification model may be trained using various conventional machine learning techniques (e.g., supervised machine learning techniques such as SVM (Support Vector Machine)). Furthermore, deep learning techniques may be used for training the classification model. For example, the classification model may be trained using various deep learning techniques such as RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network).

[0042] Specifically, the model generation unit 32 machine-learns the parameters of the classification model by backpropagation so that the classification item assigned to the answer sentence as the correct label is output from the output layer when the answer sentence is input to the input layer of the classification model in the learning data 21. The model generation unit 32 stores the parameters of the classification model after such machine learning as model information 22 in the storage unit 11.

[0043] The classification unit 33 is a processing unit that receives input of questionnaire response results from the service users via the terminal device 2, classifies the received response results for each of a plurality of classification items, and outputs the results.

[0044] For example, the classification unit 33 classifies multiple-choice answer results into items corresponding to the selected contents based on rule definitions (not shown) previously set in the storage unit 11.

[0045] Furthermore, the classification unit 33 classifies free-form comments (answer sentences) into categories corresponding to the answer sentences based on the classification model generated by the model generation unit 32.

[0046] FIG. 5 is an explanatory diagram illustrating an example of the operation of the information processing device 1 according to the embodiment, and more specifically, a diagram illustrating an example of processing related to the generation of a classification model M1 (S3) and classification based on the generated classification model M1 (S4).

[0047] 5, the model generation unit 32 generates a classification model M1 by machine learning based on the learning data 21 (S3). The classification unit 33 classifies the comments (answer sentences) actually entered in the questionnaire based on the classification model M1 generated by the model generation unit 32 (S4).

[0048] More specifically, the classification unit 33 accepts input of a questionnaire response from the service user via a questionnaire input screen or the like of the terminal device 2. Fig. 6 is an explanatory diagram showing an example of the questionnaire input screen.

[0049] As shown in FIG. 6, the classification unit 33 presents a questionnaire input screen G2 to the terminal device 2 of the service user, and accepts input of comments (answer text) to the questionnaire from the service user.

[0050] 5, the classification unit 33 inputs each comment R11 received via the questionnaire input screen G2 into the classification model M1 and obtains a predicted probability R13 for each label (classification item). The classification unit 33 outputs the label (classification item) with the highest value among the predicted probabilities R13 for each label as the classification result R12.

[0051] For example, for a comment R11 that "it is easy to find news related to a topic," the classification unit 33 outputs the highest label "operability" with a predicted probability R13 of "0.98098516" as the classification result R12. Here, the classification unit 33 may output the predicted probability R13 together with the classification result R12 indicating the classification item (label).

[0052] The output destination of the classification unit 33 may be a file output to the storage unit 11, or may be the display screen of the terminal device 2 of the person in charge of the quality evaluation department.

[0053] The information processing device 1 or the terminal device 2 according to the above-described embodiment is realized by a computer 80 having a configuration as shown in Fig. 7, for example. The information processing device 1 will be described below as an example. Fig. 7 is a hardware configuration diagram showing an example of a computer 80 that realizes the functions of the information processing device 1 according to the embodiment. The computer 80 has a CPU 81, a RAM 82, a ROM (Read Only Memory) 83, an HDD (Hard Disk Drive) 84, a communication interface (I / F) 85, an input / output interface (I / F) 86, and a media interface (I / F) 87.

[0054] The CPU 81 operates based on a program stored in the ROM 83 or the HDD 84, and controls each unit. The ROM 83 stores a boot program executed by the CPU 81 when the computer 80 starts up, programs dependent on the hardware of the computer 80, and the like. The HDD 84 stores programs executed by the CPU 81 and data used by such programs, and the like. The communication I / F 85 receives data from other devices via the network N (see FIG. 1) and sends it to the CPU 81, and transmits data generated by the CPU 81 to other devices via the network N.

[0055] The CPU 81 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output I / F 86. The CPU 81 acquires data from the input devices via the input / output I / F 86. The CPU 81 also outputs data generated via the input / output I / F 86 to the output devices.

[0056] The media I / F 87 reads a program or data stored in a recording medium 88 and provides it to the CPU 81 via the RAM 82. The CPU 81 loads the program from the recording medium 88 onto the RAM 82 via the media I / F 87 and executes the loaded program. The recording medium 88 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0057] For example, when the computer 80 functions as the information processing device 1 according to the embodiment, the CPU 81 of the computer 80 executes a program loaded onto the RAM 82 to realize the function of the processing unit 12. In addition, the HDD 84 stores data in the storage unit 11. The CPU 81 of the computer 80 reads and executes these programs from a recording medium 88, but as another example, these programs may be obtained from another device via the network N.

[0058] In addition, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0059] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0060] For example, the information processing device 1 described above may be realized by multiple server computers, and depending on the functions, the configuration can be flexibly changed by calling an external platform using an API (Application Programming Interface) or network computing.

[0061] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0062] As described above, the information processing device 1 according to the embodiment accepts input of a response sentence to a questionnaire. The information processing device 1 also outputs a classification item corresponding to the input response sentence based on a classification model M1 generated by machine learning using learning data 21 in which a classification item corresponding to the response sentence among a plurality of classification items is assigned as a correct answer. This allows the information processing device 1 to efficiently classify the questionnaire response sentences without manually checking and classifying the questionnaire response sentences. This allows a user (e.g., a quality evaluation department) to prevent a large amount of manual labor from being required when classifying the questionnaire response sentences into a plurality of classification items, and reduces the cost of classification.

[0063] Furthermore, the information processing device 1 generates learning data 21 in which a classification item that is a correct answer is assigned to each of a plurality of answer sentences, based on a classification result 20 in which a classification item corresponding to the answer sentence is set for each of the plurality of answer sentences. The information processing device 1 machine-learns a classification model M1 based on the generated learning data 21 so that, when an answer sentence included in the learning data 21 is input, the classification model M1 outputs the classification item assigned to the answer sentence as the correct answer. This allows the information processing device 1 to generate the classification model M1 using the classification result 20 set by a user or the like.

[0064] Furthermore, the information processing device 1 accepts input of a classification item for the presented answer sentence, assigns the accepted classification item to this answer sentence as a correct answer, and adds it to the learning data 21. This enables the information processing device 1 to expand the examples (answer sentences and classification items that are correct answers to the answer sentences) included in the learning data 21, and can prepare learning data 21 for generating a classification model M1 with higher accuracy.

[0065] Furthermore, the multiple classification items in the information processing device 1 are evaluation items included in the UX pyramid. This allows the information processing device 1 to easily perform classification into the evaluation items in the UX pyramid.

[0066] In addition, the information processing device 1 outputs information indicating the accuracy of the classification (prediction accuracy) together with the classification item corresponding to the input answer sentence. This allows the user of the information processing device 1 to easily know the accuracy of the classification predicted by the information processing device 1.

[0067] The above describes the embodiment of the present application in detail based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that include various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0068] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit", etc. For example, the learning data generation unit can be read as learning data generation means or a learning data generation circuit. [Explanation of symbols]

[0069] 1...Information processing device 2...Terminal device 10. Communications Department 11...Storage section 12... Processing section 20…Classification result 21…Learning data 22…Model information 30…Learning data generation unit 31…Learning Data Expansion Department 32…Model generation section 33…Classification section 80…Computer 81…CPU 82…RAM 83...ROM 84…HDD 85…Communication I / F 86... Input / Output Interface 87…Media I / F 88...Recording media 100...Information processing systems G1…display screen G2: Questionnaire entry screen N…Network M1…Classification model R1: Input / Output data R11…Comments R12…Classification result R13: Predicted probability

Claims

1. an input unit for accepting input of answers to the questionnaire; A classification unit that outputs a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence is assigned as a correct answer among a plurality of classification items; An information processing device comprising:

2. a learning data generation unit that generates the learning data in which a classification item that is a correct answer is assigned to each of the plurality of answer sentences based on a classification result in which a classification item corresponding to the answer sentence is set for each of the plurality of answer sentences; and a model generation unit that performs machine learning on the classification model based on the generated learning data so that, when an answer sentence included in the learning data is input, a classification item assigned as a correct answer to the answer sentence is output.

2. The information processing apparatus according to claim 1,

3. a learning data expanding unit that accepts an input of a classification item for the presented answer sentence, assigns the accepted classification item to the answer sentence as a correct answer, and adds the classification item to the learning data; 3. The information processing apparatus according to claim 2.

4. The plurality of classification items are evaluation items included in the UX pyramid, 2. The information processing apparatus according to claim 1,

5. The classification unit outputs information indicating the accuracy of classification together with the classification items.

2. The information processing apparatus according to claim 1,

6. 1. A computer-implemented information processing method, comprising: An input step of accepting input of a questionnaire response text; A classification process for outputting a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer; 13. An information processing method comprising:

7. An input procedure for accepting input of a questionnaire response text; A classification step of outputting a classification item corresponding to the input answer sentence based on a classification model generated by machine learning using learning data in which a classification item corresponding to the answer sentence among a plurality of classification items is assigned as a correct answer; An information processing program characterized by causing a computer to execute the above.

Images