Information processing device and program

Abstract

To provide an information processing device and a program that can perform learning of the classification of human utterances, confirmation of classification results, and the like.SOLUTION: In a system in which a plurality of information processing devices are communicatively connected via a network, an information processing device 1 includes a display information generation unit 101 and a selection information generation unit 102. The display information generation unit 101 generates first display information for displaying a plurality of objects associated with respective options. The options correspond to types of utterances defined according to conversation information. The selection information generation unit 102 generates, each time one of the objects is selected, selection information associating an option corresponding to the selected object with selected time.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] A technique for classifying utterances in a conversation has been proposed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-144906 A Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, regarding the classification of speech, there is a demand not for classified results but for people to learn how to classify speech. For example, when training trainers and therapists for Parent Child Interaction Therapy (hereinafter referred to as PCIT), training in speech classification is required.

[0005] In view of the above circumstances, the present invention provides an information processing device and a program capable of learning classification of human utterances and confirming the classification results. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing device. The information processing device includes a display information generating unit and a selection information generating unit. The display information generating unit generates first display information displaying a plurality of objects each associated with an option. The option corresponds to a type of utterance determined according to the conversation information. Each time any one of the objects is selected, the selection information generating unit generates selection information that associates an option corresponding to the selected object with a time of the selection.

[0007] According to one aspect of the present invention, it becomes possible to easily perform learning of utterance classification, etc. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram showing a configuration of an information processing device 1. FIG. [Diagram 3] 1 is a block diagram showing a functional configuration of an information processing device 1. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a screen display. [Diagram 5] FIG. 13 is a diagram showing a display example of a first area. [Figure 6] FIG. 13 is a diagram showing a display example of a second display area. [Figure 7] The classification results by two selectors are shown. [Figure 8] The classification results by three selectors are shown. [Figure 9] FIG. 13 is a diagram showing an example of correspondence between classification results. [Figure 10] FIG. 2 is an activity diagram showing the flow of operations of the information processing device 1. [Figure 11] FIG. 2 is an activity diagram showing the flow of operations of the information processing device 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, in this embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Overall structure 1 is a diagram showing an example of the configuration of a system according to an embodiment of the present invention. In the system shown in the figure, an information processing device 1, an information processing device 2, and an information processing device 3 are connected to each other via a network 4 so as to be able to communicate with each other.

[0014] The information processing device 1 functions as a server, and generates display information to be displayed on the information processing device 2 and receives operation contents on the information processing device 2. The information processing device 1 can also provide conversation information and obtain selection information from the information processing device 3.

[0015] The information processing device 2 is used by a selector, for example, a learner who is conducting learning, and functions as a client. This information processing device 2 needs to be in a state in which it can communicate with the information processing device 1 during operation.

[0016] The information processing device 3 is used by the selector and can be used without communicating with the information processing device 1. However, it is preferable that the information processing device 3 is capable of communicating with the information processing device 1 before and after use by the selector.

[0017] The network 4 is a network that enables communication between the information processing device 1, the information processing device 2, and the information processing device 3, and includes, for example, the Internet.

[0018] 2. Configuration of information processing device Next, a description will be given of the configuration of the information processing device 1. Fig. 2 is a diagram showing the configuration of the information processing device 1. As shown in the figure, the information processing device 1 has a processing unit 11, a storage unit 12, a temporary storage unit 13, an external device connection unit 14, and a communication unit 15, and these components are electrically connected inside the information processing device 1 via a communication bus 16.

[0019] The processing unit 11 is realized by, for example, a central processing unit (CPU), and operates according to a predetermined program stored in the storage unit 12 to realize various functions.

[0020] The storage unit 12 is a non-volatile storage medium that stores various information. This is realized by a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 12 can also be arranged in another device that can communicate with the information processing device 1.

[0021] The temporary storage unit 13 is a volatile storage medium. This is realized by a memory such as a random access memory (RAM), and stores information (arguments, arrays, etc.) that is temporarily required when the processing unit 11 operates.

[0022] The external device connection unit 14 is a connection unit that conforms to standards such as Universal Serial Bus (USB) and High-Definition Multimedia Interface (HDMI), and allows input devices such as a keyboard and display devices such as a monitor to be connected.

[0023] The communication unit 15 is, for example, a communication means conforming to a local area network (LAN) standard, and realizes communication between the information processing device 1 and the local area network or a network 4 such as the Internet via the local area network.

[0024] It should be noted that the information processing device 1 may be a general-purpose server computer or a personal computer, and the information processing device 1 may be configured using a plurality of computers.

[0025] Moreover, the configurations of the information processing device 2 and the information processing device 3 are similar to that of the information processing device 1, and therefore a description thereof will be omitted.

[0026] 3. Functions of the information processing device 1 Next, a description will be given of the functions of the information processing device 1. The information processing device 1 performs processing to support a person who is learning to classify utterances, and realizes each of the functional units described below by operating according to a program. This program causes a computer to operate or function as an information processing device.

[0027] 3 is a block diagram showing the functional configuration of the information processing device 1. As shown in the figure, the information processing device 1 includes a display information generating unit 101, a selection information generating unit 102, a storage unit 103, a determination unit 104, a playback unit 105, and a recording unit 106.

[0028] The display information generating unit 101 generates first display information that displays a plurality of objects associated with each of the options. The first display information is information to be displayed on the screen of the information processing device 2, and the objects are, for example, selection buttons. The options correspond to types of speech determined according to conversation information, and in the case of PCIT, there are eight types: neutral conversation (TA), behavior explanation (BD), repetition (RF), specific praise (LP), general praise (UP), question (QU), command (CM), and negative conversation (NTA). An example of the display of the screen based on the first display information will be described later. The information processing device 3 also has a functional unit equivalent to the display information generating unit 101, and this functional unit generates the first display information as information to be displayed on the screen of the information processing device 3.

[0029] Furthermore, the display information generating unit 101 generates second display information in which symbols representing options are arranged according to the time corresponding to the options, based on selection information described later. The second display information is information to be displayed on the screen of the information processing device 2, and the symbols are any of characters, character strings, symbols, and icons. The symbols are arranged on the time axis. The information processing device 3 also generates second display information as information to be displayed on the screen of the information processing device 3.

[0030] Each time any of the objects is selected, the selection information generating unit 102 generates selection information that associates an option corresponding to the selected object with the time of selection. Specifically, when a selector (learner) who operates the information processing device 2 presses a selection button corresponding to an utterance in the conversation while listening to the conversation, the information processing device 1 is notified of this, and the selection information generating unit 102 generates selection information that associates the content of the notified option with the time when the selection button was pressed. In the case of PCIT, the conversation is a parent-child conversation. Also, in the case of the information processing device 3, the selection information is generated in response to a selection operation of the selection button on the information processing device 3.

[0031] Furthermore, the selection information generating unit 102 may include information on the selector (learner) who selected the object in the selection information to be generated. When the selection information includes information on the selector, the second display information generated by the display information generating unit 101 can be configured to distinguish between a first symbol corresponding to the selection information generated by the selection of the first selector and a second symbol corresponding to the selection information generated by the selection of the second selector.

[0032] The storage unit 103 stores the selection information generated by the selection information generation unit 102. The storage unit 103 also stores conversation information played back by a playback unit 105 (described later) and conversation information recorded by a recording unit 106 (described later). Note that the storage unit 103 functions as a unit that performs an operation of storing the selection information and conversation information or an operation of reading out the stored selection information and conversation information according to a program, and the storage of the selection information and conversation information itself is performed by the storage unit 12.

[0033] The determination unit 104 is configured to be able to determine the correspondence between the selection information generated by the selection of the first selector and the selection information generated by the selection of the second selector. For example, when the option selected by the first selector and the option selected by the second selector match and the time selected by both of them match, it is determined that the two correspond. However, a predetermined error may be allowed for the time. On the other hand, when the contents of the options of both of them do not match, or when only one of them has made a selection, it is determined that the two do not correspond.

[0034] Furthermore, when the determination unit 104 makes a determination, the display information generating unit 101 generates second display information including an indication of the correspondence between the first symbol and the second symbol. For example, the second display information displays the first symbol and the second symbol when the selection information generated by the selection of the first selector does not correspond to the selection information generated by the selection of the second selector, and hides the first symbol and the second symbol when the selection information generated by the selection of the first selector corresponds to the selection information generated by the selection of the second selector.

[0035] The playback unit 105 is configured to be able to play back conversation information. The conversation information includes audio recorded from a conversation between subjects. The conversation between subjects is, for example, a conversation between a parent and child. When the playback unit 105 plays back the conversation information, the selection information generation unit 102 associates a time with the conversation information. For example, the selection information includes a time based on the playback start time of the conversation information.

[0036] The conversation information may include a video of the subject. In this case, the video is displayed on the screen of the information processing device 2 or the information processing device 3 as the third display information.

[0037] The recording unit 106 is configured to be able to record the conversation information. The recording unit 106 is not necessarily required, and is used when creating conversation information or when operating the information processing device 3 in front of the subject.

[0038] 4. Screen display example Next, an example of the screen display of the information processing device 2 or the information processing device 3 will be described. FIG. 4 is a diagram showing an example of the screen display. As shown in the figure, a screen 200 displayed on the information processing device 2 or the information processing device 3 has a first display area 201, a second display area 202, and a third display area 203. The first display area 201 is an area where display based on the first display information generated by the display information generating unit 101 is performed, and the second display area 202 is an area where display based on the second display information generated by the display information generating unit 101 is performed. The third display area 203 is an area where display based on the third display information is performed when the playback unit 105 plays back a video.

[0039] In the first display area 201, a plurality of objects are displayed as shown in Fig. 5. Fig. 5 is a diagram showing a display example of the first area. As shown in Fig. 5, in the first display area 201, buttons 211, 212, 213, 214, 215, 216, 217, and 218 are displayed as objects. Note that the example shown here is used for learning when training PCIT trainers and therapists, and each button is assigned to an option of neutral conversation (TA), behavior explanation (BD), repetition (RF), specific praise (LP), general praise (UP), question (QU), command (CM), and negative conversation (NTA). These options can be classified into a first group: neutral conversation (TA), behavior description (BD), repetition (RF), specific praise (LP), and general praise (UP) in a second group, and questions (QU), commands (CM), and negative conversation (NTA) in a third group. Therefore, button 211 is displayed in a first color, buttons 212, 213, 214, and 215 are displayed in a second color, and buttons 216, 217, and 218 are displayed in a third color.

[0040] These buttons 211, 212, 213, 214, 215, 216, 217, and 218 are used by a selector (a learner or instructor) who, while listening to a conversation between the parent and child who are the subjects, classifies the parent's speech into one of eight types and presses the button corresponding to the classification result.

[0041] In the second display area 202, a display as shown in FIG. 6 is performed. FIG. 6 is a diagram showing a display example of the second display area. In the example shown in the figure, a symbol 221 is displayed on the time axis. This symbol 221 is displayed based on selection information generated when a selector (a learner or an instructor) presses a button displayed in the first display area 201. The example shown in FIG. 6 indicates that the button 212 was pressed at 14 seconds, 27 seconds, and 29 seconds. Moreover, the display in the second display area 202 can be scrolled horizontally, that is, in the direction along the time axis. Note that the example shown in FIG. 6 is displayed while the selector is classifying utterances or when the selector is checking only his / her own classification results.

[0042] The selector can also compare the classification results of the utterance with the classification results of others, in which case the second display area 202 displays as shown in Fig. 7 or 8. Fig. 7 shows the classification results of two selectors, and Fig. 8 shows the classification results of three selectors. In the example shown in the figure, the classification result of the selector (learner) given the identification information "S01" is represented by symbol 222, the classification result of the selector (instructor) given the identification information "T" is represented by symbol 223, and the classification result of the selector (learner) given the identification information "S02" is represented by symbol 224.

[0043] Incidentally, when displaying the classification results by a plurality of selectors, it is possible to display only the matching classification results, only the mismatching classification results, etc. In order to determine whether the classification results match or mismatch, it is necessary to identify the correspondence between the two. The correspondence between the two can be identified, for example, as shown in FIG. 9. FIG. 9 is a diagram showing an example of the correspondence between classification results. The classification result shown at time a in the figure matches both the selection contents and the selection time. In other words, the two match. The classification result shown at time b in the figure is mismatched because the selector of identification information "S01" has performed classification, but the selector of identification information "T" has not performed classification. The classification result shown at time c in the figure is mismatched because the selection times match but the selection contents are different. The classification result shown at time d in the figure is also mismatched. The classification result shown at time e in the figure is mismatched as in the case of time b. The classification results at time f and time f' in the figure differ in the selection times, but the difference is within an allowable range and the selection contents match, so the two are identified as matching. In the classification result shown at time b in the figure, the selector with identification information "T" has performed classification, but the selector with identification information "S01" has not performed classification, so the two do not match.

[0044] The correspondence of the classification results can be determined based on the distance between the two, for example, by using Dynamic Time Warping (DTW). Of course, it is also possible to use other methods, such as an algorithm for calculating the distance between words used in spell checking.

[0045] In the third display area 203, display is performed when the playback unit 105 operates. When a video is played, the video and objects for controlling the playback of the video, such as a seek bar indicating the playback position, a play button, a pause button, a 10 second rewind button, and a 10 second forward button, are displayed. When only audio is played, only the same objects are displayed.

[0046] 5. Operation of the information processing device 1 Next, a description will be given of the operation of the information processing device 1. Figures 10 and 11 are activity diagrams showing the flow of the operation of the information processing device 1.

[0047] First, the operation of the information processing device 1 when classifying utterances will be described. When the information processing device 1 starts its operation, if the conversation information is to be reproduced, the reproduction unit 105 starts reproduction of the conversation information (A101), and if the utterances are to be classified in front of the subject, the recording unit 106 starts recording of the conversation information (A102). Then, while the conversation continues, each time the selector selects an utterance classification, the selection information generating unit 102 generates selection information (A103). Then, when the conversation ends, the reproduction of the conversation information by the reproduction unit 105 or the recording of the conversation information by the recording unit 106 is stopped (A104), and the selection information generated by the selection information generating unit 102 is saved as a file (A105), and the process ends. The selection information saved as a file is associated with the conversation information, or saved as one file.

[0048] Next, the operation of the information processing device 1 when confirming the classification of an utterance will be described. When the information processing device 1 starts its operation, first, the determination unit 104 determines whether the classification result corresponds (A201), and based on the result, the display information generation unit 101 generates the second display information (A202). Note that the determination of the correspondence of the classification result may be omitted.

[0049] The flow of operations described here is just an example.

[0050] 5.Other Here, the case of carrying out learning conforming to PCIT has been described, but the information processing device 1 and the information processing device 3 can also be used for other purposes, such as for studying or reflecting on business negotiations or meetings.

[0051] The present invention may be provided in the following aspects. In the information processing device, the display information generation unit generates, based on the selection information, second display information in which symbols representing the options are arranged according to the times corresponding to the options. In the information processing device, the symbol is any one of a character, a character string, a sign, and an icon, and the second display information is an information processing device in which the symbols are arranged on a time axis. In the information processing device, the selection information includes information of a selector who selected the object, and the second display information is configured to distinguish the symbols into a first symbol corresponding to selection information generated by selection of a first selector and a second symbol corresponding to selection information generated by selection of a second selector. The information processing device further comprises a determination unit, the determination unit being configured to be able to determine a correspondence between selection information generated by a selection of the first selector and selection information generated by a selection of the second selector, and the second display information includes a display of the correspondence between the first symbol and the second symbol. In the information processing device, the second display information displays the first symbol and the second symbol, which do not correspond to selection information generated by selection of the first selector and selection information generated by selection of the second selector. The information processing device includes a playback unit, the playback unit is configured to be able to play back the conversation information, the conversation information includes audio recorded of the subject's conversation, and the time corresponds to the conversation information. In the information processing device, the conversation information includes a video recorded of the subject. The information processing device further comprises a recording unit, the recording unit being configured to be able to record the conversation information. A program that causes a computer to operate as an information processing device, the program causing the computer to function as the information processing device. Of course, this is not the case. [Explanation of symbols]

[0052] 1: Information processing equipment 2: Information processing equipment 3: Information processing equipment 4: Network 11: Processing section 12: Storage section 13:Temporary storage 14: External device connection 15: Communications Department 16: Communication bus 101:Display information generation section 102: Selection information generation unit 103: Storage section 104: Judgment section 105: Playback section 106: Recording section 200: Screen 201: First display area 202: Second display area 203: Third display area 211: Button 212: Button 213: Button 214: Button 215: Button 216: Button 217: Button 218: Button 221: Symbol 222: Symbol 223: Symbol 224: Symbol

Claims

1. An information processing device, A first display unit and a second display unit are provided, the first display unit generates first display information for displaying a plurality of objects associated with a plurality of utterance types, the second display unit generates second display information for displaying a symbol image in a lane assigned to the type of utterance corresponding to the selected object each time any one of the objects is selected in a display area in which a plurality of lanes assigned to each of the plurality of utterance types are arranged, and The lane indicates a time by a position in the extension direction, the display position of the symbol image corresponds to the time when the object is selected; Information processing device.

2. In the information processing device according to claim 1, the second display unit displays a first symbol image indicating a type of utterance selected by a first selector and a second symbol image indicating a type of utterance selected by a second selector, and when the selection contents of the first selector and the second selector match and the difference in the selection times is within an allowable range, displays the first symbol image and the second symbol image in a manner indicating that the selection results of the first selector and the second selector match. Information processing device.

3. In the information processing device according to claim 2, when the selection contents of the first selector and the second selector do not match, or a difference in selection time is outside an allowable range, the second display unit displays the first symbol image and the second symbol image in a manner indicating that the selection results of the first selector and the second selector do not match. Information processing device.

4. In the information processing device according to any one of claims 1 to 3, The symbol image is any one of a character, a character string, a symbol, and an icon. Information processing device.

5. 5. The information processing device according to claim 1, Equipped with a playback section, The playback unit is configured to be able to play back conversation information, The conversation information includes a recorded voice of the subject's conversation, The time corresponds to the conversation information. Information processing device.

6. 6. The information processing device according to claim 5, The conversation information includes a video recording of the subject. Information processing device.

7. 7. The information processing device according to claim 1, A recording unit is provided, The recording unit is configured to be able to record conversation information. Information processing device.

8. A program for causing a computer to operate as an information processing device, A computer that functions as the information processing device according to any one of claims 1 to 7. to make program.

Images