Information processing device, information processing system, information processing method, and information processing program

Abstract

To provide a technology capable of acquiring information about response of a recipient such as a participant to information transmitted by a transmitting person such as an instructor and information about a communication environment in information transmission such as a seminar using a network.SOLUTION: An information processing device according to one aspect includes a processing part for acquiring first information to store it in a storage part, a processing part for transmitting the first information to one or more other information processing devices, a processing part for receiving response information to the first information acquired by the other information processing devices to the first information, and communication information showing a communication state while the other information processing device receive the first information, a processing part for storing the response information and the communication information received from the other information processing devices in the storage part, a processing part for editing the response information, the communication information, and the first information to generate edition information, and a processing part for removing the response information corresponding to the communication information from the edition information when the communication information shows that a communication state gets worse than a reference value, and displaying the edition information without removing the response information when it shows that the communication state does not get worse.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] In recent years, online conferences via communication networks such as the Internet, known as web conferences, have become increasingly popular. A variety of products are available for implementing these web conferences (see, for example, Non-Patent Documents 1 and 2). Furthermore, web conferences are increasingly being used to hold training sessions, lectures, and the like between instructors and participants in remote locations. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Webex”, [online], [retrieved March 19, 2024], Internet<URL:https: / / www.webex.com / ja / index.html> [Non-patent document 2] “teams”, [online], [retrieved March 19, 2024], Internet<URL:https: / / www.microsoft.com / ja-jp / microsoft-teams / group-chat-software> Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when reviewing or improving the content of training, instructors often improve the training content by constantly checking and recording the reactions and expressions of participants. However, in training using web conferences, it is difficult to read the reactions and expressions of participants from the images on the terminal screen, making it difficult to record them. Furthermore, when the line becomes unstable during remote communication, it is difficult to determine whether the participants' reactions are due to the training or a communication problem.

[0005] This invention has been made with the above-mentioned circumstances in mind, and its purpose is to provide a technology that makes it possible to obtain information regarding the reaction of recipients, such as trainees, to information transmitted by a transmitter, such as an instructor, and information regarding the communication environment, when transmitting or transmitting information for training, etc., using a communication network. [Means for solving the problem]

[0006] An information processing device of one embodiment of the present invention is an information processing device that transmits information between one or more other information processing devices via a network, and includes: a first processing unit that acquires first information and stores it in a memory unit; a second processing unit that transmits the first information to the other information processing devices; a third processing unit that receives reaction information to the first information, each acquired by the other information processing devices, and communication information indicating the communication status while the other information processing devices are receiving the first information; a fourth processing unit that stores the reaction information and the communication information received from the other information processing devices in the memory unit; a fifth processing unit that edits the reaction information, the communication information, and the first information stored in the memory unit to generate edited information; and a sixth processing unit that, when the communication information indicates that the communication status has deteriorated from a reference value, deletes the reaction information corresponding to the communication information from the edited information, and, when the communication information indicates that the communication status has not deteriorated from the reference value, does not delete the reaction information corresponding to the communication information from the edited information and displays the edited information. [Effects of the Invention]

[0007] According to one aspect of the present invention, a technology can be provided that, in transmitting or transferring information such as in a training session using a communication network, enables a transmitter such as a lecturer to obtain information regarding the reaction of a receiver such as a student to the information transmitted, as well as information regarding the communication environment, thereby enabling the transmitter to grasp the receiver's reaction to the transmitted information and to review and improve the transmitted content. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram showing the configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of a lecturer terminal and a student terminal according to the first embodiment. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a lecturer terminal according to the first embodiment. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a student terminal according to the first embodiment. [Figure 5] 2 is a block diagram showing the hardware configuration of a server device according to the first embodiment. FIG. [Figure 6] FIG. 2 is a block diagram showing the functional configuration of a server device according to the first embodiment. [Figure 7] 10 is a flowchart showing a training motion process in the information processing system according to the first embodiment. [Figure 8] 10 is a flowchart showing a process of acquiring reaction information in a student terminal according to the first embodiment. [Figure 9] FIG. 10 is a diagram showing Example 1 of input information input to a student terminal according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing a second example of input information input to a student terminal according to the first embodiment. [Figure 11] 4A and 4B are diagrams showing facial expression images and facial expression information of students evaluated by the student terminal according to the first embodiment. [Figure 12] FIG. 4 is a diagram showing an example of communication information acquired by a student terminal according to the first embodiment. [Figure 13] FIG. 10 is a diagram showing a first display example of editing information generated by the instructor terminal according to the first embodiment. [Figure 14] 10 is a flowchart showing a process of generating editing information in the instructor terminal according to the first embodiment. [Figure 15] FIG. 10 is a diagram showing a second display example of editing information generated by the instructor terminal according to the first embodiment. [Figure 16]FIG. 10 is a schematic diagram showing the configuration of an information processing system according to a second embodiment. [Figure 17] 10 is a flowchart showing a training motion process in the information processing system according to the second embodiment. [Figure 18] FIG. 11 is a block diagram showing the functional configuration of a server device according to a third embodiment. [Figure 19] 11 is a flowchart showing a training motion process in the information processing system according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, components having the same function and configuration will be assigned the same reference numerals. In the embodiment described below, an information processing system using a client-server system or a peer-to-peer system will be described, in which information is transmitted between a client device or information processing device and one or more other client devices or information processing devices via a communication network. The information processing system will be described with reference to an example in which a training operation is performed in which information (e.g., training information) is transmitted from a client device used by a transmitter (e.g., a lecturer) to client devices used by one or more receivers (e.g., students).

[0010] 1. First embodiment An information processing system according to a first embodiment of the present invention will be described.

[0011] 1.1 Configuration example An example of the configuration of the information processing system of the first embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing the configuration of the information processing system of the first embodiment.

[0012] The information processing system includes a plurality of client devices (or information processing devices, information terminals) 1 and 2, a server device 3, and a communication network NW. The client devices 1 and 2 and the server device 3 are communicably connected via a communication network NW such as the Internet. Hereinafter, the client device owned by the instructor will be referred to as an instructor terminal 1, and the client device owned by the student will be referred to as a student terminal 2.

[0013] Next, an example of the hardware configuration of the instructor terminal 1 and the student terminal 2 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the hardware configuration of the instructor terminal 1 and the student terminal 2 in the first embodiment. The instructor terminal 1 and the student terminal 2 have, for example, the same hardware configuration.

[0014] 2, each of the instructor terminal 1 and the student terminal 2 is realized by a computer such as a PC (Personal Computer) or a smartphone. Each of the instructor terminal 1 and the student terminal 2 includes a processor 11, a program memory 12, a data memory 13, a data storage 14, an input / output interface 15, a communication interface 16, a bus 17, an operation device 21, a microphone 22, a facial expression capture camera 23, a display device 24, and a speaker 25. The processor 11, the program memory 12, the data memory 13, the data storage 14, the input / output interface 15, and the communication interface 16 are connected to each other via the bus 17 so as to be able to communicate with each other.

[0015] The processor 11 includes, for example, a CPU (Central Processing Unit). The processor 11 may be a multi-core / multi-thread processor, and can execute multiple processes in parallel.

[0016] The program memory 12 is a storage medium that combines a nonvolatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), with a nonvolatile memory such as a read-only memory (ROM). The program memory 12 stores programs necessary for the processor 11 to execute various processes. The programs include an operating system (OS) and various application programs. The application programs include a program for executing a training operation as an information processing program according to the first embodiment.

[0017] The data memory 13 is a general-purpose memory or a memory area capable of holding data for graphic processing. The data memory 13 includes a volatile memory such as a RAM (Random Access Memory) as a storage medium.

[0018] The data storage 14 is a storage area capable of holding data. The data storage 14 includes, as a storage medium, a nonvolatile memory such as an HDD or SSD that can be written to and read from as needed. The data storage 14 is used to store data created by executing application programs.

[0019] The input / output interface 15 is an interface with input / output devices. The input devices may include an operation device 21, a microphone 22, and a facial expression capturing camera 23. The output devices may include a display device 24 and a speaker 25.

[0020] The operation device 21 is a pointing device such as a keyboard, a mouse, etc. If the student terminal 2 is a smartphone, the operation device 21 corresponds to a touch panel or the like arranged on a screen such as a liquid crystal display.

[0021] The microphone 22 captures the speech of the instructor using the instructor terminal 1 or the student using the student terminal 2 .

[0022] The facial expression capturing camera 23 is installed so as to capture the face of the instructor or student within the capturing range, and captures an image of the appearance including the face of the instructor or student.

[0023] The display device 24 is a liquid crystal display or the like that displays information relating to the training transmitted and received between the instructor terminal 1 and the student terminal 2, such as training information, reaction information, communication information, or editing information.

[0024] The speaker 25 outputs the speech of the instructor or students participating in the training activity.

[0025] It is not essential to provide all of these input / output devices. Typically, in a web conference, an operation device 21, a microphone 22, a facial expression capturing camera 23, a display device 24, and a speaker 25 are used.

[0026] The communication interface 16 is a wired or wireless communication device for connecting the instructor terminal 1 or the student terminal 2 to the communication network NW.

[0027] Next, an example of the functional configuration of the instructor terminal 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the functional configuration of the instructor terminal 1 included in the information processing system of the first embodiment.

[0028] The instructor terminal 1 includes, as functional blocks, a control unit 110, a program storage unit 120, an information storage unit 130, a student information storage unit 140, an information input / output unit 150, and an information transmission / reception unit 160. The control unit 110 includes a training information acquisition unit 111 and an information editing unit 112. Furthermore, the information editing unit 112 includes an information deletion unit 112A.

[0029] The control unit 110 including the training information acquisition unit 111 and the information editing unit 112 corresponds to, for example, the processor 11 and the program memory 12. The program storage unit 120 corresponds to, for example, the program memory 12. The information storage unit 130 and the student information storage unit 140 correspond to, for example, the data memory 13 and the data storage 14. The information input / output unit 150 corresponds to, for example, the operation device 21, the microphone 22, the facial expression capture camera 23, the display device 24, and the speaker 25. The information transmission / reception unit 160 corresponds to, for example, the communication interface 16.

[0030] The training information acquisition unit 111 acquires training information input from the instructor via the information input / output unit 150. The information editing unit 112 edits the input information and reaction information including facial expression information of the student in response to the training information, and the communication information (and the training information) to generate edited information. The input information, facial expression information, reaction information, and communication information will be described in detail later.

[0031] The program storage unit 120 stores a program for executing a training action as an information processing program according to the first embodiment (hereinafter referred to as a training action program). The information storage unit 130 stores training information input by the instructor, reaction information and communication information received from the student terminals, and generated editing information. The student information storage unit 140 stores information about students who participate in the training action using the student terminals. The student information includes various information such as information identifying the training action, specific information identifying the student such as the student's identification code or name, and a password required to participate in the training action.

[0032] The information input / output unit 150 receives training information from the instructor and transmits the training information to the training information acquisition unit 111. The information input / output unit 150 also outputs edited information generated by the information editing unit 112 to the instructor. The information transmission / reception unit 160 transmits the training information to the student terminal via the communication network NW. The information transmission / reception unit 160 also receives reaction information and communication information from the student terminal via the communication network NW.

[0033] Next, an example of the functional configuration of the student terminal 2 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the student terminal 2 included in the information processing system of the first embodiment.

[0034] The student terminal 2 includes, as functional blocks, a control unit 210, a program storage unit 220, an information storage unit 230, an information input / output unit 240, and an information transmission / reception unit 250. The control unit 210 includes a reaction information acquisition unit 211 and a communication status acquisition unit 212. Furthermore, the reaction information acquisition unit 211 includes an input information acquisition unit 211A and a facial expression evaluation unit 211B.

[0035] The control unit 210 including the reaction information acquisition unit 211 and the communication status acquisition unit 212 corresponds to, for example, the processor 11 and the program memory 12. The program storage unit 220 corresponds to, for example, the program memory 12. The information storage unit 230 corresponds to, for example, the data memory 13 and the data storage 14. The information input / output unit 240 corresponds to, for example, the operation device 21, the microphone 22, the facial expression capture camera 23, the display device 24, and the speaker 25. The information transmission / reception unit 250 corresponds to, for example, the communication interface 16.

[0036] The input information acquisition unit 211A acquires input information input by the student in response to the training information. The facial expression evaluation unit 211B evaluates facial expression images obtained by capturing the facial expressions of the student in response to the training information, and generates facial expression information. The reaction information acquisition unit 211 acquires the input information acquired by the input information acquisition unit 211A and the facial expression information generated by the facial expression evaluation unit 211B as reaction information.

[0037] The communication status acquisition unit 212 measures the communication status while the student terminal 2 is receiving the training information, and generates communication information indicating the communication status.

[0038] The program storage unit 220 stores a training action program as an information processing program according to the first embodiment. The information storage unit 230 stores reaction information including input information and facial expression information, and communication information generated by the communication status acquisition unit 212.

[0039] The information input / output unit 240 outputs training information to the student. The information input / output unit 240 receives input information for the training information from the student, captures an image of the student's facial expression, and transmits the facial expression image to the facial expression evaluation unit 211B.

[0040] The information transmitting / receiving unit 250 receives training information from the instructor terminal via the communication network NW, and also transmits reaction information and communication information to the instructor terminal via the communication network NW.

[0041] Next, an example of the hardware configuration of the server device 3 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the hardware configuration of the server device 3 of the first embodiment.

[0042] The server device 3 is realized by a computer. The server device 3 includes a processor 31, a program memory 32, a data memory 33, a data storage 34, a communication interface 35, and a bus 36. The processor 31, the program memory 32, the data memory 33, the data storage 34, and the communication interface 35 are connected to each other via the bus 36 so as to be able to communicate with each other.

[0043] The processor 31 includes, for example, a CPU. The processor 31 may be a multi-core / multi-thread processor, and can execute multiple processes in parallel.

[0044] The program memory 32 is a storage medium that combines a nonvolatile memory that can be written to and read from at any time, such as an HDD or SSD, with a nonvolatile memory such as a ROM. The program memory 32 stores programs necessary for the processor 31 to execute various processes. The programs include an OS, a training operation program as an information processing program according to the first embodiment, and other server programs.

[0045] The data memory 33 is a general-purpose memory and includes, as a storage medium, a volatile memory such as a RAM.

[0046] The data storage 34 is a storage area capable of holding data. The data storage 34 includes, as a storage medium, a nonvolatile memory that can be written to and read from as needed, such as an HDD or SSD.

[0047] The communication interface 35 is a wired or wireless communication device for connecting the communication network NW and the server device 3.

[0048] Next, an example of the functional configuration of the server device 3 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the functional configuration of the server device 3 included in the information processing system of the first embodiment.

[0049] The server device 3 includes a control unit 310, a program storage unit 320, an information storage unit 330, and an information transmission / reception unit 340 as functional blocks.

[0050] The control unit 310 corresponds to, for example, the processor 31 and the program memory 32. The program storage unit 320 corresponds to, for example, the program memory 32. The information storage unit 330 corresponds to, for example, the data memory 33 and the data storage 34. The information transmission / reception unit 340 corresponds to, for example, the communication interface 35.

[0051] The control unit 310 controls the operations of the program storage unit 320, the information storage unit 330, and the information transmission / reception unit 340, and executes transmission and reception of information between the instructor terminal 1, the student terminal 2, and the server device 3.

[0052] The control unit 310 may also be implemented in the form of various other hardware circuits, including integrated circuits such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), etc.

[0053] The program storage unit 320 stores a training operation program as an information processing program according to the first embodiment. The information storage unit 330 stores training information, reaction information, and communication information received from the instructor terminal 1 or the student terminal 2.

[0054] The information transmitting / receiving unit 340 receives training information from the instructor terminal 1 via the communication network NW and transmits it to the student terminal 2. The information transmitting / receiving unit 250 also receives reaction information and communication information from the student terminal 2 via the communication network NW and transmits it to the instructor terminal 1.

[0055] The server device 3 performs operations to realize the training operation of the first embodiment, which is executed via the communication network NW, using the above-mentioned control unit 310, program memory unit 320, information memory unit 330, and information transmission / reception unit 340.

[0056] 1.2 Example of operation An example of operation of the information processing system of the first embodiment will be described with reference to Figures 3, 4, 6, and 7. As an example of operation, a case will be described in which the instructor transmits training information from the instructor terminal 1 to the student terminal 2, the students receive the training information via the student terminal 2, input reaction information to the training information into the student terminal 2, and the instructor receives the reaction information transmitted from the student terminal 2 to the instructor terminal 1. In other words, the instructor terminal 1 transmits the training information input by the instructor to the student terminal 2. The student terminal 2 outputs the received training information to the students. The student terminal 2 transmits the reaction information to the training information and communication information input by the students to the instructor terminal 1.

[0057] Fig. 7 is a flowchart showing the processing of training actions in the information processing system of the first embodiment. The processing of the flowchart shown in Fig. 7 is controlled by the processor 11 of the instructor terminal 1 and the student terminal 2, and the processor 31 of the server device 3. Furthermore, programs for executing training actions as information processing programs are loaded from the program storage units 120, 220, and 320 into the instructor terminal 1, the student terminal 2, and the server device 3, respectively. Information about students is read into the instructor terminal 1 from the student information storage unit 140. As a result, the instructor terminal 1, the student terminal 2, and the server device 3 are set to a state in which the training actions can be executed.

[0058] First, the instructor inputs training information to the information input / output unit 150 of the instructor terminal 1. Specifically, the information input / output unit 150 receives, for example, text information, image information, and audio information as training information from the instructor (S1). The training information acquisition unit 111 acquires the training information input to the information input / output unit 150. The information storage unit 130 stores the acquired training information in the training information acquisition unit 111 (S2). Furthermore, the training information acquisition unit 111 transmits the training information to the information transmission / reception unit 160.

[0059] The information transmitting / receiving unit 160 of the instructor terminal 1 transmits the training information to the server device 3 via the communication network NW (S3).

[0060] The information transmitting / receiving unit 340 of the server device 3 receives the training information from the instructor terminal 1 via the communication network NW. The information storage unit 330 stores the received training information. Furthermore, the information transmitting / receiving unit 340 transmits the training information to the student terminal 2 via the communication network NW (S4).

[0061] The information transmitting / receiving unit 250 of the student terminal 2 receives the training information from the server device 3 via the communication network NW (S5). The information transmitting / receiving unit 250 transmits the received training information to the information input / output unit 240.

[0062] The information input / output unit 240 of the student terminal 2 outputs the received training information to the student (S6). Specifically, the information input / output unit 240 displays text information and image information as the training information to the student, and outputs audio information.

[0063] The student receives the training information output from the information input / output unit 240 of the student terminal 2. That is, the student watches the training information output from the information input / output unit 240.

[0064] After that, the trainee who received the training information inputs his / her reaction to the training information, for example, information such as the level of understanding (understandable / incomprehensible) or confusion (hereinafter referred to as input information) to the information input / output unit 240. The information input / output unit 240 transmits the input information to the input information acquisition unit 211A of the reaction information acquisition unit 211. As a result, the input information acquisition unit 211A acquires the input information input by the trainee.

[0065] Furthermore, the information input / output unit 240 captures the facial expression of the student in response to the training information, and transmits the facial expression image of the student to the facial expression evaluation unit 211B of the reaction information acquisition unit 211. The facial expression evaluation unit 211B evaluates the received facial expression image and generates facial expression information.

[0066] The reaction information acquisition unit 211 of the student terminal 2 acquires the input information and facial expression information as reaction information to the training information. The information storage unit 230 stores the reaction information (S7). Furthermore, the reaction information acquisition unit 211 transmits the reaction information to the information transmission / reception unit 250. The reaction information includes the time when the reaction information was acquired, i.e., the time when the input information was entered by the student, and the time when the student's facial expression was photographed. The reaction information acquired by the reaction information acquisition unit 211 will be described in detail later.

[0067] The communication status acquisition unit 212 of the student terminal 2 measures the communication status while the information transmitting / receiving unit 250 is receiving the training information, and acquires communication information indicating the communication status. The information storage unit 230 stores the communication information (S8). Furthermore, the communication status acquisition unit 212 transmits the communication information to the information transmitting / receiving unit 250. Note that, although the case where reaction information is acquired first and then communication information is acquired has been described here, the order of these processes may be reversed, or these processes may be executed in parallel. The communication information acquired by the communication status acquisition unit 212 will be described in detail later.

[0068] The information transmitting / receiving section 250 of the student terminal 2 transmits the reaction information and communication information to the server device 3 via the communication network NW (S9).

[0069] The information transmitting / receiving unit 340 of the server device 3 receives the reaction information and communication information from the student terminal 2 via the communication network NW. The information storage unit 330 stores the received reaction information and communication information. Furthermore, the information transmitting / receiving unit 340 transmits the reaction information and communication information to the instructor terminal 1 via the communication network NW (S10).

[0070] The information transmitting / receiving unit 160 of the instructor terminal 1 receives the reaction information and the communication information from the server device 3 via the communication network NW. The information storage unit 130 stores the reaction information and the communication information (S11). Furthermore, the information transmitting / receiving unit 160 transmits the reaction information and the communication information to the information editing unit 112.

[0071] The information editing unit 112 of the instructor terminal 1 edits the reaction information, communication information, and training information to generate edited information. The information storage unit 130 stores the edited information (S12). The information editing unit 112 generates, for example, information in which the reaction information, communication information, and training information are arranged in chronological order as the edited information. Furthermore, if the communication information indicates that the communication situation has deteriorated below a reference value, the information deletion unit 112A of the information editing unit 112 deletes the reaction information if the reaction information exists within a predetermined time from the time the communication information was generated. On the other hand, if the communication information indicates that the communication situation has not deteriorated below a reference value, the information deletion unit 112A does not delete the reaction information. Furthermore, the information editing unit 112 transmits the edited information to the information input / output unit 150. The edited information edited by the information editing unit 112 will be described in detail later.

[0072] The information input / output unit 150 of the instructor terminal 1 outputs the edited information to the instructor (S13). Specifically, the information input / output unit 150 displays, as the edited information, for example, information in which the reaction information and the communication information are arranged in chronological order.

[0073] Next, the control unit 110 determines whether the training operation is to be ended (S14). If it is determined that the training operation is not to be ended (No), the control unit 110 proceeds to the processing of step S1. If it is determined that the training operation is to be ended (Yes), the control unit 110 ends the training operation.

[0074] In the above description, the server device 3 transmits, receives, and stores training information, reaction information, and communication information between the instructor terminal 1 and the student terminal 2, but may transmit and receive only training information, reaction information, and communication information. Furthermore, the reaction information acquisition unit 211 acquires both input information and facial expression information as reaction information, but may acquire either the input information or the facial expression information as reaction information. The information transmission / reception unit 250 may transmit reaction information including either the input information or the facial expression information to the instructor terminal 1.

[0075] The sending and receiving of reaction information and communication information, the generation of edited information, and the output of edited information in steps S9 to S13 may be performed in real time while the training information, reaction information, and communication information are being sent and received, or may be performed at regular intervals. Alternatively, the generation and output of edited information in steps S12 and S13 may be performed after the training operation has ended.

[0076] Next, with reference to FIGS. 8 to 11, the reaction information acquired by the reaction information acquisition unit 211 of the student terminal 2 and an example of its input (or display) will be described in detail.

[0077] The students view the training information output from the student terminal 2. Then, the students' reactions and facial expression images generated by viewing the training information are input to the student terminal 2 by the operation device 21 such as a keyboard or mouse, and the facial expression capturing camera 23, respectively. The students' reactions input by the operation device 21 are acquired as input information by the input information acquiring unit 211A. The facial expression images of the students input by the facial expression capturing camera 23 are acquired as facial expression information by the facial expression evaluating unit 211B. Reaction information including the input information and facial expression information is acquired by the reaction information acquiring unit 211.

[0078] 8 is a flowchart showing the process of acquiring reaction information in the student terminal 2. The process of the flowchart shown in FIG.

[0079] First, the information input / output unit 240 of the student terminal 2 receives input from the student regarding the training information via the operation device 21 (S21). The input information acquisition unit 211A acquires the input from the student entered into the information input / output unit 240 as input information (S22).

[0080] The information input / output unit 240 of the student terminal 2 captures the facial expression of the student in response to the training information using the facial expression capture camera 23, and transmits the facial expression image to the facial expression evaluation unit 211B (S23). The facial expression evaluation unit 211B evaluates the facial expression image of the student and acquires facial expression information (S24).

[0081] The reaction information acquisition unit 211 acquires reaction information including input information and facial expression information (S25). The information storage unit 230 stores the reaction information (S26).

[0082] Next, the reaction information acquisition unit 211 determines whether the reaction information acquisition process is to be terminated (S27). If it is determined that the acquisition process is not to be terminated (No), the reaction information acquisition unit 211 proceeds to the process of step S21. If it is determined that the acquisition process is to be terminated (Yes), the reaction information acquisition unit 211 terminates the acquisition process.

[0083] FIG. 9 shows Example 1 of input information entered by a student. This input information is displayed on the display screen of the display device 24 of the student terminal 2 as text data in a format such as CSV (Comma Separated Values), with the following items displayed from left to right: "Elapsed time (or time) since the start of the training action," "Response category," and "Intensity." The "Response category" represents the student's level of understanding and reflection regarding the training information. The "Intensity" represents the strength of the response in the "Response category." For example, as shown in FIG. 9, the input information is displayed on the display screen of the display device 24 as "10:00, don't understand, 30."

[0084] Fig. 10 shows Example 2 of input information input by a student. As shown in Fig. 10, a graphical user interface (GUI) is displayed on the display screen of the display device 24 of the student terminal 2. The student selects input information such as the level of understanding of the training information or thoughts from the GUI and inputs it using the operation device 21 such as a mouse or keyboard.

[0085] 11 shows an example of a facial expression image and facial expression information of a student evaluated by the facial expression evaluation unit 211B. The facial expression capturing camera 23 of the student terminal 2 captures the student's facial expression in response to the training information and transmits the facial expression image of the student to the facial expression evaluation unit 211B. The facial expression evaluation unit 211B performs image analysis of, for example, the positions of the corners of the mouth and eyebrows in the facial expression image of the student, and obtains the student's feelings, such as impatience or boredom, as facial expression information.

[0086] The reaction information including the input information and facial expression information shown in Figures 9 to 11 above is transmitted to the instructor terminal 1 in real time or at a predetermined timing, edited by the information editing unit 112, and displayed on the display screen of the display device 24 of the instructor terminal 1.

[0087] Next, the communication information acquired by the communication status acquisition section 212 of the student terminal 2 will be described in detail with reference to FIG.

[0088] The training information transmitted from the instructor terminal 1 is received by the student terminal 2. When the training information is received by the student terminal 2, a communication problem such as a delay in receiving the training information or a momentary line interruption (instantaneous disconnection) may occur. This may result in, for example, a delay or instantaneous disconnection in the display of text information and image information, or in the output of audio information.

[0089] FIG. 12 shows an example of communication information acquired by the communication status acquisition unit 212 of the student terminal 2. This communication information is displayed as text data in CSV format or the like on the display screen of the display device 24 of the student terminal 2, and includes, from left to right, "time elapsed since the start of the training operation (or time)," "delay time," and "connection status." "Delay time" indicates the delay time that occurred when the student terminal 2 was receiving the training information. "Connection status" indicates the connection status of communication between the instructor terminal 1 and the student terminal 2. For example, as shown in FIG. 12, the communication information is displayed as "10:00, 100 ms, connecting" on the display screen of the display device 24.

[0090] The above-mentioned communication information is transmitted to the instructor terminal 1 in real time or at a predetermined timing, and is edited by the information editing unit 112.

[0091] Next, the edited information generated by the information editing unit 112 of the instructor terminal 1 and its display example will be described in detail with reference to FIGS.

[0092] Reaction information and communication information in response to the training information are transmitted from the student terminal 2 to the instructor terminal 1 and stored in the information storage unit 130. The information editing unit 112 of the instructor terminal 1 edits the reaction information and communication information received from the student terminal 2 and the training information stored in the information storage unit 130 to generate edited information. Alternatively, the information editing unit 112 edits the reaction information, communication information, and training information stored in the information storage unit 130 to generate edited information. The edited information is displayed on the display screen of the display device 24 of the instructor terminal 1 and provided to the instructor.

[0093] FIG. 13 shows a display example 1 of edited information generated by the information editing unit 112. In display example 1, reaction information is shown, for example, by reaction icons R1, R2, and R3. R1 indicates a good reaction. R2 indicates an average reaction, i.e., a reaction that is not as good as R1. Furthermore, R3 indicates a bad reaction, i.e., a reaction that is not as good as R2. Communication information is shown, for example, by a communication icon X. The communication icon X indicates that the communication situation has deteriorated compared to the reference value. In other words, the time when the communication icon X is added indicates that the communication situation has deteriorated compared to the reference value.

[0094] In display example 1, training images as training information, and reaction information and communication information of multiple students are arranged in accordance with the elapsed time from the start of the training operation. That is, reaction icons R1, R2, and R3 as reaction information and a communication icon X as communication information are arranged in chronological order for each of student terminals 2A, 2B, and 2C on the display screen of display device 24 of instructor terminal 1. Furthermore, a timeline of training images corresponding to the reaction information is displayed.

[0095] While FIG. 13 shows one example of editing information, other editing can be performed on the training images, reaction information, communication information, and student terminals. Furthermore, the training images are images showing the state of the training, and may be still images corresponding to the elapsed time, or videos from the elapsed time to a predetermined time. When there are multiple students, i.e., when multiple student terminals 2 are operating, the reaction information acquired from the multiple student terminals 2 may be displayed with emphasis on the reaction information (e.g., reaction icons) of the elapsed time where the same reactions are prevalent. Furthermore, the information editing unit 112 of the instructor terminal 1 may acquire other information of the student terminal 2 as communication information, such as the microphone mute status and the camera on / off status, and display this information.

[0096] Next, a function of the information deletion unit 112A of the information editing unit 112 to delete reaction information generated when communication conditions deteriorate will be described.

[0097] 14 is a flowchart showing an example of a process for generating edit information in the instructor terminal 1. The process of the flowchart shown in FIG.

[0098] First, the information editing unit 112 of the instructor terminal 1 edits the reaction information, communication information, and training information to generate edited information (S31). The information storage unit 130 stores the reaction information, communication information, training information, and edited information (S32).

[0099] Next, the information editing unit 112 determines whether the communication information indicates that the communication situation has deteriorated below the reference value (S33). If the communication situation indicates that the communication situation has deteriorated below the reference value (Yes), the information deletion unit 112A deletes the reaction information corresponding to the communication information (S34). Then, the process proceeds to step S35. On the other hand, if the communication situation indicates that the communication situation has not deteriorated below the reference value (No), the reaction information corresponding to the communication information is not deleted, and the process proceeds to step S35.

[0100] In step S35, the information input / output unit 150 outputs the edited information through the display device 24. That is, the edited information is displayed on the display screen of the display device 24 of the instructor terminal 1.

[0101] Next, the information editing unit 112 determines whether the process of generating edited information is to be terminated (S36). If it is determined that the process of generating edited information is not to be terminated (No), the information editing unit 112 proceeds to the process of step S31. If it is determined that the process of generating edited information is to be terminated (Yes), the information editing unit 112 terminates the process of generating edited information.

[0102] In addition, the determination of whether the communication conditions have deteriorated below the reference value is made when the packet loss rate of the training information is greater than the judgment value or when the communication delay time is greater than the judgment value in the communication of training information from the instructor terminal 1 to the student terminal 2.

[0103] Furthermore, in the above-described process for generating edited information, in step S33, a process is executed to determine whether the communication information indicates that the communication conditions have deteriorated below a reference value, and in step S34, a process is executed to delete the reaction information corresponding to the communication information if the communication conditions indicate that the communication conditions have deteriorated below a reference value. However, it is also possible to skip the processes of steps S33 and S34 and move on to the process of displaying the edited information in step S35.

[0104] In the above-described editing information generation process, the editing information is generated in step S31, and then steps S33 and S34 are executed, but the order of steps S31, S33, and S34 may be reversed. That is, the communication information determination in step S33 and the reaction information deletion in step S34 may be executed first, and then the editing information generation process in step S31 may be executed.

[0105] (a) and (b) of FIG. 15 show a second display example of edited information in which part of the reaction information has been deleted by the information deletion unit 112A.

[0106] As shown in the figure, the reaction information (reaction icons R1, R2, R3) and communication information (communication icons X1, X2) of the student terminal 2A are arranged along the elapsed time from the start of the training operation. In the display example shown in Figure 15(a), reaction icon R3 exists between the occurrence time of communication icon X2 and a predetermined time.

[0107] The information deletion unit 112A of the information editing unit 112 deletes this reaction icon R3. Fig. 15(b) shows a display example in which the reaction icon R3 has been deleted. In the display example shown in Fig. 15(b), the reaction icon R3 that was located immediately after the communication icon X2 has been deleted.

[0108] The editing information shown in the above-mentioned display example 1 and display example 2 is displayed on the display device 24 of the instructor terminal 1 and provided to the instructor.

[0109] 1.3 Effects etc. According to the first embodiment of the present invention, it is possible to provide a technology that can obtain information on the reaction of a recipient, such as a student, to information transmitted by a transmitter, such as a lecturer, and information on the communication environment, when transmitting or transferring information for training or the like using a communication network. This enables the transmitter to understand the reaction of the recipient to the transmitted information, and to review and improve the transmitted content.

[0110] The effects of the first embodiment will be described in detail below.

[0111] In the configuration of the first embodiment, the student directly inputs reaction information (i.e., input information) indicating the student's reaction to the training information into the student terminal 2, and the instructor terminal 1 receives and displays the reaction information from the student terminal 2. This allows the instructor to accurately know the student's reaction to the training information.

[0112] Furthermore, in the configuration of the first embodiment, the communication status is measured while the student terminal 2 is receiving the training information, and the communication information indicating the communication status is transmitted to the instructor terminal 1 together with the student's reaction information to the training information. The instructor terminal 1 displays the reaction information and communication information (and the training information corresponding to the reaction information) in chronological order. This allows the instructor, when the reaction information indicates a poor reaction, to determine whether this is due to a deterioration in the communication status or the training information.

[0113] Furthermore, when the communication information indicates that the communication conditions have deteriorated below the reference value, the instructor terminal 1 deletes the reaction information corresponding to this communication information. This allows the instructor to eliminate the reaction information of the students that occurred when the communication conditions were deteriorating. Therefore, the instructor can know only the reactions of the students to the training information, excluding the reaction information due to the deterioration of the communication conditions.

[0114] Furthermore, in the configuration of the first embodiment, the facial expression of the student in response to the training information is captured by the student terminal 2, reaction information (i.e., facial expression information) relating to the facial expression image of the student is generated, and the generated reaction information is transmitted to the instructor terminal 1. As a result, even if the student does not directly input reaction information into the student terminal 2, the instructor can know the student's reaction to the training information from the reaction information obtained from the facial expression image of the student.

[0115] As described above, the instructor can accurately grasp the students' reactions to the training information by using the edited information including the reaction information and communication information (and the training information corresponding to the reaction information) output and displayed from the instructor terminal 1. Furthermore, the edited information output and displayed from the instructor terminal 1 allows the instructor to accurately review and improve the training information (or training content).

[0116] 2. Second embodiment An information processing system according to a second embodiment of the present invention will now be described. In the first embodiment described above, an information processing system using a client-server system was described as an example, but in this second embodiment, an information processing system using a peer-to-peer system without a server device will be described. The following mainly describes the differences from the first embodiment.

[0117] 2.1 Configuration example An example of the information processing system of the second embodiment will be described with reference to Fig. 16. Fig. 16 is a schematic diagram showing the configuration of the information processing system of the second embodiment.

[0118] The information processing system includes a plurality of information processing devices (or information terminals) 4 and 5, and a communication network NW. The plurality of information processing devices 4 and 5 are communicably connected via the communication network NW. Hereinafter, the information processing device owned by the instructor will be referred to as an instructor terminal 4, and the information processing device owned by the student will be referred to as a student terminal 5.

[0119] Each of the instructor terminal 4 and the student terminal 5 is a computer such as a PC or a smartphone, and has the same hardware configuration as the instructor terminal 1 or the student terminal 2 shown in Fig. 2. The functional configuration of each of the instructor terminal 4 and the student terminal 5 is almost the same as the configuration shown in Figs. 3 and 4, respectively.

[0120] 2.2 Example of operation An example of the operation of the information processing system of the second embodiment will be described with reference to Figures 3, 4, and 17. As in the first embodiment, the case will be described in which the instructor transmits training information from the instructor terminal 4 to the student terminal 5, the students receive the training information via the student terminal 5, input reaction information to the training information into the student terminal 5, and the instructor receives the reaction information transmitted from the student terminal 5 to the instructor terminal 4.

[0121] Fig. 17 is a flowchart showing the processing of training actions in the information processing system of the second embodiment. The processing of the flowchart shown in Fig. 17 is controlled by the processors 11 of the instructor terminal 4 and the student terminal 5. Furthermore, training action programs as information processing programs are loaded from the program storage units 120 and 220 into the instructor terminal 4 and the student terminal 5, respectively. Information about students is read into the instructor terminal 4 from the student information storage unit 140. As a result, the instructor terminal 4 and the student terminal 5 are set to a state in which training actions can be executed.

[0122] First, the information input / output unit 150 of the instructor terminal 4 receives training information, such as text information, image information, and audio information, from the instructor (S1). The training information acquisition unit 111 acquires the training information input to the information input / output unit 150. The information storage unit 130 stores the acquired training information in the training information acquisition unit 111 (S2).

[0123] The information transmitting / receiving unit 160 of the instructor terminal 4 transmits the training information to the student terminal 5 via the communication network NW (S3).

[0124] The information transmitting / receiving unit 250 of the student terminal 5 receives the training information from the instructor terminal 4 via the communication network NW (S5).

[0125] The information input / output unit 240 of the student terminal 5 outputs the received training information to the student (S6).

[0126] The student receives the training information output from the information input / output unit 240 of the student terminal 5. That is, the student watches and listens to the training information output from the information input / output unit 240.

[0127] After that, the trainee who received the training information inputs his / her reaction to the training information, for example, information such as the level of understanding (understandable / incomprehensible) or confusion (hereinafter referred to as input information) to the information input / output unit 240. The information input / output unit 240 transmits the input information to the input information acquisition unit 211A of the reaction information acquisition unit 211. As a result, the input information acquisition unit 211A acquires the input information input by the trainee.

[0128] Furthermore, the information input / output unit 240 captures the facial expression of the student in response to the training information, and transmits the facial expression image of the student to the facial expression evaluation unit 211B of the reaction information acquisition unit 211. The facial expression evaluation unit 211B evaluates the received facial expression image and generates facial expression information.

[0129] The reaction information acquisition unit 211 of the student terminal 5 acquires the input information and facial expression information as reaction information to the training information. The information storage unit 230 stores the reaction information (S7). The reaction information includes the time when the reaction information was acquired, i.e., the time when the input information was entered by the student, and the time when the student's facial expression was photographed. The reaction information acquired by the reaction information acquisition unit 211 is the same as that described in the first embodiment.

[0130] The communication status acquisition unit 212 of the student terminal 5 measures the communication status while the information transmitting / receiving unit 250 is receiving the training information, and acquires communication information indicating the communication status. The information storage unit 230 stores the communication information (S8). Note that the order of the processes for acquiring the reaction information and the communication information may be reversed, or these processes may be executed in parallel. The communication information acquired by the communication status acquisition unit 212 is the same as that described in the first embodiment.

[0131] The information transmitting / receiving unit 250 of the student terminal 5 transmits the reaction information and communication information to the instructor terminal 4 via the communication network NW (S9).

[0132] The information transmitting / receiving unit 160 of the instructor terminal 4 receives the reaction information and communication information from the student terminal 5 via the communication network NW. The information storage unit 130 stores the reaction information and communication information (S11). Furthermore, the information transmitting / receiving unit 160 transmits the reaction information and communication information to the information editing unit 112.

[0133] The information editing unit 112 of the instructor terminal 4 edits the reaction information, communication information, and training information to generate edited information. The information storage unit 130 stores the edited information (S12). The information editing unit 112 generates, for example, information in which the reaction information, communication information, and training information are arranged in chronological order as the edited information. Furthermore, if the communication information indicates that the communication situation has deteriorated below a reference value, the information deletion unit 112A of the information editing unit 112 deletes the reaction information if the reaction information exists within a predetermined time from the time the communication information was generated. On the other hand, if the communication information indicates that the communication situation has not deteriorated below a reference value, the information deletion unit 112A does not delete the reaction information. The edited information edited by the information editing unit 112 is the same as that described in the first embodiment.

[0134] The information input / output unit 150 of the instructor terminal 4 outputs the edited information to the instructor (S13). Specifically, the information input / output unit 150 displays, as the edited information, for example, information in which the reaction information and the communication information are arranged in chronological order.

[0135] Next, the control unit 110 determines whether the training operation is to be ended (S14). If it is determined that the training operation is not to be ended (No), the control unit 110 proceeds to the processing of step S1. If it is determined that the training operation is to be ended (Yes), the control unit 110 ends the training operation.

[0136] As described above, the differences from the training motion processing described in the first embodiment are as follows.

[0137] In the first embodiment, in steps S3, S4, and S5, the information transmitting / receiving unit 160 of the instructor terminal 1 transmits the training information to the student terminal 2 via the communication network NW and the server device 3. In the second embodiment, in steps S3 and S5, the information transmitting / receiving unit 160 of the instructor terminal 4 transmits the training information to the student terminal 5 via the communication network NW without passing through the server device 3.

[0138] In the first embodiment, in steps S9, S10, and S11, the information transmitting / receiving unit 250 of the student terminal 2 transmits the reaction information and communication information to the instructor terminal 1 via the communication network NW and the server device 3. In the second embodiment, in steps S9 and S11, the information transmitting / receiving unit 250 of the student terminal 5 transmits the reaction information and communication information to the instructor terminal 4 via the communication network NW, without passing through the server device 3.

[0139] Other processes are almost the same as the processes of the training operations described in the first embodiment. In this way, the information processing system of this embodiment can be applied even when a peer-to-peer system is used in which information is transmitted and received between the instructor terminal 4 and the student terminal 5 without going through a server device.

[0140] 3. Third embodiment An information processing system according to a third embodiment of the present invention will be described. In the first embodiment described above, the instructor terminal 1 includes the information editing unit 112, but in this third embodiment, a case will be described in which the server device 3 includes the information editing unit 112. The following mainly describes the differences from the first embodiment.

[0141] 3.1 Configuration example An example of the functional configuration of the server device 3 will be described with reference to Fig. 18. Fig. 18 is a block diagram showing the functional configuration of the server device 3 included in the information processing system of the third embodiment.

[0142] The server device 3 includes, as functional blocks, a control unit 310, a program storage unit 320, an information storage unit 330, and an information transmission / reception unit 340. The control unit 310 has an information editing unit 112 including an information deletion unit 112A.

[0143] 3.2 Example of operation 19 is a flowchart showing the processing of a training action in the information processing system of the third embodiment. The processing of the flowchart shown in FIG. 19 is controlled by the processor 11 of the instructor terminal 1 and the student terminal 2, and the processor 31 of the server device 3.

[0144] Below, differences from the training motion processing described in the first embodiment will be described.

[0145] The information transmitting / receiving unit 340 of the server device 3 receives the reaction information and communication information from the student terminal 2 via the communication network NW. The information storage unit 330 stores the received reaction information and communication information (S10A). The information transmitting / receiving unit 340 transmits the reaction information and communication information to the information editing unit 112 of the server device 3.

[0146] The information editing unit 112 of the server device 3 edits the reaction information, communication information, and training information to generate edited information. The information storage unit 330 stores the edited information (S12A). Furthermore, if the communication information indicates that the communication conditions have deteriorated below a reference value, the information deletion unit 112A of the information editing unit 112 deletes the reaction information if the reaction information exists within a predetermined time from the time the communication information was generated. On the other hand, if the communication information indicates that the communication conditions have not deteriorated below the reference value, the information deletion unit 112A does not delete the reaction information. The information editing unit 112 transmits the edited information to the information transmission / reception unit 340. Furthermore, the information transmission / reception unit 340 transmits the edited information to the instructor terminal 1 via the communication network NW (S12B).

[0147] The information transmitting / receiving unit 160 of the instructor terminal 1 receives the editing information from the server device 3 via the communication network NW. The information storage unit 130 stores the editing information (S12C). The information transmitting / receiving unit 160 transmits the editing information to the information input / output unit 150. The information input / output unit 150 of the instructor terminal 1 outputs the editing information to the instructor (S13). Other processes are substantially the same as the processes of the training actions described in the first embodiment.

[0148] In the third embodiment, the server device 3 includes an information editing unit 112. Therefore, the processing of the information editing unit 112 that has been performed by the instructor terminal 1 can be reduced, and the load on the instructor terminal 1 can be reduced.

[0149] In the above description, the server device 3 is provided with the information editing unit 112, but this is not limiting. The server device 3 may be configured to include at least one of the information editing unit 112, the training information acquisition unit 111, and the reaction information acquisition unit 211. For example, the server device 3 may be configured to include three units: the information editing unit 112, the training information acquisition unit 111, and the reaction information acquisition unit 211. With this configuration, the processing performed by the instructor terminal 1 and the student terminal 2 can be reduced, thereby reducing the load on the instructor terminal 1 and the student terminal 2.

[0150] 4.Other The functional blocks described in the above embodiments can be realized as either hardware or computer software, or a combination of both. It is not essential that the functional blocks be distinguished as in the above example. For example, some functions may be performed by functional blocks other than the illustrated functional blocks. Furthermore, the illustrated functional blocks may be further divided into smaller functional sub-blocks.

[0151] Furthermore, the flow of each operation process described with reference to the flowchart is not limited to the order described, and the order of some processes may be changed, some processes may be performed simultaneously in parallel, or the content of some processes may be modified.

[0152] Furthermore, the information processing program according to each embodiment can be stored as a processing program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be transmitted and distributed via a communication medium. The program stored on the recording medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by having the operation controlled by this software means. The recording medium referred to in this specification is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a communication network NW.

[0153] It should be noted that the present invention is not limited to the above-described embodiment, and can be practiced in various modified forms without departing from the spirit and scope of the present invention.

[0154] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0155] 1...Client device (teacher terminal) 2...Client device (student terminal) 3. Server equipment 4...Information processing device (instructor terminal) 5...Information processing device (student terminal) 11, 31...processor 12, 32...Program memory 13, 33...Data memory 14, 34…Data storage 15...Input / output interface 16, 35...Communication interface 17, 36... Bus 21...Operating device 22...Microphone 23...Facial expression camera 24…Display device 25...Speaker 110...Control unit 111...Training Information Acquisition Department 112... Information Editorial Department 112A... Information deletion section 120...Program memory section 130...Information storage unit 140... Participant information storage unit 150...Information input / output section 160...Information transmission and reception unit 210...Control unit 211...Reaction information acquisition unit 211A...Input information acquisition unit 211B...Facial Expression Evaluation Unit 212...Communication status acquisition unit 220...Program memory unit 230...Information storage unit 240...Information input / output section 250...Information transmission and reception unit 310...Control unit 320...Program memory unit 330...Information storage unit 340...Information transmission and reception unit

Claims

1. An information processing device that transmits information to and from one or more other information processing devices via a network, a first processing unit that acquires first information and stores it in a storage unit; a second processing unit that transmits the first information to the other information processing device; a third processing unit that receives reaction information to the first information, which is acquired in each of the other information processing devices, and communication information indicating a communication status while the other information processing devices are receiving the first information; a fourth processing unit that stores the reaction information and the communication information received from the other information processing devices in the storage unit; a fifth processing unit that edits the reaction information, the communication information, and the first information stored in the storage unit to generate edited information; a sixth processing unit that deletes the reaction information corresponding to the communication information from the edited information when the communication information indicates that the communication situation has deteriorated compared to a reference value, and that does not delete the reaction information corresponding to the communication information from the edited information when the communication information indicates that the communication situation has not deteriorated compared to the reference value, and displays the edited information; An information processing device comprising:

2. the reaction information includes input information and facial expression information; the input information is information that is input to the input devices of the other information processing devices when the first information is output from the output devices of the other information processing devices, the facial expression information is information generated by evaluating an image of a facial expression of a person using each of the other information processing devices when the first information is output from an output device of each of the other information processing devices, The information processing device according to claim 1 .

3. An information processing system that connects a first information processing device and a second information processing device via a network and transmits information between the first information processing device and the second information processing device, The first information processing device a first processing unit that acquires first information and stores it in a storage unit; a second processing unit that transmits the first information to the second information processing device; Equipped with The second information processing device a third processing unit that outputs the first information received from the first information processing device from an output device; a fourth processing unit that transmits, to the first information processing device, reaction information to the first information acquired by an input device and communication information indicating a communication status while the first information is being received; Equipped with The first information processing device a fifth processing unit that stores the reaction information and the communication information received from the second information processing device in the storage unit; a sixth processing unit that edits the reaction information, the communication information, and the first information stored in the storage unit to generate edited information; a seventh processing unit that deletes the reaction information corresponding to the communication information from the edited information when the communication information indicates that the communication situation has deteriorated compared to a reference value, and that does not delete the reaction information corresponding to the communication information from the edited information when the communication information indicates that the communication situation has not deteriorated compared to the reference value, and displays the edited information; An information processing system comprising:

4. The second information processing device an eighth processing unit that acquires input information input to the input device when the first information is output from the output device; a ninth processing unit that, when the first information is output from the output device, evaluates an image of a facial expression of a person using the second information processing device and generates facial expression information; Equipped with The reaction information includes the input information and the facial expression information. The information processing system according to claim 3 .

5. An information processing method executed by an information processing device that transmits information to one or more other information processing devices via a network, comprising: acquiring first information and storing it in a storage unit; transmitting the first information to the other information processing device; receiving reaction information to the first information, which is acquired by each of the other information processing devices, and communication information indicating a communication status while the other information processing devices are receiving the first information; storing the reaction information and the communication information received from the other information processing devices in the storage unit; editing the reaction information, the communication information, and the first information stored in the storage unit to generate edited information; When the communication information indicates that the communication situation has deteriorated compared to a reference value, deleting the reaction information corresponding to the communication information from the edited information, and when the communication information indicates that the communication situation has not deteriorated compared to the reference value, not deleting the reaction information corresponding to the communication information from the edited information and displaying the edited information; An information processing method including:

6. An information processing method executed by an information processing system that connects a first information processing device and a second information processing device via a network and transmits information between the first information processing device and the second information processing device, The first information processing device acquiring first information and storing it in a storage unit; transmitting the first information to the second information processing device; Run The second information processing device outputting the first information received from the first information processing device from an output device; transmitting, to the first information processing device, reaction information to the first information acquired by an input device and communication information indicating a communication status while receiving the first information; Run The first information processing device storing the reaction information and the communication information received from the second information processing device in the storage unit; editing the reaction information, the communication information, and the first information stored in the storage unit to generate edited information; When the communication information indicates that the communication situation has deteriorated compared to a reference value, deleting the reaction information corresponding to the communication information from the edited information, and when the communication information indicates that the communication situation has not deteriorated compared to the reference value, not deleting the reaction information corresponding to the communication information from the edited information and displaying the edited information; An information processing method that performs the above.

7. The second information processing device acquiring input information input to the input device when the first information is output from the output device; generating facial expression information by evaluating an image of a facial expression of a person using the second information processing device when the first information is output from the output device; Run The reaction information includes the input information and the facial expression information. The information processing method according to claim 6.

8. 5. An information processing program that causes processors provided in the first information processing device and the second information processing device to execute processes performed by the first information processing device and the second information processing device of the information processing system according to claim 3.

Images