Information processing device

Abstract

To provide an information processing device that, when a plurality of participants perform a predetermined body movement in common, improves the efficiency of other participants' responses.SOLUTION: In an information processing system, an information processing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 receives captured images of participants 4 captured by the imaging device 30. The control unit 13 detects a predetermined body movement from the received captured images of the participants 4, and records captured images of participants 4 from which the predetermined body movement was detected in the storage unit 12. The control unit 13 transmits information corresponding to the detected predetermined body movement to terminal devices 20 used by participants other than the participants who performed the predetermined body movement.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device. [Background technology]

[0002] There is a known technology for transmitting information corresponding to the physical movements of a student. For example, Patent Literature 1 discloses a technology for evaluating the concentration level of a student from the physical movements of the student in a remote class or the like and notifying the instructor of the evaluation result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-167631 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is room for improvement in the efficiency of processing information corresponding to the trainee's physical movements.

[0005] In consideration of such circumstances, an information processing device that makes it possible for recipients to respond more efficiently is disclosed. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure is an information processing device having a communication unit and a control unit that communicates via the communication unit, and when the control unit detects specified physical movements of multiple students from captured images of the multiple students, it transmits information corresponding to the specified physical movements to a terminal device used by a student other than the multiple students. Effect of the Invention

[0007] According to an embodiment of the present disclosure, it is possible to make the recipient's response more efficient. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic configuration example of an information processing system according to an embodiment. [Diagram 2] 11 is a flowchart illustrating an example of the operation of the information processing device. [Diagram 3] FIG. 13 is a schematic diagram showing an example of a physical movement of a participant. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described. <Overview of the embodiment> 1 is a block diagram showing a schematic configuration example of an information processing system 1 according to an embodiment. The information processing system 1 includes an information processing device 10, a terminal device 20, and an imaging device 30. The information processing device 10, the terminal device 20, and the imaging device 30 are communicatively connected via a network 2.

[0010] The information processing device 10 is, for example, a computer such as a server device, etc. The information processing device 10 is connected to a terminal device 20 and an imaging device 30 via a network 2 so as to be able to communicate with each other.

[0011] The terminal device 20 is, for example, a smartphone, a tablet, or a personal computer (PC), but is not limited to these and may be any information processing terminal. The terminal device 20T is a terminal device used by the instructor 3, and the terminal devices 20A, 20B, and 20C are terminal devices used by the students 4A, 4B, and 4C, respectively. The terminal device 20 is connected to the information processing device 10 and the imaging device 30 via the network 2 so as to be able to communicate with each other.

[0012] The imaging device 30 is, for example, a video camera installed in a lecture hall or a classroom, and captures images (moving images or still images) of the participants 4. The imaging device 30A captures an image of the upper body of the seated participant, and the imaging device 30B captures an image of the participants' hands as they read and write on paper medium 5 or the like on a desk. The imaging devices 30A and 30B are the same imaging device. The imaging device 30 is connected to the information processing device 10 and the terminal device 20 via the network 2 so as to be able to communicate with each other.

[0013] When the information processing device 10 detects a predetermined body movement of a plurality of students from the captured images of the plurality of students, the information processing device 10 transmits information corresponding to the predetermined body movement to the terminal device 20 used by students other than the plurality of students. With this configuration, the information processing device 10 transmits the information when a common predetermined body movement is performed by a plurality of students, so that the complexity of the response of the students receiving the information can be reduced. Therefore, it is possible to make the response of the students receiving the information more efficient.

[0014] (Configuration of information processing device) The information processing device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0015] The communication unit 11 includes one or more communication interfaces connected to the network 2. The communication interface corresponds to, for example, a mobile communication standard, a wired LAN standard, or a wireless LAN standard, but is not limited to these, and may correspond to any communication standard. In this embodiment, the information processing device 10 communicates with the terminal device 20 and the imaging device 30 via the communication unit 11 and the network 2.

[0016] The storage unit 12 includes one or more memories. Each memory included in the storage unit 12 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores any information used in the operation of the information processing device 10. For example, the storage unit 12 may store a system program, an application program, a database, and a captured image of a participant performing a predetermined body movement received from the imaging device 30. The information stored in the storage unit 12 may be updateable with information acquired from the network 2 via the communication unit 11.

[0017] The control unit 13 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The processor is, for example, but is not limited to, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The programmable circuit is, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit is, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 13 controls the operation of the entire information processing device 10.

[0018] (Terminal Device Configuration) The terminal device 20 includes a communication unit 21, a storage unit 22, a display unit 23, and a control unit 24.

[0019] The communication unit 21 includes one or more communication interfaces connected to the network 2. The communication interfaces correspond to, for example, a mobile communication standard such as 4G (4th Generation) or 5G (5th Generation), a wired LAN (Local Area Network) standard, or a wireless LAN standard, but are not limited thereto. In this embodiment, the terminal device 20 communicates with the information processing device 10 and the imaging device 30 via the communication unit 21 and the network 2.

[0020] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores any information used in the operation of the terminal device 20. For example, the storage unit 22 may store a system program, an application program, a database, information corresponding to a predetermined body movement received from the information processing device 10, and the like. The information stored in the storage unit 22 may be updateable with information acquired from the network 2 via the communication unit 21.

[0021] The display unit 23 includes at least one display interface capable of displaying data. The display interface is, for example, an LCD or an organic EL display, but is not limited to these.

[0022] The control unit 24 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination of these. The control unit 24 controls the operation of the entire terminal device 20.

[0023] (Configuration of imaging device) The imaging device 30 includes a communication unit 31, a storage unit 32, an imaging unit 33, and a control unit 34. The imaging device 30 is a video camera that captures images (moving images or still images). The imaging device 30A captures an image of the upper body (bust-up) of a seated participant, and the imaging device 30B captures an image of the participant's hands reading and writing on a paper medium 5 or the like on a desk. The imaging devices 30A and 30B are the same imaging device.

[0024] The communication unit 31 includes one or more communication interfaces connected to the network 2. The communication interfaces correspond to, for example, a mobile communication standard such as 4G or 5G, a wired LAN standard, or a wireless LAN standard, but are not limited to these. In this embodiment, the imaging device 30 is communicably connected to the information processing device 10 and the terminal device 20 via the network 2. The communication unit 31 transmits a captured image captured by the imaging unit 33 and temporarily stored in the storage unit 32 to the information processing device 10.

[0025] The storage unit 32 includes one or more memories. The memories are, for example, but not limited to, semiconductor memories, magnetic memories, or optical memories. Each memory included in the storage unit 32 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 32 stores any information used in the operation of the imaging device 30. For example, the storage unit 32 stores system programs, application programs, built-in software, captured images of the students 4, and the like. The information stored in the storage unit 32 may be updatable with information acquired from the network 2 via the communication unit 31, for example.

[0026] The imaging unit 33 is an imaging element that captures an image. The imaging element is, for example, a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor, but is not limited to these.

[0027] The control unit 34 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 34 transmits the captured image stored in the storage unit 32 to the information processing device 10 via the communication unit 31 and the network 2.

[0028] (Operation flow of information processing device) 2 is a flowchart showing an example of the operation of the information processing device 10. This operation relates to the transmission of information corresponding to the physical movements of the students by the information processing device 10. This operation flow may be started when the control unit 13 of the information processing device 10 receives predetermined voice information or text information from the terminal device 20T of the instructor 3.

[0029] S101 : The communication unit 11 receives an image of the student 4 captured by the imaging device 30 .

[0030] As shown in FIG. 1, imaging device 30A that captures an image of the upper body of seated students 4 and imaging device 30B that captures an image of the hands of the students as they read or write on paper media 5 or the like on the desk are installed in front of and above the seats where each student 4 is seated.

[0031] S102: The control unit 13 detects a predetermined body movement from the received captured image of the student 4, and records the captured image of the student 4 from which the predetermined body movement is detected in the storage unit 12. An example of the body movement of the student 4 is shown in FIG. 3. In this disclosure, for the sake of convenience, the students 4 are described as three people, 4A, 4B, and 4C, but the number of students may be more than three. The body movement of the student 4 is, for example, nodding, face direction (turning right, turning left), resting chin on hand, degree of eye opening, dozing, facial expression (smiling, furrowing brow), gaze, or hand movement such as writing letters or figures in a notebook (paper medium 5), but is not limited to these.

[0032] The control unit 13 selects at least one of the body movements of the students 4 and sets the selected body movement as a predetermined body movement. The control unit 13 detects the set predetermined body movement from the received captured images of the students 4A, 4B, and 4C, and records the captured images of the students 4A, 4B, or 4C in which the predetermined body movement is detected in the storage unit 12. The control unit 13 further generates a timeline that chronologically records the detection of the predetermined body movement performed by the students 4A, 4B, or 4C, and records the generated timeline in the storage unit 12. In the example of the timeline shown in FIG. 3, nodding, face direction, opening eyes, and resting chin on hands are set as the predetermined body movements.

[0033] Further, Fig. 3 illustrates an image of the hands of the students 4A, 4B, and 4C captured by the imaging device 30B. The control unit 13 may detect the movement of the hands of the student 4, for example, using skeletal estimation. Skeletal estimation is a technique for estimating a human skeleton from a 2D image or video. Skeletal estimation estimates a human posture or movement from an image by using computer vision or artificial intelligence. In the example shown in Fig. 3, the control unit 13 detects the movement of the student 4 holding a pencil by skeletal estimation. The control unit 13 may also detect the movement of the student 4 writing on a notebook from the video.

[0034] S103: The control unit 13 detects predetermined body movements of the multiple students from the captured images of the multiple students.

[0035] The control unit 13 detects a common predetermined body motion taken by a plurality of students 4 within a predetermined time from the captured images of the students 4 recorded in the storage unit 12. In the example of the timeline shown in Fig. 3, the control unit 13 detects the resting of chin on hands taken by the plurality of students 4 (students 4A and 4B) within the predetermined time surrounded by the dashed line.

[0036] S104: The control unit 13 transmits information corresponding to the detected predetermined body movement to the terminal devices 20 used by students other than the student who performed the predetermined body movement.

[0037] The information corresponding to the predetermined body movement may include information on the state of the student 4 determined by the control unit 13 in addition to the captured image of the student 4 performing the predetermined body movement and the information on the predetermined body movement performed by the student 4 recorded on a timeline. The information corresponding to the predetermined body movement is not limited to these. The control unit 13, for example, determines the state of the student 4 as a positive state or a negative state. For example, the control unit 13 defines the state of the student 4 as a positive state when the student 4 has a smiling expression, and as a negative state when the student 4 has a small degree of eye opening (small degree of eye opening), is dozing, and is resting his / her chin on his / her hand. The control unit 13 may further classify the state of the student 4 into finer states. For example, the control unit 13 may define the state of the student 4 nodding as a state of high concentration. Furthermore, the control unit 13 may define the state of the student 4 writing predetermined characters or figures in a notebook as a state of high comprehension when the imaging device 30B reads the state. On the other hand, if the degree to which the eyes of the students 4 are open is less than a predetermined level, it may be defined as a state of low concentration. The control unit 13 may determine the degree to which the eyes are open by the number of times the students blink. If the number of times the students blink is high, it is determined that the students are drowsy (their concentration level is low). In this way, the control unit 13 can determine the state of the students 4 by detecting predefined body movements of multiple students 4 within a predetermined period of time.

[0038] When the control unit 13 detects that the students 4A and 4B are resting their chins on their hands, it transmits to the terminal device 20 of the student 4C the captured image of the students 4A and 4B resting their chins on their hands, the above-mentioned timeline information recording that the students 4A and 4B rested their chins on their hands, or information indicating that the students 4A and 4B are in a negative state. With this configuration, the student 4C who receives information corresponding to the predetermined body movements of the students 4A and 4B (image of students resting their chins on their hands, being in a negative state) can encourage the students 4A and 4B to improve their states.

[0039] S105: When the lecture ends, the control unit 13 ends the information processing.

[0040] As described above, when the information processing device 10 detects a predetermined body movement of a plurality of students from the captured images of the students, it transmits information corresponding to the predetermined body movement to the terminal device 20 used by students other than the plurality of students. With this configuration, the information processing device 10 transmits the information when a common predetermined body movement is performed by a plurality of students, so that the complexity of the response of the students receiving the information can be reduced. Therefore, it is possible to make the response of the students receiving the information more efficient.

[0041] For example, in the above-described embodiment, the configuration and operation of the information processing device 10 may be distributed among multiple computers that can communicate with each other. Also, for example, a general-purpose computer may be made to function as the information processing device 10 according to the above-described embodiment. Specifically, a program describing the processing contents for realizing each function of the information processing device 10 according to the above-described embodiment is stored in the memory of a general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure may also be realized as a program executable by a processor, or a non-transitory computer-readable medium that stores the program.

[0042] In the above-described embodiment, the control unit 13 transmits information corresponding to the detected predetermined body movement to the terminal device 20 used by a student other than the student who performed the predetermined body movement. However, the control unit 13 may transmit such information to the terminal device 20T of the instructor 3 in addition to the terminal device 20 of the student 4. With this configuration, the instructor 3 can adjust the difficulty level or progress speed of the lecture based on the state of the student 4 determined by the control unit 13.

[0043] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one. [Explanation of symbols]

[0044] 1. Information Processing Systems 2 Network 3. Instructor 4, 4A, 4B, 4C students 5 Media (paper media) 10. Information processing device 11 Communications Department 12 Storage section 13 Control section 20, 20A, 20B, 20C, 20T Terminal equipment 21 Communications Department 22 Memory section 23 Display section 24 Control Unit 30, 30A, 30B Imaging device (camera) 31 Communications Department 32 Storage section 33 Imaging unit 34 Control section

Claims

[Claim 1] The Communications Department and A control unit that performs communication by the communication unit, When the control unit detects specified body movements of multiple students from captured images of the multiple students, it transmits information corresponding to the specified body movements to terminal devices used by students other than the multiple students.

Images