Information sharing system, information sharing method, server device, program, and apparatus

Abstract

To suppress bias in users who are selected because they have made gestures.SOLUTION: In an information sharing system 100 in which an apparatus 10 and a terminal device 30 can communicate with a server device 50, the apparatus or the server device includes a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of the terminal device and recognizes gestures by analyzing the image data, a sharing count management unit that manages the number of times information displayed by the terminal device has been showed by the apparatus in association with the user of the terminal device, a face recognition unit that performs facial recognition on the image data and identifies users who made the gestures, an identification unit that identifies a user with the fewest number of shares among the users who made the gestures, and a communication unit that receives information from the terminal device associated with the user identified by the identification unit. The apparatus displays the information received by the communication unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information sharing system, an information sharing method, a server device, a program, and a device. [Background technology]

[0002] In schools and other educational institutions, systems are being used to teach classes using electronic teaching materials. For example, a device operated by a teacher or other person communicates with a terminal device operated by each student or other user, and the terminal device can display the teaching materials sent by the device and the answers sent by the terminal device.

[0003] One such system known is a technology that identifies a user through facial recognition (see, for example, Patent Document 1). Patent Document 1 discloses a technology that detects a hand-raising gesture and the user's face, identifies the user and the terminal device through facial recognition, and projects the screen of the terminal device using a projector. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, the user who first makes the gesture of raising their hand is selected, which causes a problem of bias in the users who are selected.

[0005] In view of the above-mentioned problems, the present invention provides a technology for suppressing bias in users being selected based on gestures. [Means for solving the problem]

[0006] In view of the above-mentioned problems, there is provided an information sharing system in which an apparatus and a terminal device can communicate with a server device, wherein the apparatus or the server device has a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of the terminal device and recognizes gestures by analyzing the image data, a sharing count management unit that manages the number of times that information displayed by the terminal device has been shared by the apparatus in association with the user of the terminal device, a face recognition unit that performs facial recognition on the image data and identifies the user who made the gesture, an identification unit that identifies the user who made the gesture with the fewest number of shares, and a communication unit that receives the information from the terminal device associated with the user identified by the identification unit, and the apparatus displays the information received by the communication unit. [Effects of the Invention]

[0007] The present invention can suppress bias in users being selected based on gestures. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating the configuration of an information sharing system having student terminals, a teacher terminal, and a lesson management server. [Figure 2] 1 is an example of a system configuration diagram of an information sharing system. [Figure 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of a lesson management server, a student terminal, and a teacher terminal. [Figure 4] FIG. 2 is an example of a functional block diagram illustrating functions of a lesson management server, a student terminal, and a teacher terminal, divided into blocks. [Figure 5] FIG. 10 is a diagram illustrating an example of sharing count information stored in a sharing count storage unit. [Figure 6] 10 is a diagram illustrating a process in which the lesson management server determines which student terminal's screen data is to be shared with the teacher terminal, and shares the screen data of the determined student terminal. FIG. [Figure 7]10 is an example of a sequence diagram illustrating a process in which the lesson management server determines which student terminal's screen data should be shared with the teacher terminal, and causes the teacher terminal to share the screen data of the determined student terminal. [Figure 8] FIG. 2 is an example of a functional block diagram illustrating functions of a lesson management server, a student terminal, and a teacher terminal, divided into blocks. [Figure 9] 10 is a diagram for explaining a process in which the lesson management server displays students in the order in which they raised their hands; FIG. [Figure 10] 10 is an example of a sequence diagram illustrating a process in which the lesson management server determines the order in which students raise their hands, and the teacher's terminal displays a list of students who have raised their hands in the order in which they raised their hands, along with the number of times they have shared the information. [Figure 11] FIG. 2 is an example of a functional block diagram illustrating functions of a lesson management server, a student terminal, and a teacher terminal, divided into blocks. [Figure 12] 10A and 10B are diagrams for explaining a process in which a teacher terminal displays a list of students who have raised their hands in descending order of the number of times they have shared information; [Figure 13] 10 is an example of a sequence diagram illustrating a process in which the lesson management server sorts students by the number of times they have shared their hands, and the teacher terminal displays a list of students who have raised their hands in descending order of the number of times they have shared their hands. [Figure 14] FIG. 2 is an example of a functional block diagram illustrating functions of a lesson management server, a student terminal, and a teacher terminal, divided into blocks. [Figure 15] FIG. 2 is a diagram schematically showing supervised data information stored in a supervised data storage unit. [Figure 16] 10 is a diagram illustrating a process in which the class management server compares the answers entered into the student terminals by students who have raised their hands with the correct answer data and determines the student terminals with which the screen data will be shared. FIG. [Figure 17] 10 is an example of a sequence diagram illustrating a process in which the lesson management server determines student terminals to be shared with the teacher terminal based on similarity, and shares screen data of the determined student terminals. [Figure 18] FIG. 10 is a diagram illustrating an example of a determination method setting screen displayed on the teacher terminal. [Figure 19]FIG. 2 is an example of a functional block diagram illustrating functions of a lesson management server, a student terminal, and a teacher terminal, divided into blocks. [Figure 20] FIG. 2 is an example of a functional block diagram illustrating functions of a student terminal and a teacher terminal, divided into blocks. DETAILED DESCRIPTION OF THE INVENTION

[0009] An information sharing system and an information sharing method performed by the information sharing system will be described below as an example of an embodiment of the present invention with reference to the drawings.

[0010] [First embodiment] First, an outline of the operation of the information sharing system 100 will be explained with reference to Fig. 1. Fig. 1 is a schematic diagram of the information sharing system 100 having a student terminal 30, a teacher terminal 10, and a lesson management server 50. In the following, any one of the multiple student terminals 30A to 30C will be referred to as a "student terminal 30."

[0011] First, a description will be given of the configuration of information sharing system 100. Details of the system configuration will be described later. The lesson management server 50 stores student information (authentication information, facial photographs, etc.). The teacher terminal 10 and student terminals 30A to 30C can communicate with the class management server 50 via a communication network. The class management server 50 manages the teacher terminals 10 and student terminals 30 participating in a class. For example, when a student logs in to the class management server 50 with his or her student ID and password, the class ID is identified based on the timetable of the class to which the student belongs. Similarly, when a teacher logs in to the class management server 50 with his or her teacher ID and password, the class ID of the class that the teacher is teaching is identified. The class management server 50 places students and teachers who share a class ID in the same virtual room. The lesson management server 50 establishes a session between the teacher terminal 10 and the student terminals 30 belonging to the virtual room. The information sharing system 100 transmits screen data displayed on the teacher terminal 10 to each of the student terminals 30A-30C, allowing each of the student terminals 30A-30C to share the screen of the teacher terminal 10. The teacher terminal 10 can also receive answers entered by students into each of the student terminals 30A-30C from each of the student terminals 30A-30C. The teacher terminal 10 can display the answers from each student terminal 30 and determine whether an answer is correct or incorrect.

[0012] Next, an outline of the procedure by which the teacher terminal 10 shares the screen data displayed on the student terminals 30A to 30C will be described. (1) The lesson management server 50 analyzes the image data captured by the imaging device 9 and detects a gesture of raising a hand made by a student. (2) The lesson management server 50 identifies the student (and the student terminal 30) by performing facial recognition on the face of the student who raised his / her hand. (3) In one embodiment, the class management server 50 records the number of times each student terminal 30 has shared information in the past, and identifies the student terminal 30 with the fewest number of sharing sessions among the students who have raised their hands. The class management server 50 requests the student terminal 30 that has raised its hand to capture the screen (on which the answer is displayed), and obtains the screen data from the student terminal 30. The class management server 50 transfers this screen data to the teacher terminal 10, which displays it.

[0013] This allows the teacher terminal 10 to display the answers of the students who raised their hands. In addition, the student who made the first gesture of raising his hand is selected, which can prevent bias from occurring among the student terminals 30 that share screen data.

[0014] <Terminology> Gestures are a form of body language that are expressed through bodily movements, such as gestures and hand movements. In this embodiment, when a student makes a gesture instructed by a teacher, the lesson management server recognizes it, thereby facilitating educational activities.

[0015] Sharing refers to the teacher terminal 10 displaying information that is displayed by the student terminal 30, and the number of times of sharing refers to the number of times the information is displayed, counted for each student terminal 30.

[0016] The information displayed by the student terminal 30 is information that the teacher terminal 10 should share with the student terminal 30. This information is information entered by the student. In this embodiment, an answer (screen data) by the student will be described as an example.

[0017] The equipment is a device operated by a teacher, and the terminal device is a device operated by a student. In this embodiment, the term "teachers" is used, but teachers may also be called teachers, faculty members, lecturers, professors, instructors, or instructors. Students may also be called students, learners, or participants. Furthermore, classes may also be called lectures, meetings, lectures, lessons, seminars, forums, panel discussions, roundtables, lecture meetings, or the like.

[0018] <System configuration example> Fig. 2 is an example of a system configuration diagram of an information sharing system 100. The information sharing system 100 has a class management server 50, a teacher terminal 10, and one or more student terminals 30, all connected via a communication network N. Although three student terminals 30A to 30C are shown in Fig. 2, the number of student terminals 30 may be the same as the number of students in one class, or one student may have multiple student terminals 30.

[0019] The communication network N is constructed including at least one of a LAN installed in each classroom in the school, a provider network of a provider that connects the LAN to the Internet, and a line provided by a line operator. For example, the teacher terminal 10 and the student terminal 30 are connected to the LAN, and the lesson management server 50 is connected to the Internet or the cloud. The teacher terminal 10 and the student terminal 30 may be directly connected to a telephone line network or a mobile phone network without going through a LAN. The communication network N may be constructed as either a wired or wireless network, or may be a combination of wired and wireless networks.

[0020] The teacher terminal 10 has an imaging device 9. In FIG. 2, the imaging device 9 is integrated (built-in) with the teacher terminal 10, but the imaging device 9 may be separate from the teacher terminal 10 and connected to the teacher terminal 10 via a wired or wireless connection. The imaging device 9 is installed above the display of the teacher terminal 10, on a podium, the ceiling, wall, or desk of the classroom. The imaging device 9 is a camera that captures images of students participating in class. The imaging device 9 may also be called a digital still camera or digital video camera. The imaging device 9 may be an omnidirectional camera that captures 360 degrees of the surrounding area in a single capture and creates an omnidirectional image. Regardless of the imaging device 9, multiple imaging devices 9 may be arranged to reduce blind spots. The imaging device 9 may also be a camera that can acquire distance information, such as a stereo camera or LiDAR.

[0021] The imaging device 9 may also be a camera built into each student terminal 30. In this case, each student terminal 30 can capture images of each student without any obstructions, reducing blind spots. The imaging device 9 may also be connected to the communication network N instead of the teacher terminal 10, or to the lesson management server 50.

[0022] The imaging device 9 periodically captures images of students. This does not necessarily have to be periodic; images may be captured irregularly, or when the students are moving. Alternatively, the teacher may operate the teacher terminal 10 to request the lesson management server 50 to capture images, and the imaging device 9 may capture images in response to a command from the lesson management server 50.

[0023] The imaging device 9 may also have a microphone. The audio data collected by the microphone is sent to the lesson management server 50. However, the microphone may be provided separately from the imaging device 9. For example, it may be placed on the ceiling or wall separately from the imaging device 9, or a microphone provided on the student terminal 30 used by each student may be used. The students' voices are used to recognize the students' levels of understanding and gestures, etc.

[0024] The teacher terminal 10 may be placed in a location where the teacher can operate it, but it does not necessarily have to be within reach. For example, only the keyboard and mouse of the teacher terminal 10 may be located near the teacher, and the main body of the teacher terminal 10 may be located in another location. Also, a large display separate from the teacher terminal 10 may be placed in the classroom, and the screen displayed by the teacher terminal 10 may be displayed on the large display. Note that the teacher terminal 10 does not need to be connected to the communication network N all the time; it only needs to be connected during class.

[0025] The teacher terminal 10 is, for example, an electronic whiteboard (an example of a device) and communicates data with the class management server 50 via a communication network N. In addition to an electronic whiteboard, the teacher terminal 10 may also be a PC, tablet terminal, PDA (Personal Digital Assistant), video conference terminal, wearable PC, game console, mobile phone, car navigation system, smartphone, etc. The teacher terminal 10 is not limited to these, and can communicate with the class management server 50 as long as it runs a web browser or a dedicated application on the class management server 50.

[0026] The teacher terminal 10 can send its own screen data to the lesson management server 50 and can also display screen data (screen data of the student terminal 30) sent from the lesson management server 50. The teacher terminal 10 can control the screen displayed on each student terminal 30. For example, while the teacher is giving an explanation, the teacher can instruct the students on the page they want to see, and the student terminal 30 will be fixed to display the page sent from the teacher terminal 10. If the teacher allows any page to be displayed, the students can display any screen on the student terminal 30.

[0027] When the teacher terminal 10 displays the screen data displayed on each student terminal 30 transmitted from the lesson management server 50, if the teacher terminal 10 displays all the screen data at once, visibility will decrease if there are many students. The teacher terminal 10 can display only the screen data of a specific student terminal 30 larger than the others. The specific student terminal 30 may be the student terminal 30 of a student selected by the teacher, the student terminal 30 of a student who is speaking, etc.

[0028] The student terminal 30 is a terminal device used by a student. It is preferable that the student terminal 30 is placed in a location where it can be operated by the student. For example, the student terminal 30 may be provided exclusively for the student, or may be fixed to a desk and become the student terminal 30 dedicated to that student only while the student is logged in. The student terminal 30 does not need to be connected to the communication network N all the time, as long as it is connected during class.

[0029] The student terminal 30 is, for example, a PC, and performs data communication with the class management server 50 via the communication network N. The student terminal 30 may be a tablet terminal, PDA, smartphone, etc. in addition to a PC. The student terminal 30 is not limited to these, and can communicate with the class management server 50 as long as it runs a web browser or a dedicated application on the class management server 50.

[0030] The student terminal 30 can send its own screen data to the class management server 50 and can also display the screen data sent from the class management server 50. The screen data sent from the class management server 50 is the screen data displayed on the teacher terminal 10. The student terminal 30 can display a different screen independently from the teacher terminal 10. For example, if each student's progress differs, the student terminal 30 can display a page for any teaching material or subject through the operation of each student. The screen data of the student terminal 30 sent to the class management server 50 includes, for example, the student's answers, questions, etc.

[0031] The lesson management server 50 can recognize students' gestures, faces, and utterances, and notifies the teacher terminal 10 of the status of each student accordingly. The lesson management server 50 can display screen data of the student terminal 30 of a student who has made a predetermined gesture on the teacher terminal 10. The lesson management server 50 is, for example, one or more information processing devices (server devices), and can communicate data with the imaging device, teacher terminal 10, and student terminal 30 via the communication network N.

[0032] The class management server 50 may be installed on-premise or in the cloud. Although there is no strict definition of the physical configuration of cloud computing, a known configuration is one in which resources such as CPU, RAM, and storage that make up an information processing device are dynamically connected and disconnected depending on the load, allowing the configuration and installation location of the information processing device to be flexibly changed.

[0033] <Hardware configuration example> The hardware configuration of the class management server 50, student terminal 30, and teacher terminal 10 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the hardware configuration of the class management server 50, student terminal 30, and teacher terminal 10 according to this embodiment. As shown in Fig. 3, the class management server 50, student terminal 30, and teacher terminal 10 are configured by a computer 500. The computer 500 includes a CPU 501, a ROM 502, a RAM 503, a hard disk (HD) 504, a hard disk drive (HDD) controller 505, a display 506, an external device connection interface (I / F) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, and a media I / F 516.

[0034] Of these, the CPU 501 controls the overall operation of the computer 500. The ROM 502 stores programs used to drive the CPU 501, such as an IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The display 506 displays various information, such as a cursor, menus, windows, characters, or image data. The external device connection I / F 508 is an interface for connecting various external devices. In this case, external devices include, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using a network. The bus line 510 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 501, shown in FIG. 3.

[0035] The keyboard 511 is a type of input means having multiple keys used to input characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from an optical storage medium 513, which is an example of a removable storage medium. The optical storage medium 513 may be a CD, a DVD, Blu-ray (registered trademark), etc. The media I / F 516 controls reading and writing (storing) of data from a storage medium 515, such as a flash memory.

[0036] <About the function> FIG. 4 is a functional block diagram explaining the functions of the lesson management server 50, the student terminal 30, and the teacher terminal 10, divided into blocks.

[0037] <<Teacher terminal 10>> The teacher terminal 10 has a communication unit 11, a display control unit 12, an operation reception unit 13, an imaging unit 14, a gesture recognition unit 15, and a screen data generation unit 16. Each of these functional units of the teacher terminal 10 is a function or a means for performing a function that is realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 to the RAM 503.

[0038] The communication unit 11 connects to the communication network N and communicates with the class management server 50. In this embodiment, the communication unit 11 transmits image data captured by the imaging unit 14 or cropped facial images of each student to the class management server 50. The communication unit 11 also receives screen data displayed on the display 506 of the student terminal 30 from the class management server 50. The communication unit 11 may also transmit screen data displayed on the display 506 of the teacher terminal 10 to the class management server 50.

[0039] The display control unit 12 displays the screen data of the student terminal 30 received by the communication unit 11 on the display 506, and displays teaching materials to be used in the lesson on the display 506. The teaching materials to be used in the lesson may be stored in the HD 504 or may be transmitted from a web server. The operation receiving unit 13 receives operations performed by the teacher on the teacher terminal 10.

[0040] The imaging unit 14 captures images of one or more students to generate image data showing the students. The imaging unit 14 may generate video images captured repeatedly at a rate of 30 to 60 fps, or may generate still images. The imaging unit 14 may capture images only when there is a certain amount of movement in the image.

[0041] The gesture recognition unit 15 recognizes gesture operations of students from image data captured by the imaging unit 14. One method for recognizing gestures from screen data is to use a model generated by machine learning, for example. This model outputs the type of gesture represented by the input image data along with the probability of the gesture. The gesture recognized in this embodiment is, for example, a gesture of raising a hand.

[0042] For example, it is assumed that image data and gesture types are associated with each other using a model created by machine learning such as deep learning. A known model for determining movements from image data is to estimate a person's joint coordinates from the image data and associate them with the movements. That is, the gesture recognition unit 15 obtains the joint coordinates using an existing model that detects joint coordinates (movements) from image data. The gesture recognition unit 15 then uses machine learning to prepare a model that inputs the joint coordinates (movements) and outputs the gesture type, thereby enabling it to accurately analyze the gesture of raising a student's hand.

[0043] When the gesture recognition unit 15 recognizes the gesture of raising a hand, it further detects a human face. The human face does not necessarily need to be identified as the face of a specific student, but refers to the presence of any human face. Individuals are identified by the class management server 50. However, the teacher terminal 10 may also recognize individual students. Known methods for detecting faces include using Haar-like features, CNN (Convolutional Neural Network), and YOLO (You Only Look Once). The gesture recognition unit 15 generates a face image by cropping the area that includes the face.

[0044] The screen data generator 16 captures in real time the screen displayed on the display 506 of the teacher terminal 10, and generates screen data of the current screen displayed on the teacher terminal 10. This screen data is sent to the student terminals 30 via the lesson management server 50, mainly while the teacher is giving a lecture.

[0045] 4, the imaging unit 14 and the gesture recognition unit 15 may be included in the lesson management server 50. In other words, the teacher terminal 10 may include some or all of the components of the lesson management server 50.

[0046] <<Class Management Server 50>> The class management server 50 has an identification unit 51, a second communication unit 52, a sharing count management unit 53, a face recognition unit 54, and a first communication unit 55. Each of these functional units of the class management server 50 is a function or a means for performing the function, which is realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 to the RAM 503. The class management server 50 also has a sharing count storage unit 56 built in the HD 504 or RAM 503 shown in Fig. 3. The sharing count storage unit 56 may be located in a location that the class management server 50 can access via the communication network N.

[0047] The first communication unit 55 connects to the communication network N and receives face images from the teacher terminal 10. The first communication unit 55 also transmits screen data received from the student terminal 30 to the teacher terminal 10.

[0048] The face recognition unit 54 is a means for recognizing faces by analyzing the face image transmitted from the teacher terminal 10. The face recognition unit 54 may identify a student who has made a gesture of raising their hand. For example, the face recognition unit 54 registers facial features of each student in advance, and identifies the student and their name by pattern matching or the like. By identifying the student, a student ID can be obtained.

[0049] Another method for such facial recognition is to use a model constructed by machine learning. Image data has sufficient resolution to analyze student faces. The face recognition unit 54 recognizes (identifies) students by using a model in which image data features and student IDs are previously associated using machine learning such as deep learning. The image data features used in facial recognition are, for example, the relative positions of detectable facial features such as eyebrows, pupils, nostrils, nose tip, corners of the mouth, and mouth. The face recognition unit 54 extracts features from the face of a student who makes a hand-raising gesture and inputs them into the model to output the student ID.

[0050] The sharing count management unit 53 manages the number of times with which the teacher terminal 10 has shared screen data with students in the sharing count storage unit 56. The sharing count storage unit 56 will be described with reference to FIG.

[0051] The identification unit 51 refers to the sharing count information and determines the student who has shared screen data the fewest times among the students who have made a hand-raising gesture. The screen data displayed on the student terminal 30 of this student is shared by the teacher terminal 10.

[0052] The second communication unit 52 requests screen data from the student terminal 30 determined by the identification unit 51, and receives the screen data from the student terminal 30. Then, the first communication unit 55 transmits this screen data to the teacher terminal 10.

[0053] 4, the teacher terminal 10 may include the identification unit 51, the sharing count management unit 53, and the face recognition unit 54. In other words, the lesson management server 50 may include some or all of the components of the teacher terminal 10.

[0054] <<Student terminal 30>> The student device has a communication unit 31, a display control unit 32, an operation reception unit 33, and a screen data generation unit 34. Each of these functional units of the teacher terminal 10 is a function or a means for performing a function that is realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 501 in accordance with a program loaded from the HD 504 to the RAM 503.

[0055] The communication unit 31 connects to the communication network N and communicates with the class management server 50. In this embodiment, the communication unit 31 transmits screen data displayed on the display 506 to the class management server 50 in response to a request from the class management server 50 or at any time. The functions of the display control unit 32, operation reception unit 33, and screen data generation unit 34 may be the same as those of the teacher terminal 10.

[0056] <<Share count storage section>> 5 shows the sharing count information stored in the sharing count storage unit 56. The sharing count information stores the number of times that screen data displayed by the student terminal 30 has been shared with the teacher terminal 10 in association with the student ID. The student ID is identification information of the student. When one student uses one student terminal 30, the student ID may be considered as identification information of the student terminal 30. The number of times of sharing is the number of times that the screen data displayed by the student terminal 30 has been shared with the teacher terminal 10. In this case, sharing means that the data has been displayed on the teacher terminal 10. The number of times of sharing may be the number of times over the entire past period for which a history can be kept, or the number of times over the most recent fixed period (one month, one week, six months, etc.). The IP address is the IP address of each student terminal 30 that is registered when the student logs in to the class management server 50. The class management server 50 can communicate with any student terminal 30 by using this IP address as the destination.

[0057] <Example of screen sharing> A method for determining the sharing source student terminal 30 using the sharing count information will be described with reference to Fig. 6. Fig. 6 is a diagram that schematically explains the process in which the class management server 50 determines which student terminal 30's screen data should be shared with the teacher terminal 10, and shares the screen data of the determined student terminal 30.

[0058] (1) For example, a teacher gives an assignment, and each student inputs an answer to the student terminals 30A to 30C. Each of the student terminals 30A to 30C displays the answer input by each student.

[0059] (2) When the teacher says, "Raise your hand if you understand," the students who input their answers raise their hands. The image capture device 9 repeatedly captures images of the students, and the image data includes students who raise their hands. In Figure 6, Student A and Student B raised their hands.

[0060] (3) The teacher terminal 10 recognizes the gesture and sends a facial image of the student who raised his / her hand to the lesson management server 50.

[0061] (4) The class management server 50 identifies the student IDs by recognizing the faces of the students in the facial images, and determines that students A and B have made the gesture of raising their hands.

[0062] (5) When the identification unit 51 refers to the sharing count information, it finds that the number of sharing times for student A who raised his hand is 2, and the number of sharing times for student B who raised his hand is 1. In order to reduce the imbalance in the number of sharing times, the identification unit 51 determines that the image data displayed by student terminal 30B, which has the fewest number of sharing times, will be shared.

[0063] (6) The lesson management server 50 requests screen data from the student terminal 30B, and in response to the request, the student terminal 30B transmits the screen data to the lesson management server 50. The screen data includes the answer.

[0064] (7) The lesson management server 50 transmits the screen data received from the student terminal 30B to the teacher terminal 10. The teacher terminal 10 displays the screen data received from the student terminal 30B on the display 506. In Figure 6, it can be seen that the teacher terminal 10 is displaying the screen data of the student terminal 30B.

[0065] <Action or Processing> 7 is a sequence diagram illustrating the process in which the class management server 50 determines which student terminal 30's screen data should be shared with the teacher terminal 10, and then shares the screen data of the determined student terminal 30 with the teacher terminal 10. Note that, in FIG. 7, to simplify the drawing, the lines connecting the communication units 11 and 31, the first communication unit 55, and the second communication unit 52 are not drawn.

[0066] S1: The teacher gives an assignment to the student, and the student answers on the student terminal 30. The answer is reflected in the screen data (i.e., it is displayed). The teacher inputs a start of answering into the teacher terminal 10. Alternatively, the teacher may set a time limit in advance when giving the assignment to the student and input it into the teacher terminal 10.

[0067] S2: The operation receiving unit 13 of the teacher terminal 10 receives an operation to start answering, and the imaging unit 14 controls the imaging device 9 to repeatedly capture images of a plurality of students. The imaging itself may be performed continuously.

[0068] S3: The imaging unit 14 transmits the image data to the gesture recognition unit 15.

[0069] S4: The gesture recognition unit 15 analyzes the image data and identifies the gesture. In this embodiment, at least a gesture of raising a hand is detected. For example, when the teacher says, "Please raise your hand if you answered correctly," each student who inputted an answer raises their hand.

[0070] S5: The gesture recognition unit 15 extracts facial images of all students who have made a hand-raising gesture. The communication unit 11 of the teacher terminal 10 transmits all facial images to the lesson management server 50. It is not necessary to transmit all facial images at the same time; a facial image may be transmitted each time a hand-raising gesture is recognized.

[0071] S6: The first communication unit 55 of the class management server 50 receives a facial image of the student who made the hand-raising gesture. Upon receiving the facial image, the facial recognition unit 54 analyzes the facial image and identifies the student. If the students do not necessarily raise their hands simultaneously, each of the multiple students who raised their hands will be identified until the teacher inputs the end of answering on the student terminal 30 into the teacher terminal 10, or until the time limit set by the teacher. The facial recognition unit 54 identifies all students who raised their hands until the end of answering or the time limit.

[0072] S7: The face recognition unit 54 transmits the identified student ID to the identification unit 51.

[0073] S8: The teacher inputs the end of answering into the teacher terminal 10. If a time limit is set, the operation to end answering is not necessary.

[0074] S9: The operation reception unit 13 receives the operation, and the communication unit 11 transmits a message to the class management server 50 indicating that the student terminal selection has been completed.

[0075] S10, S11: The first communication unit 55 receives the notification that the selection of the student terminal 30 has been completed, and the identification unit 51 acquires the number of shares associated with the student ID from the number of shares management unit 53.

[0076] S12: The identification unit 51 determines the student terminal 30 of the student who has shared the screen data the least number of times from among the students who raised their hands as the student terminal 30 with which the screen data will be shared.

[0077] S13: The identification unit 51 requests the second communication unit 52 to share the screen data by specifying the IP address or the like of the determined student terminal 30. The second communication unit 52 requests the screen data from the student terminal 30 determined by the identification unit 51. The sharing count management unit 53 acquires the student ID determined by the identification unit 51 from the identification unit 51, and increments the sharing count of the student in the sharing count storage unit 56 by one.

[0078] S14: When the communication unit 31 of the student terminal 30 receives a request for screen data, the screen data generation unit 16 captures the information displayed on the display 506 and generates screen data. The communication unit 31 of the student terminal 30 sends the screen data to the teacher terminal 10 via the class management server 50. The screen data is captured in order to obtain the handwritten content if the answer is handwritten. Depending on the format of the answer (options, digital data), capturing may not be necessary. In this embodiment, the options and answers such as digital data are also acquired by capturing. An example of an answer (correct answer data) is shown in Figure 15.

[0079] S15: When the communication unit 11 of the teacher terminal 10 receives the screen data, it transmits it to the display control unit 12. The display control unit 12 of the teacher terminal 10 causes the display 506 to display the screen data transmitted from the student terminal 30.

[0080] In this embodiment, the condition for sharing screen data is a student's raising of their hand, but the teacher may designate any student terminal 30, and the screen data of that student terminal 30 may be shared with the teacher terminal 10. Alternatively, the teacher terminal 10 may acquire screen data from all student terminals 30, and the teacher may select the student terminal 30 to share. For example, the teacher may use this to introduce typical incorrect answers, or to grasp the comprehension level of each student.

[0081] <Major Effects> The lesson management server 50 of this embodiment refers to the sharing count information and determines the student terminal 30 that has been shared the least, so that it is possible to prevent the teacher terminal 10 from sharing the student terminals 30 in an unbalanced manner.

[0082] [Second embodiment] In this embodiment, an information sharing system 100 will be described in which each student is displayed on the teacher terminal 10 in the order in which they raised their hand.

[0083] <About the function> Figure 8 is a functional block diagram explaining the functions of the class management server 50, student terminal 30, and teacher terminal 10 in this embodiment. The explanation of Figure 8 will mainly focus on the differences from Figure 4. The class management server 50 in this embodiment newly includes an order determination unit 57. The order determination unit 57 determines the order in which students who raise their hands are placed, in order of the earliest hand-raising.

[0084] <Example of screen sharing> Next, an example of how students are displayed in the order in which they raised their hands will be described with reference to Fig. 9. Fig. 9 is a diagram that schematically illustrates the process by which the class management server 50 displays students in the order in which they raised their hands. In this embodiment, the teacher selects which student terminal 30's screen data to share with the teacher terminal 10, but this may be the same as in the first embodiment (fewer sharing times).

[0085] (1) For example, a teacher gives an assignment, and each student inputs an answer to the student terminals 30A to 30C. Each of the student terminals 30A to 30C displays the answer input by each student.

[0086] (2) When the teacher says, "Raise your hand if you understand," the students who input their answers raise their hands. The image capture device 9 repeatedly captures images of the students, and the image data includes students who make the gesture of raising their hands. In Figure 9, Student A and Student B raised their hands.

[0087] (3) The teacher terminal 10 recognizes the gesture of raising the hand and transmits a facial image of the student who raised the hand to the lesson management server 50.

[0088] (4) The class management server 50 identifies the student ID by recognizing the faces of the students in the facial images, and first determines that Student B has made a gesture of raising their hand. The order determination unit 57 determines that Student B's hand-raising order number is 1. Next, the class management server 50 recognizes the faces of the students in the facial images, and determines that Student A has made a gesture of raising their hand. The order determination unit 57 determines that Student A's hand-raising order number is 2.

[0089] (5) The lesson management server 50 associates students A and B with the order numbers of their hand-raising and sends them to the teacher terminal 10 each time. As a result, the teacher terminal 10 displays a list of students who have raised their hands along with their order numbers. The identifying unit 51 may also refer to the number of sharing times information and display the number of sharing times of students A and B who have raised their hands in the list of students who have raised their hands. This allows the teacher to select students who have shared the least number of times.

[0090] (6) The teacher selects a student based on the number of times the information has been shared. For example, the student terminal 30B is selected.

[0091] (7) The lesson management server 50 requests the screen data from the student terminal 30B, and the student terminal 30B transmits the screen data to the lesson management server 50 in response to the request.

[0092] (8) The lesson management server 50 transmits the screen data received from the student terminal 30B to the teacher terminal 10. The teacher terminal 10 displays the screen data on the display 506. In Figure 9, it can be seen that the teacher terminal 10 is displaying the screen data of the student terminal 30B.

[0093] <Action or Processing> 10 is a sequence diagram illustrating the process in which the class management server 50 determines the order in which students raise their hands, and the teacher terminal 10 displays a list of students who have raised their hands in the order in which they raised their hands, along with the number of times they have shared their hands. The explanation of FIG. 10 will mainly focus on the differences from FIG. 7. The processes of steps S21 to S26 may be the same as steps S1 to S6 in FIG. 7. The class management server 50 then repeatedly executes steps S26 to S34.

[0094] S26: The first communication unit 55 of the class management server 50 receives a facial image of the student who made the gesture of raising their hand. Upon receiving the facial image, the facial recognition unit 54 analyzes the facial image and identifies the student. The facial recognition unit 54 repeatedly identifies all students who have raised their hands until the teacher selects a student from the student list, or until the time limit is reached.

[0095] S27: The face recognition unit 54 transmits the determined student ID to the order determination unit 57.

[0096] S28: The order determination unit 57 determines the order numbers in the order in which the student IDs are received, and associates them with the student IDs.

[0097] S29: Furthermore, the face recognition unit 54 transmits the identified student ID to the identification unit 51.

[0098] S30, S31: The identification unit 51 acquires the number of sharing times associated with the student ID from the sharing number management unit 53.

[0099] S32: The identification unit 51 transmits the number of sharing times of the students who raised their hands to the order determination unit 57.

[0100] S33: The order determination unit 57 generates a student list in which the raising order number is associated with the number of times of sharing, and the first communication unit 55 transmits the student list to the teacher terminal 10. In this way, every time the order determination unit 57 associates the raising order number with the student ID, the first communication unit 55 transmits the student list to the teacher terminal 10.

[0101] S34: When the communication unit 11 of the teacher terminal 10 receives the student list, it transmits the student list to the display control unit 12. The display control unit 12 causes the display 506 to display the student list.

[0102] S35: The teacher inputs an operation to select a student from the student list into the teacher terminal 10. Instead of selecting a student, the teacher may input "answer end" as in the first embodiment, and the student with the fewest number of sharing times may be selected. When the student with the fewest number of sharing times is selected and a time limit is set, the operation to "answer end" is not necessary.

[0103] S36: The operation receiving unit 13 receives the operation, and the communication unit 11 transmits the selected student ID to the class management server 50.

[0104] S37: The first communication unit 55 of the class management server 50 receives the student ID, and the second communication unit 52 requests the sharing of screen data by specifying the IP address or the like of the student terminal 30 identified by the student ID. The sharing count management unit 53 increments the number of sharing times for the student in the sharing count storage unit 56 by one based on the received student ID. The subsequent steps S38 and S39 may be the same as steps S14 and S15 in FIG. 7.

[0105] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, the teacher terminal 10 can display the students who raised their hands in the order in which they raised their hands, in association with the number of times they shared. The teacher can select students with the fewest number of times they shared, referring to the display order of the students or the number of times they shared.

[0106] [Third embodiment] In this embodiment, an information sharing system 100 will be described that displays a list of students in descending order of the number of times they have been shared.

[0107] <About the function> Figure 11 is a functional block diagram explaining the functions of the class management server 50, student terminal 30, and teacher terminal 10 in this embodiment. The explanation of Figure 11 will mainly focus on the differences from Figure 4. The class management server 50 in this embodiment newly includes a sorting unit 58. The sorting unit 58 sorts students who have made a hand-raising gesture in descending order of the number of times they have shared their data.

[0108] <Example of screen sharing> An example of displaying the number of sharing times will be described with reference to Fig. 12. Fig. 12 is a diagram that schematically illustrates the process in which the teacher terminal 10 displays a list of students who raised their hands in order of the least number of sharing times. Note that the teacher selects which student terminal 30's screen data to share with the teacher terminal 10, but it may also be the same as in the first embodiment (the student with the least number of sharing times).

[0109] (1) For example, a teacher gives an assignment, and each student inputs an answer to the student terminals 30A to 30C. Each of the student terminals 30A to 30C displays the answer input by each student.

[0110] (2) When the teacher says, "Raise your hand if you understand," the students who input their answers raise their hands. The image capture device 9 repeatedly captures images of the students, and the image data includes students who make the gesture of raising their hands. In Figure 12, Student A and Student B raised their hands.

[0111] (3) The teacher terminal 10 recognizes the gesture of raising the hand and transmits a facial image of the student who raised the hand to the lesson management server 50.

[0112] (4) The lesson management server 50 recognizes the faces of the students in the facial images and determines that students A and B have made the gesture of raising their hands.

[0113] (5) When the identification unit 51 refers to the sharing count information, it finds that the number of sharing times for Student A who raised his hand is 2, and the number of sharing times for Student B who raised his hand is 1. The sorting unit 58 sorts the students in descending order of the number of sharing times for each student, and generates a list of students in sorted order.

[0114] (6) Each time a student raises their hand, the class management server 50 sends a list of students to the teacher terminal 10, and the teacher terminal 10 displays the list of students. As a result, the teacher terminal 10 displays the list of students in order of least number of sharing times.

[0115] (7) The teacher selects a student based on the display order based on the number of times the information has been shared. For example, the student terminal 30B is selected.

[0116] (8) The lesson management server 50 requests the screen data from the student terminal 30B, and the student terminal 30B transmits the screen data to the lesson management server 50 in response to the request.

[0117] (9) The lesson management server 50 transmits the screen data received from the student terminal 30B to the teacher terminal 10. The teacher terminal 10 displays the screen data on the display 506. In FIG. 12, it can be seen that the teacher terminal 10 is displaying the screen data of the student terminal 30B.

[0118] <Action or Processing> 13 is a sequence diagram illustrating the process in which the class management server 50 sorts students who have raised their hands by the number of times they have shared, and the teacher terminal 10 displays a list of students who have raised their hands in order of the number of times they have shared. The explanation of FIG. 13 will mainly focus on the differences from FIG. 7. The processes of steps S51 to S56 may be the same as steps S1 to S6 in FIG. 7. The class management server 50 then repeatedly executes steps S56 to S63.

[0119] S56: The first communication unit 55 of the class management server 50 receives a facial image of the student who made the gesture of raising their hand. Upon receiving the facial image, the facial recognition unit 54 analyzes the facial image and identifies the student. The facial recognition unit 54 either allows the teacher to select a student from a student list, or repeatedly identifies all students who have raised their hands until the time limit is reached.

[0120] S57: The face recognition unit 54 transmits the determined student ID to the identification unit 51.

[0121] S58, S59: The identification unit 51 acquires the number of times of sharing associated with the student ID from the number of times of sharing management unit 53.

[0122] S60: The identification unit 51 transmits the number of sharing times of the students who raised their hands to the sorting unit 58.

[0123] S61: The sorting unit 58 sorts the students who have raised their hands by the number of times they have shared the class. Each time the number of students who have raised their hands increases, the sorting unit 58 repeats the sorting process.

[0124] S62: The sorting unit 58 transmits a student list showing the students in the sorted order to the first communication unit 55. The student list may include the number of times the student has been shared. The first communication unit 55 transmits the student list to the teacher terminal 10.

[0125] S63: When the communication unit 11 of the teacher terminal 10 receives the student list, it transmits the student list to the display control unit 12. The display control unit 12 causes the display 506 to display the student list.

[0126] S64: The teacher checks the students displayed in descending order of the number of times of sharing, and inputs an operation to select a student into the teacher terminal 10. Instead of selecting a student, the teacher may input "end answer" as in the first embodiment, and the student with the fewest number of times of sharing may be selected. When the student with the fewest number of times of sharing is selected and a time limit is set, the operation to "end answer" is not necessary.

[0127] S65: The operation reception unit 13 receives the operation, and the communication unit 11 transmits the selected student ID to the class management server 50.

[0128] S66: The first communication unit 55 of the lesson management server 50 receives the student ID, and the second communication unit 52 requests the sharing of screen data by specifying the IP address of the student terminal 30 identified by the student ID. The sharing count management unit 53 increments the number of sharing times for the student in the sharing count storage unit 56 by one based on the received student ID.

[0129] The subsequent steps S67 and S68 may be the same as steps S14 and S15 in FIG.

[0130] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, the teacher terminal 10 can display students who have raised their hands in order of the least number of times they have shared. From the list of students displayed in order of the least number of times they have shared, the teacher can select a student with the least number of times they have shared, referring to the display order.

[0131] [Fourth embodiment] In this embodiment, an information sharing system 100 will be described in which the screen data of a student who has raised their hand is compared with the correct answer data, and the teacher terminal 10 displays the screen data of the student that has the highest similarity to the correct answer data.

[0132] <About the function> Figure 14 is a functional block diagram explaining the functions of the lesson management server 50, student terminal 30, and teacher terminal 10 in this embodiment. The explanation of Figure 14 will mainly focus on the differences from Figure 4. The lesson management server 50 in this embodiment newly includes a comparison unit 59 and a correct answer data storage unit 60.

[0133] The comparison unit 59 compares the answer (screen data) sent by the student terminal 30 of the student who made the hand-raising gesture with the correct answer data stored in the correct answer data storage unit 60, and calculates the similarity. Then, the identification unit 51 determines the student terminal 30 that sent the answer with the highest similarity as the student terminal 30 on which the teacher terminal 10 will display screen data.

[0134] FIG. 15 is a diagram showing a schematic diagram of correct answer data information stored in the correct answer data storage unit 60. The correct answer data information stores question IDs and correct answer data in association with each other. The question ID is identification information for the question. The correct answer data is the model answer for the question. The correct answer data may be in the form of multiple choice options, digital data, or raster data (image data). The multiple choice options are the numbers of the choices when a question requires multiple choices. The digital data is for written tests in which answers are input digitally. "Neptune" is correct answer data for a question asking what the outermost planet is. The raster data is for written tests in which answers are input by hand. "Question" is correct answer data for a question asking how to write "gimon" (question) in kanji. The raster data answer may also be any other handwritten form by the user, such as a diagram, a mathematical formula, or written English words. Therefore, the comparison unit 59 compares the answer with the correct answer data according to the format of the correct answer data.

[0135] <Example of screen sharing> A method for determining a shared terminal based on the similarity between the correct answer data and the answer will be described with reference to Fig. 16. Fig. 16 is a diagram that schematically explains the process in which the class management server 50 compares the answer entered into the student terminal 30 by the student who raised their hand with the correct answer data and determines the student terminal 30 with which to share the screen data.

[0136] (1) For example, a teacher gives an assignment, and each student inputs an answer to the student terminals 30A to 30C. Each of the student terminals 30A to 30C displays the answer input by each student.

[0137] (2) When the teacher says, "Raise your hand if you understand," the students who input their answers raise their hands. The image capture device 9 repeatedly captures images of the students, and the image data includes students who raise their hands. In Figure 16, Student A and Student B raised their hands.

[0138] (3) The teacher terminal 10 recognizes the gesture of raising the hand and transmits a facial image of the student who raised the hand to the lesson management server 50.

[0139] (4) The lesson management server 50 recognizes the faces of the students in the facial images and determines that students A and B have made the gesture of raising their hands.

[0140] (5) The lesson management server 50 requests screen data from the student terminals 30A and 30B associated with the identified students A and B. The student terminals 30A and 30B respond by sending the screen data to the lesson management server 50. (6) The lesson management server 50 compares the screen data received from the student terminals 30A and 30B with the correct answer data 61 and calculates the similarity. The lesson management server 50 determines the student terminal 30 whose similarity is equal to or greater than a threshold as the student terminal 30 from which the screen data will be shared. Note that if there are multiple student terminals 30 whose similarity between the screen data and the correct answer data is equal to or greater than the threshold, the student terminal 30 with the least number of sharing attempts may be determined as the student terminal 30, or the teacher may select the student terminal 30. In FIG. 16, it is assumed that student terminal 30B is determined as the sharer.

[0141] (7) The lesson management server 50 transmits the screen data received from the student terminal 30B to the teacher terminal 10. The teacher terminal 10 displays the screen data on the display 506. In FIG. 16, it can be seen that the teacher terminal 10 is displaying the screen data of the student terminal 30B.

[0142] <Action or Processing> 17 is a sequence diagram illustrating the process in which the class management server 50 determines the student terminals 30 to be shared with the teacher terminal 10 based on the similarity, and shares the screen data of the determined student terminals 30. The explanation of FIG. 17 will mainly focus on the differences from FIG. 7. The processes of steps S71 to S76 may be the same as steps S1 to S6 in FIG. 7.

[0143] S76: The first communication unit 55 of the class management server 50 receives a facial image of the student who made the gesture of raising their hand. Upon receiving the facial image, the facial recognition unit 54 analyzes the facial image and identifies the student. The facial recognition unit 54 repeatedly identifies all students who raised their hand until the teacher inputs an end of answering or until the time limit is reached.

[0144] S77: The face recognition unit 54 transmits the student ID identified by recognizing the face image to the comparison unit 59.

[0145] S78: When the comparison unit 59 receives the student ID, the comparison unit 59 requests screen data from each student terminal 30 identified by the student ID via the second communication unit 52.

[0146] S79: When the communication unit 31 of the student terminal 30 receives the request for screen data, the communication unit 31 transmits the screen data to the class management server 50. The second communication unit 52 receives the screen data and transmits it to the comparison unit 59.

[0147] S80: The comparison unit 59 calculates the similarity between the screen data (answer) sent from the student terminal 30 and the correct answer data. In the case of options, the similarity is 0 or 100, and in the case of digital data, the similarity may be the ratio of the number of characters in the string of characters in the correct answer that match the answer. In the case of raster data, the similarity is determined by pattern matching, with the maximum being 100 points, for example.

[0148] The lesson management server 50 may send to the teacher terminal 10 a list of students sorted in descending order of similarity, or may send to the teacher terminal 10 a list of students sorted in descending order of similarity, order of hand raising, and number of times shared. By displaying the student list on the teacher terminal 10, the teacher can select students based on the display order.

[0149] S81: Next, the teacher inputs an operation to close the answering process into the teacher terminal 10. If a time limit is set, the operation to close the answering process is not necessary.

[0150] S82: The operation receiving unit 13 receives the operation, and the communication unit 11 transmits a response completion message to the class management server 50.

[0151] S83: When the first communication unit 55 of the class management server 50 receives the message that the answer has been completed, the comparison unit 59 transmits the similarity of each student terminal 30 that has raised its hand to the identification unit 51.

[0152] S84: The identification unit 51 determines the student terminal 30 whose similarity is equal to or greater than the threshold as the student terminal 30 that will share the screen data. If there are multiple student terminals 30 whose similarity is equal to or greater than the threshold, the identification unit 51 refers to the number of times of sharing and determines the student terminal 30 that has been shared the least as the student terminal 30 that will share the screen data. Alternatively, the student who raised their hand first or the student selected by the teacher may be selected. The sharing count management unit 53 increments the number of times of sharing for the student in the sharing count storage unit 56 by one based on the student ID identified by the identification unit 51.

[0153] S85: The identifying unit 51 transmits the student ID of the determined student terminal 30 to the comparing unit 59.

[0154] S86: The comparison unit 59 transmits the screen data of the student terminal 30 identified by the student ID to the teacher terminal 10 via the first communication unit 55.

[0155] S87: When the communication unit 11 of the teacher terminal 10 receives the screen data, it transmits it to the display control unit 12. The display control unit 12 of the teacher terminal 10 causes the display 506 to display the screen data transmitted from the student terminal 30.

[0156] In this embodiment, the screen data of the student terminal 30 that answered correctly is shared, but the teacher terminal 10 may display the screen data of the student terminal 30 that the comparison unit 59 determines not to be similar. The teacher can grasp the tendency of mistakes and provide guidance on misunderstandings.

[0157] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, the student terminals 30 that have sent answers similar to the correct answer data are determined from among the students who raised their hands, thereby reducing the risk that the teacher terminal 10 will share an incorrect answer.

[0158] [Fifth embodiment] In this embodiment, an information sharing system 100 will be described in which a teacher can set how the identification unit 51 determines the student terminal 30.

[0159] 18 shows a determination method setting screen 200 displayed on the teacher terminal 10. The determination method setting screen 200 displays options corresponding to radio buttons: first come, first served 201, least shared 202, teacher's selection (first come, first served) 203, teacher's selection (least shared) 204, and similarity to correct answer 205. Least shared 202 corresponds to the first embodiment, teacher's selection (first come, first served) 203 corresponds to the second embodiment, teacher's selection (least shared) 204 corresponds to the third embodiment, and similarity to correct answer 205 corresponds to the fourth embodiment.

[0160] Therefore, according to this embodiment, the teacher can set a method for determining which student terminals 30 the teacher terminal 10 displays.

[0161] [Sixth embodiment] In this embodiment, an information sharing system will be described in which the teacher terminal 10 determines the student terminals 30 with which screen data is to be shared.

[0162] Figure 19 is a functional block diagram that explains the functions of the lesson management server 50, student terminal 30, and teacher terminal 10 by dividing them into blocks. The explanation of Figure 19 will mainly focus on the differences from Figure 4. The teacher terminal 10 of this embodiment has a communication unit 11, a display control unit 12, an operation reception unit 13, an imaging unit 14, a gesture recognition unit 15, a screen data generation unit 16, an identification unit 51, a sharing count management unit 53, and a face recognition unit 54. In other words, the teacher terminal 10 has the functions of the lesson management server 50.

[0163] Meanwhile, the class management server 50 has a first communication unit 55 and a second communication unit 52. In this configuration, the class management server 50 manages the student terminals 30 and teacher terminals 10 participating in the same class. The class management server 50 also controls communication, such as session management between the student terminals 30 and the teacher terminal 10. Meanwhile, the teacher terminal 10 recognizes students and determines the student terminals 30 with which to share screen data.

[0164] Also, as shown in Figure 20, the lesson management server 50 may not be necessary. Figure 20 is a functional block diagram that explains the functions of the student terminal 30 and the teacher terminal 10. As shown in Figure 20, if the student terminal 30 and the teacher terminal 10 communicate directly, the lesson management server 50 becomes unnecessary and installation costs can be reduced.

[0165] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0166] For example, the teacher terminal 10 may send a model answer to a student who raises their hand. The student can then grade their answer to see if it is correct. The teacher terminal 10 may also determine whether the answer is correct or incorrect, and if the answer is correct, send the model answer to the student terminal 30, or if the answer is incorrect, send a hint to that effect. If the number of incorrect answers exceeds a certain number, the teacher terminal 10 may send the model answer to the student terminal 30.

[0167] The teacher terminal 10 may be called an electronic whiteboard, an electronic information board, or the like. This embodiment is not limited to electronic whiteboards and can be suitably applied to any information processing device with a touch panel. Examples of information processing devices equipped with a touch panel include PCs, tablet terminals, and smartphones with touch panels. These are normally general-purpose information processing devices, but when an application that functions as the teacher terminal 10 is executed, the user can operate them as the teacher terminal 10.

[0168] The teacher terminal 10 may also be a device that projects an image using a projector. In this case, the teacher terminal 10 detects the coordinates pointed to by the method described in this embodiment, and the projector projects a handwritten line or the like based on the trajectory of the coordinates.

[0169] Furthermore, in this embodiment, a gesture of raising a hand is detected, but a nodding gesture, a thumbs-up gesture, or a head-shaking gesture may also be detected as having the same meaning as raising a hand.

[0170] In addition, the configuration examples in Figure 6 and other figures are divided according to main functions to make it easier to understand the processing by the class management server 50, teacher terminal 10, and student terminal 30. The method of dividing the processing units and their names do not limit the present invention. The processing by the class management server 50, teacher terminal 10, and student terminal 30 can be divided into even more processing units depending on the processing content. Furthermore, one processing unit can be divided so that it includes even more processes.

[0171] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0172] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful, and robust user interfaces. Such user interfaces provide better interaction between humans and machines. [Explanation of symbols]

[0173] 10 Teacher's terminal 30 Student Terminals 50 Class Management Server 100 Information Sharing System [Prior art documents] [Patent documents]

[0174] [Patent Document 1] JP 2017-173927 A

Claims

1. An information sharing system in which devices and terminal devices can communicate with a server device, The device or the server device a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of the terminal device and recognizes a gesture by analyzing the image data; a sharing count management unit that manages the number of times the information displayed by the terminal device has been shared by the device in association with a user of the terminal device; a face recognition unit that performs face recognition on the image data and identifies the user who made the gesture; an identification unit that identifies the user who has the fewest number of sharing counts among the users who have made the gesture; a communication unit that receives the information from the terminal device associated with the user identified by the identification unit, The device comprises: displaying the information received by the communication unit; An information sharing system characterized by:

2. The device or the server device: an order determination unit that determines the order of the users in the order in which the face recognition unit recognizes the users who have made the gesture; The device comprises: and displaying the identification information of the users in the order determined by the order determination unit in association with the number of times the users have shared the information. The information sharing system according to claim 1 .

3. The communication unit receiving, from the terminal device, the information associated with the user whose selection has been accepted, from the identification information of the users displayed in the order determined by the order determination unit; The device comprises: displaying the information received by the communication unit; The information sharing system according to claim 2 .

4. The device or the server device: a sorting unit that sorts the users who have made the gesture in descending order of the number of times of sharing, The device comprises: displaying the identification information of the user who performed the gesture in the order sorted by the sorting unit; The information sharing system according to claim 1 .

5. the communication unit receives the information from the terminal device associated with the user with the smallest number of sharing times; The device comprises: displaying the information received by the communication unit; The information sharing system according to claim 4.

6. The communication unit receiving, from the terminal device, the information associated with the user whose selection has been accepted, from the identification information of the users displayed in the order sorted by the sorting unit; The device comprises: displaying the information received by the communication unit; The information sharing system according to claim 4.

7. The device or the server device: The terminal device stores correct answer data for questions answered by the user, a comparison unit that calculates a similarity between the information acquired from the terminal device associated with the user who performed the gesture and the correct answer data; The communication unit receiving the information from the terminal device that transmitted the response that is determined to be most similar by the comparison unit; The device comprises: displaying the information received by the communication unit; The information sharing system according to claim 1 .

8. The communication unit receiving the information from the terminal device associated with the user with the fewest number of sharing counts among users of the terminal devices that have transmitted the information for which the degree of similarity calculated by the comparison unit is equal to or greater than a threshold value; The device comprises: displaying the information received by the communication unit; The information sharing system according to claim 7.

9. The device comprises: displaying the information acquired from the terminal device associated with the user who performed the gesture earliest; displaying the information acquired from the terminal device associated with the user who has performed the gesture with the smallest number of sharing counts, or displaying the information acquired from the terminal device associated with the user who has selected the user identification information displayed in the order in which the gesture was made, or Displaying the information acquired from the terminal device associated with the user who performed the gesture and accepted the selection from the identification information of the users displayed in descending order of the number of times of sharing, or displaying the information acquired from the terminal device that transmitted the response that is determined to be most similar by the comparison unit; A screen is displayed in which the user can select one of the following methods: displaying the information received from the terminal device determined by the determination method; The information sharing system according to claim 7.

10. An information sharing method performed by an information sharing system in which devices and terminal devices can communicate with a server device, The device or the server device A process of acquiring image data from an imaging device that captures an image of a user of the terminal device and recognizing a gesture by analyzing the image data; The number of times the information displayed by the terminal device has been shared by the device is stored in association with the user of the terminal device; performing face recognition on the image data to identify the user who made the gesture; A process of identifying the user who has the fewest number of sharing counts among the users who have performed the gesture; receiving the information from the terminal device associated with the identified user; The device comprises: displaying the information received from the terminal device; An information sharing method comprising:

11. An apparatus having a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of a terminal device and recognizes gestures by analyzing the image data, and a server device that communicates with the terminal device via a network, a sharing count management unit that manages the number of times the information displayed by the terminal device has been shared by the device in association with a user of the terminal device; a face recognition unit that performs face recognition on the image data received from the device and identifies the user who made the gesture; an identification unit that identifies the user who has the fewest number of sharing counts among the users who have made the gesture; a communication unit that receives the information from the terminal device associated with the user identified by the identification unit and transmits the information to the device; A server device comprising:

12. A device having a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of a terminal device and recognizes gestures by analyzing the image data, and a server device that communicates with the terminal device via a network, a sharing count management unit that manages the number of times the information displayed by the terminal device has been shared by the device in association with a user of the terminal device; a face recognition unit that performs face recognition on the image data received from the device and identifies the user who made the gesture; an identification unit that identifies the user who has the fewest number of sharing counts among the users who have made the gesture; a communication unit that receives the information from the terminal device associated with the user identified by the identification unit and transmits the information to the device; A program to function as a

13. A device that communicates with a terminal device, a gesture recognition unit that acquires image data from an imaging device that captures an image of a user of the terminal device and recognizes a gesture by analyzing the image data; a sharing count management unit that manages the number of times the information displayed by the terminal device has been shared by the device in association with a user of the terminal device; a face recognition unit that performs face recognition on the image data received from the device and identifies the user who made the gesture; an identification unit that identifies the user who has the fewest number of sharing counts among the users who have made the gesture; a communication unit that receives the information from the terminal device associated with the user identified by the identification unit; a display control unit that displays the information received by the communication unit; An apparatus characterized by having:

Images