Data processing system and data processing method

Abstract

The aim is to reduce the burden on teachers who are responsible for multiple students. [Solution] The data processing system comprises a teacher-side terminal worn by a teacher, a student-side terminal worn by each of a plurality of students, and a data processing device that is communicatively connected to the teacher-side terminal and the student-side terminal, wherein the data processing device includes a processor, the processor collects data output from at least one of a microphone, sensor, and camera provided on the student-side terminal, analyzes the data to estimate the state of the plurality of students, and provides feedback to the teacher-side terminal based on the estimated result.

Description

Technical Field

[0001] The technology of the present disclosure relates to a data processing system and a data processing method.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the educational field, it is necessary for teachers to accurately grasp the state of each student and provide optimal guidance. However, it has been difficult for teachers to grasp the state of all students. Therefore, an object of the present disclosure is to provide a system and method for reducing the burden on teachers in charge of a plurality of students.

Means for Solving the Problems

[0005] A data processing system according to a first aspect of this disclosure comprises a teacher-side terminal worn by a teacher, a student-side terminal worn by each of a plurality of students, and a data processing device that is communicatively connected to the teacher-side terminal and the student-side terminal, wherein the data processing device includes a processor, the processor collects data output from at least one of a microphone, a sensor, and a camera provided on the student-side terminal, analyzes the data to estimate the state of the plurality of students, and provides feedback to the teacher-side terminal based on the estimation results.

[0006] A data processing system according to a second aspect of this disclosure, in a data processing system according to a first aspect, wherein the processor analyzes the biometric data detected by the sensor to estimate the mental state of the multiple students and provides feedback to the teacher's terminal based on the mental state.

[0007] A data processing system according to a third aspect of this disclosure, in a data processing system according to the first or second aspect, the processor analyzes at least one of the audio data collected by the microphone and the image data captured by the camera to estimate the emotional state of the multiple students, and provides feedback to the teacher's terminal based on the emotional state.

[0008] A data processing system according to a fourth aspect of this disclosure, in a data processing system according to any one of the first to third aspects, the processor provides feedback to the teacher terminal based on the results of statistical processing of the states of the plurality of students.

[0009] A data processing system according to a fifth aspect of this disclosure, in a data processing system according to any one of the first to fourth aspects, the processor provides feedback to the student terminal based on the estimated result.

[0010] A data processing system according to the sixth aspect of this disclosure is a data processing system according to any one of the first to fifth aspects, wherein the processor acquires learning data indicating the learning history of each of the plurality of students, analyzes the learning data to generate a curriculum for each of the plurality of students, and sends a message corresponding to the curriculum to the teacher's terminal.

[0011] The data processing system according to the seventh aspect of this disclosure, in the data processing system according to the sixth aspect, includes a processor that analyzes the audio data collected by the microphone to receive questions from each of the multiple students and transmits the answers to the questions to the student terminal worn by the student who asked the question.

[0012] The data processing system according to the eighth aspect of this disclosure is a data processing system according to any one of the first to seventh aspects, wherein the student terminal is a necklace-type terminal.

[0013] The data processing system according to the ninth aspect of this disclosure is a data processing system according to the eighth aspect, wherein the teacher terminal is also the necklace-type terminal, and the processor registers whether the necklace-type terminal is the teacher terminal or the student terminal.

[0014] A data processing method according to a tenth aspect of the present disclosure is a data processing system comprising a teacher-side terminal worn by a teacher, a student-side terminal worn by each of a plurality of students, and a data processing device that is communicatively connected to the teacher-side terminal and the student-side terminal, wherein the data processing device collects data output from at least one of a microphone, a sensor, and a camera provided on the student-side terminal, analyzes the data to estimate the state of the plurality of students, and provides feedback to the teacher-side terminal based on the estimation results. [Brief explanation of the drawing]

[0015] [Figure 1]It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a necklace-type terminal. [Figure 3] It is a side view showing the configuration of the necklace-type terminal. [Figure 4] It is a top view showing the configuration of the necklace-type terminal. [Figure 5] The functional configuration of the control unit of the necklace-type terminal is schematically shown. [Figure 6] The functional configuration of the specific processing unit of the data processing device is schematically shown. [Figure 7] An example of the operation flow of specific processing by the data processing device is schematically shown. [Figure 8] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 9] An example of the first operation flow of specific processing by the data processing device according to the second embodiment is schematically shown. [Figure 10] An example of the second operation flow of specific processing by the data processing device according to the second embodiment is schematically shown.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit), etc.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory where information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F manages communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] <First Embodiment> Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of this disclosure, and the necklace-type terminal 14 is an example of a "necklace-type terminal" according to the technology of this disclosure.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The necklace-type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 38, speaker 40, and camera 42 are also connected to the bus 52.

[0027] The user 20 wearing the necklace-type terminal 14 may be, for example, a patient whose health condition is being diagnosed, or a regular user.

[0028] The microphone 38 picks up the voice emitted by the user 20, who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions from the user 20 by receiving the voice emitted by the user 20. The microphone 38 captures the voice emitted by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio according to the instructions from the processor 46. The speaker 40 is, for example, a directional speaker and outputs audio towards the user 20's ears.

[0029] Sensor 39 is a sensor that detects biometric data of the user 20, who is wearing the necklace-type terminal. For example, sensor 39 may be a heart rate sensor or a blood oxygen sensor.

[0030] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0033] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. The storage 32 stores a specific processing program 56. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58. The data generation model 58 is used by the specific processing unit 290. The storage 32 also includes a data storage unit 54.

[0035] In the necklace-type terminal 14, data acquisition processing is performed by the processor 46. The storage 50 stores the data acquisition program 60. The processor 46 reads the data acquisition program 60 from the storage 50 and executes the read data acquisition program 60 on the RAM 48. The data acquisition processing is realized by the processor 46 operating as a control unit 46A according to the data acquisition program 60 executed on the RAM 48.

[0036] As shown in Figures 3 and 4, the necklace-type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. Figures 3 and 4 show an example where two microphones 38 are positioned in front of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two sensors 39 are positioned to the right and left of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two speakers 40 are positioned to the right rear and left rear of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two cameras 42 are positioned to the right front and left front of the user 20 when the user 20 wears the necklace-type terminal 14. Finally, they show an example where two sensors 39 are positioned inside the necklace-type terminal 14 so as to contact the user 20's neck when the user 20 wears the necklace-type terminal 14.

[0037] Next, we will explain the processing of the control unit 46A when the necklace-type terminal 14 performs data collection processing to collect data.

[0038] In this embodiment, the data collection process collects the user's biometric data in real time. Furthermore, it collects not only biometric data but also all surrounding environment data. This makes it possible to detect early signs of conditions such as Alzheimer's disease and dementia. It also allows for monitoring of the user's health status (e.g., heart disease).

[0039] As shown in Figure 5, the control unit 46A includes a data acquisition unit 100 and a communication unit 102.

[0040] The data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42, respectively.

[0041] The communication unit 102 transmits the outputs of the microphone 38, sensor 39, and camera 42, which are collected by the data acquisition unit 100, to the data processing unit 12.

[0042] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs specific processing to obtain a response corresponding to a user utterance.

[0043] In the specific processing in this embodiment, a response corresponding to the user utterance picked up by the microphone 38 of the necklace-type terminal 14 is obtained using the data generation model 58.

[0044] As shown in Figure 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0045] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, respectively, received from the necklace-type terminal 14, in the data storage unit 54.

[0046] The input unit 292 acquires user utterances received by the necklace-type terminal 14. Specifically, it acquires user utterances picked up by the microphone 38 of the necklace-type terminal 14.

[0047] The processing unit 294 performs specific processing using the data generation model 58. Specifically, it inputs a prompt including user utterances to the data generation model 58 and obtains a generation result. At this time, the outputs of the sensor 39 and camera 42 collected by the data acquisition unit 100 may also be included in the prompt.

[0048] The output unit 296 transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the specific processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the specific processing. The control unit 46A transmits the audio data indicating the user utterance acquired by the microphone 38 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the user utterance.

[0049] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0050] The outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 are used, for example, to diagnose the health status of user 20. In this case, the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 may be transmitted to a terminal on the medical institution's side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 to diagnose the health status of user 20.

[0051] Next, the operation of the data processing system 10 will be explained.

[0052] First, let's explain an example of the data collection process flow.

[0053] When user 20 is wearing the necklace-type terminal 14, the data acquisition unit 100 sequentially collects the outputs of the microphone 38, sensor 39, and camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, sensor 39, and camera 42 collected by the data acquisition unit 100 to the data processing unit 12.

[0054] Next, an example of the flow of a specific process will be explained with reference to Figure 7. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 54.

[0055] In step S300, the processing unit 294 determines whether a predetermined trigger condition is met. Specifically, the trigger condition may be that the user utterance picked up by the microphone 38 contains a specific word (for example, the name of the agent installed in the necklace-type terminal 14) or a phrase (for example, "Hi! XX" (where XX is the name of the agent)).

[0056] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 terminates the specific processing.

[0057] In step S301, the processing unit 294 generates a prompt by adding an instruction to obtain the result of a specific process to the text representing the user utterance picked up by the microphone 38.

[0058] For example, a prompt such as "The user is speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech) can be generated. Alternatively, the outputs of sensor 39 and camera 42 can be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and video data representing the user's surroundings. The user is also speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech)

[0059] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58 and obtains the result of a specific process based on the output of the data generation model 58.

[0060] In step S304, the output unit 296 outputs the result of the specific processing to the necklace-type terminal 14 and terminates the specific processing.

[0061] <Second Embodiment> Figure 8 shows an example of the configuration of the data processing system 10 according to the second embodiment. In educational settings, teachers need to accurately grasp the status of each student and provide optimal guidance. However, it is difficult to grasp the status of all students in real time, which places a heavy burden on teachers. As a result, there has been a problem in that appropriate guidance and overall class management cannot be adequately carried out. In the second embodiment, in order to solve this problem, specific processing by the data processing system 10 is introduced into the educational setting.

[0062] The data processing system 10 according to the second embodiment comprises a data processing device 12 and a plurality of necklace-type terminals 14. In this figure, the data processing system 10 is shown as an example in which it comprises one data processing device 12 and four necklace-type terminals 14_0 to 14_3. However, the number of necklace-type terminals 14 is not limited to this. The data processing system 10 may have more than four necklace-type terminals 14, for example, 30 to 40.

[0063] For example, user 20 wearing necklace-type terminal 14_0 may be a teacher, while user 20 wearing necklace-type terminals 14_1 to 14_3 may be students. Here, necklace-type terminals 14_1, 14_2, and 14_3 are assumed to be worn by students A, B, and C, respectively, in the class taught by the teacher.

[0064] In other words, in this embodiment, the necklace-type terminal 14_0 is an example of a "teacher-side terminal" according to this disclosure, and the necklace-type terminals 14_1 to 14_3 are examples of "student-side terminals" according to this disclosure. Thus, the student-side terminal may be a necklace-type terminal 14, and the teacher-side terminal may also be a necklace-type terminal 14.

[0065] In this case, the specific processing unit 290 may pre-register that the necklace-type terminal 14_0 is the teacher's terminal, and the necklace-type terminals 14_1 to 14_3 are student terminals, or more specifically, that the necklace-type terminals 14_1, 14_2, and 14_3 are student terminals to be worn by student A, student B, and student C, respectively.

[0066] Such registration may be done manually by the user pressing a button (not shown) provided on the necklace-type terminal 14, or it may be done automatically based on the user's speech. In this way, when the same necklace-type terminal 14 is used as both the teacher's terminal and the student's terminal, the identification processing unit 290 may register whether the necklace-type terminal 14 is the teacher's terminal or the student's terminal.

[0067] The necklace-type terminals 14_0 to 14_3 are connected to the data processing device 12 via the network 54 in a communicative manner. Therefore, the data processing device 12 can collect the outputs of the microphone 38, sensor 39, and camera 42 provided on each of the necklace-type terminals 14_0 to 14_3. In addition, the data processing device 12 can output the results of specific processing to the speaker 40 provided on each of the necklace-type terminals 14_0 to 14_3.

[0068] Figure 9 shows an example of the first operation flow of a specific process performed by the data processing device 12 according to the second embodiment. This flow may be started automatically when the class begins, or it may be started manually by the teacher pressing the start button or by uttering a command to start.

[0069] In step S400, the processor 28 collects data output from at least one of the microphone 38, sensor 39, and camera 42 provided on the student terminal. For example, the processor 28 may collect the output of the microphone 38, sensor 39, and camera 42 provided on each of the necklace-type terminals 14_1 to 14_3 in real time.

[0070] In step S401, the processor 28 analyzes the data collected in step S400 to estimate the state of multiple students. For example, the processor 28 may analyze the biometric data detected by the sensor 39 (e.g., heart rate and blood oxygen saturation) to estimate the mental state of each of students A, B, and C (e.g., stress level and concentration level).

[0071] Alternatively, or in addition to this, the processor 28 may analyze the audio data (e.g., tone of voice) collected by the microphone 38 and the image data (e.g., facial expressions) captured by the camera 42 to estimate the emotional states of students A, B, and C. Since such analysis processing itself may be based on existing technology, a detailed explanation is omitted here.

[0072] In step S402, the processor 28 determines whether the trigger condition is met based on the results estimated in step S401. For example, the processor 28 may determine whether the mental state of any of the students meets a predetermined condition. More specifically, the processor 28 may determine whether any of the students' stress levels have exceeded a predetermined threshold, or whether any of the students' concentration levels have fallen below a predetermined threshold.

[0073] Alternatively, or in addition to the above, the processor 28 may determine whether the emotional state of any of the students is mapped to a predetermined emotion. More specifically, the processor 28 may determine whether the emotion of any of the students is mapped to "painful" or "distressed."

[0074] If it is determined in step S402 that the trigger condition is not met (No), the processor 28 proceeds to step S403. In step S403, the processor 28 determines whether the termination condition is met. For example, the processor 28 may determine whether the class has ended. Alternatively, or in addition to this, the processor 28 may determine whether the teacher has pressed the stop button or has spoken an indication that the class is ending.

[0075] If it is determined that the termination condition is met (Yes) in step S403, the processor 28 terminates this flow. On the other hand, if it is determined that the termination condition is not met (No), the processor 28 returns to step S400 and continues the flow.

[0076] If it is determined that the trigger condition is met (Yes) in step S402, the processor 28 proceeds to step S404. In step S404, the processor 28 provides feedback to the teacher terminal based on the results estimated in step S401.

[0077] For example, suppose that student A's concentration level falls below a predetermined threshold among students A, B, and C. In this case, processor 28 generates a prompt instructing the teacher to suggest advice appropriate to the student's concentration level. For example, processor 28 generates a prompt that says, "Student A's concentration level is decreasing. Please suggest advice to the teacher." Next, processor 28 inputs the generated prompt into data generation model 58 and receives a message from data generation model 58 that says, "Tell student A, 'The lesson will be over in 5 minutes, so concentrate!'"

[0078] The processor 28 then sends the acquired message to the necklace-type terminal 14_0. In response, the message is output as audio from the speaker 40 on the necklace-type terminal 14_0, allowing the teacher to say to student A, "The lesson will be over in 5 minutes, so concentrate!" In this way, the processor 28 can provide feedback to the teacher's terminal based on the student's mental state.

[0079] Similarly, suppose, for example, that the emotion of student B among students A, B, and C is mapped to "sad." In this case, processor 28 generates a prompt instructing it to suggest advice to the teacher that corresponds to the student's emotion. For example, processor 28 generates a prompt that says, "Student B is feeling sad. Please suggest advice to the teacher." Next, processor 28 inputs the generated prompt into data generation model 58 and receives a message from data generation model 58 that says, "Student B's hobby is soccer. Please talk about soccer."

[0080] The processor 28 then sends the acquired message to the necklace-type terminal 14_0. In response, the message is output as audio from the speaker 40 on the necklace-type terminal 14_0, allowing the teacher to switch to a topic about soccer. In this way, the processor 28 can also provide feedback to the teacher's terminal based on the student's emotional state.

[0081] The processor 28 can, for example, provide feedback to the teacher's terminal based on the results of estimating the student's state. While the above explanation shows feedback to the teacher's terminal as an example, it can also provide feedback to the student's terminal.

[0082] For example, if student A's level of concentration falls below a predetermined threshold, the processor 28 may send a vibration command to the necklace-type terminal 14_1. In response, the vibration function of the necklace-type terminal 14_1 is turned on, allowing student A to regain their composure and refocus. In this way, the processor 28 can also provide feedback to the student's terminal based on the estimated results.

[0083] In the above explanation, the case where the trigger condition is determined for each student is shown as an example, but it is not limited to this. For example, the processor 28 may statistically process the stress levels of students A, B, and C.

[0084] Now, suppose that a statistical value (for example, the sum or average) exceeds a predetermined threshold. In this case, processor 28 generates a prompt to the effect of, "All students' stress levels are rising. Please suggest advice for the teacher." Next, processor 28 inputs the generated prompt into data generation model 58 and receives a message from data generation model 58 to the effect of, "The class is stressed. Please take a 3-minute break."

[0085] The processor 28 then sends the acquired message to the necklace-type terminal 14_0. In response, the message is output as audio from the speaker 40 on the necklace-type terminal 14_0, allowing the teacher to prompt the class to take a break. In this way, the processor 28 can also provide feedback to the teacher's terminal based on the results of statistical processing of the status of multiple students.

[0086] Furthermore, while the above explanation provided an example of determining whether the trigger condition is met based on the student's state at that time, it is not limited to this. The processor 28 may also predict the future state of the student from the estimated trend of the student's state and determine whether the trigger condition is met based on the prediction result.

[0087] As described above, the data processing system 10 according to the second embodiment includes a teacher-side terminal worn by the teacher, student-side terminals worn by each of the multiple students, and a data processing device 12 that is communicatively connected to the teacher-side terminal and the student-side terminals. In this data processing system 10, the data processing device 12 collects data output from at least one of the microphone 38, sensor 39, and camera 42 provided on the student-side terminal, analyzes the data to estimate the state of the multiple students, and provides feedback to the teacher-side terminal based on the estimated results. As a result, the data processing system 10 according to the second embodiment can reduce the burden on the teacher to appropriately instruct a large number of students at once. This supports the optimization of instruction, thereby maximizing the learning effect of the students.

[0088] Figure 10 shows an example of a second operation flow of a specific process performed by the data processing device 12 according to the second embodiment. This flow may be executed prior to or in parallel with the flow in Figure 9.

[0089] In step S500, the processor 28 acquires learning data that shows the learning history of each of the students. For example, the processor 28 may access an external database to acquire learning data that includes each of the students' learning efforts, progress, assignment submission status, and grades.

[0090] In step S501, the processor 28 analyzes the learning data acquired in step S500 and generates a curriculum for each of the multiple students. For example, the processor 28 generates a prompt that says, "Please propose a curriculum for student C." Next, the processor 28 inputs the generated prompt, along with the learning data for student C, into the data generation model 58, and receives a message from the data generation model 58 that says, "Please have student C study page 10 of textbook X."

[0091] In step S502, the processor 28 sends a message to the necklace-type terminal 14_0 that corresponds to the curriculum generated in step S501. For example, the processor 28 may send a message obtained from the data generation model 58 in step S501 to the necklace-type terminal 14_0. In response, the message is output as sound from the speaker 40 provided on the necklace-type terminal 14_0, allowing the teacher to instruct student C to "study page 10 of textbook X." The same applies to students A and B.

[0092] In step S503, the processor 28 determines whether or not there is a question. At this time, the processor 28 may analyze the audio data collected by the microphone 38 and accept questions from each of the multiple students.

[0093] If it is determined in step S503 that there is no question (No), the processor 28 proceeds to step S504. In step S504, the processor 28 determines whether the termination condition is met. The termination condition may be the same as in step S403, for example, so a redundant explanation is omitted here.

[0094] If it is determined that the termination condition is met (Yes) in step S504, the processor 28 terminates this flow. On the other hand, if it is determined that the termination condition is not met (No), the processor 28 returns to step S503 and continues the flow.

[0095] If it is determined in step S503 that there is a question (Yes), the processor 28 proceeds to step S505. In step S505, the processor 28 sends the answer to the question to the student terminal worn by the student who asked the question.

[0096] For example, suppose that when the audio data collected by microphone 38 is analyzed, student C, one of students A, B, and C, says, "I don't understand the second question." In this case, processor 28 generates a prompt to the effect of, "Please tell me how to solve the second question on page 10 of textbook X." Next, processor 28 inputs the generated prompt into data generation model 58 to obtain the explanation for the second question on page 10 of textbook X from data generation model 58.

[0097] The processor 28 then sends the acquired explanation to the necklace-type terminal 14_3. In response, the explanation is output as audio from the speaker 40 installed in the necklace-type terminal 14_3, allowing student C to understand how to solve the second problem on page 10 of textbook X. The processor 28 repeatedly receives and answers such questions until it is determined in step S504 that the termination condition is met.

[0098] Thus, in the data processing system 10 according to the second embodiment, the data processing device 12 acquires learning data showing the learning history of each of the multiple students, analyzes the learning data to generate a curriculum for each of the multiple students, and sends a message corresponding to the curriculum to the teacher's terminal. At this time, the data processing device 12 analyzes the audio data collected by the microphone 38 to receive questions from each of the multiple students and sends the answer to the question to the student's terminal worn by the student who asked the question. As a result, the data processing system 10 according to the second embodiment can provide more personalized education to each student.

[0099] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0100] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0101] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0102] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0103] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0104] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0105] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0106] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0107] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0108] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0109] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference. [Explanation of symbols]

[0110] 10 Data Processing Systems 12 Data Processing Devices 14 Necklace-type terminal 38 Microphones 39 Sensors 40 speakers 42 cameras 46A Control Unit 100 Data Acquisition Unit 102 Communications Department 290 Specific Processing Unit 292 Input section 294 Processing Unit 296 Output section< / url:>

Claims

1. A teacher-side terminal worn by the teacher, Student-side terminals, each worn by multiple students, A data processing system comprising a data processing device that is communicatively connected to the teacher's terminal and the student's terminal, The data processing device comprises a processor, The aforementioned processor, The system collects data output from at least one of the microphone, sensor, and camera installed on the student terminal. By analyzing the aforementioned data, the status of the multiple students is estimated. Based on the estimated results, provide feedback to the teacher's terminal. Data processing system.

2. The aforementioned processor, The sensor analyzes the biometric data it detects to estimate the mental state of the multiple students. Based on the aforementioned mental state, provide feedback to the teacher's terminal. The data processing system according to claim 1.

3. The aforementioned processor, The emotional state of the multiple students is estimated by analyzing at least one of the audio data collected by the microphone and the image data captured by the camera. Provide feedback to the teacher's terminal based on the aforementioned emotional state. The data processing system according to claim 1.

4. The aforementioned processor, Based on the results of statistical processing of the status of the aforementioned multiple students, feedback is provided to the teacher's terminal. The data processing system according to claim 1.

5. The aforementioned processor, Based on the estimated results, provide feedback to the student's terminal. The data processing system according to claim 1.

6. The aforementioned processor, We obtain learning data showing the learning history of each of the aforementioned students. The learning data is analyzed to generate a curriculum for each of the multiple students. A message corresponding to the curriculum is sent to the teacher's terminal. The data processing system according to claim 1.

7. The aforementioned processor, The microphone analyzes the audio data it collects and takes questions from each of the students. The answer to the aforementioned question is transmitted to the student's device worn by the student who asked the question. The data processing system according to claim 6.

8. The aforementioned student terminal is a necklace-type terminal. A data processing system according to any one of claims 1 to 7.

9. The teacher's terminal is also the necklace-type terminal, The aforementioned processor, The necklace-type terminal is registered as either the teacher's terminal or the student's terminal. The data processing system according to claim 8.

10. A teacher-side terminal worn by the teacher, Student-side terminals, each worn by multiple students, A data processing system comprising a data processing device that is communicatively connected to the teacher's terminal and the student's terminal, The data processing device, The collection of data output from at least one of the microphone, sensor, and camera installed on the student terminal, To analyze the aforementioned data and estimate the state of the multiple students, The system includes providing feedback to the teacher's terminal based on the estimated results. Data processing method.

Images