Necklace-type terminal and data processing system

Abstract

The objective is to provide a necklace-type terminal and data processing system that can maximize students' motivation and learning efficiency. [Solution] The system includes a sensor that detects the biometric data of the student wearing the device, a camera that captures the student's facial expressions and movements, a microphone, a data collection unit that collects the outputs of the sensor, the camera, and the microphone, and a provisioning unit that provides a learning plan generated by a generative AI based on the outputs collected by the data collection unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a necklace-type terminal and a data processing system

[0002] thereof.

Background Art

[0003] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

[0004] Patent Document 2 discloses an educational support system including a performance information database for storing the degree of understanding of digital textbooks for each student, and an AR system having teacher's AR (Augmented Reality) glasses worn by a teacher. The AR system includes a selection reception unit for receiving a selection of a target student who is the student to be displayed, and a display control unit for color-coding and displaying a browsing page, which is a page being browsed in the digital textbook, according to the degree of understanding of the received target student.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, conventional technology still has room for improvement in collecting users' biometric data.

[0007] Furthermore, conventional support systems lack the functionality to dynamically optimize learning plans based on each student's cognitive characteristics and biometric data, leaving room for improvement in order to maximize students' motivation and efficiency.

[0008] This invention was made in consideration of the above facts, and aims to provide a necklace-type terminal and a data processing system that can maximize students' motivation and efficiency in learning. [Means for solving the problem]

[0009] A first aspect of the technology of this disclosure is a necklace-type terminal that includes a sensor for detecting the biometric data of a student wearing the device, a camera for capturing the student's facial expressions and movements, a microphone, a data collection unit for collecting the outputs of the sensor, the camera, and the microphone, and a providing unit for providing a learning plan generated by a generative AI based on the outputs collected by the data collection unit.

[0010] A second aspect of the technology of the present disclosure is a necklace-type terminal of the first aspect, further comprising a communication unit that transmits the outputs of the camera, the sensor, and the microphone, each collected by the data acquisition unit, to a data processing unit and receives the learning plan from the data processing unit.

[0011] A third aspect of the technology of this disclosure is a necklace-type terminal of the first aspect, wherein the providing unit provides the learning plan by transmitting and displaying the learning plan on a display device.

[0012] A fourth aspect of the technology of the present disclosure includes a necklace-type terminal of the first aspect, a data processing device that generates the learning plan using the generative AI based on the output collected by the data collection unit, and a display device that displays the learning plan provided by the provision unit.

[0013] A fifth aspect of the technology of this disclosure is a data processing system of the fourth aspect, wherein the data processing device generates a prompt requesting the generation of a learning plan, including the collected output, inputs it to the generative AI, and generates a learning plan by obtaining a generation result. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system in its basic form. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a necklace-type terminal in its basic form. [Figure 3] This is a side view showing the configuration of a necklace-type terminal in its basic form. [Figure 4] This is a top view showing the configuration of a necklace-type terminal in its basic form. [Figure 5] The functional configuration of the control unit of the necklace-type terminal in its basic form is shown in general terms. [Figure 6] The functional configuration of the specific processing unit of the data processing device in the basic configuration is outlined below. [Figure 7] An example of the operation flow of a specific process performed by a data processing device in the basic configuration is shown in outline. [Figure 8] This is a conceptual diagram showing an example of the configuration of the data processing system according to this embodiment. [Figure 9] The functional configuration of the control unit of the necklace-type terminal according to this embodiment is schematically shown. [Figure 10] This is a sequence diagram showing the flow of a specific process in this embodiment. [Figure 11] This flowchart shows an example of the processing flow performed at the necklace-type terminal of the data processing system according to this embodiment. [Figure 12] This flowchart shows an example of the processing flow performed in the data processing device of the data processing system according to this embodiment.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a data processing apparatus, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] (basic form) Before describing the embodiments, a data processing system relating to the basic form will be described. Figure 1 shows an example of the configuration of the data processing system 10 according to the basic embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this basic form and embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of this disclosure, and the necklace-type terminal 14 is an example of a "necklace-type terminal" according to the technology of this disclosure.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The necklace-type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 38, speaker 40, and camera 42 are also connected to the bus 52.

[0026] The user 20 wearing the necklace-type terminal 14 may be, for example, a patient whose health condition is being diagnosed, or a regular user.

[0027] The microphone 38 picks up the voice emitted by the user 20, who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions from the user 20 by receiving the voice emitted by the user 20. The microphone 38 captures the voice emitted by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio according to the instructions from the processor 46. The speaker 40 is, for example, a directional speaker and outputs audio towards the user 20's ears.

[0028] Sensor 39 is a sensor that detects biometric data of the user 20, who is wearing the necklace-type terminal. For example, sensor 39 may be a heart rate sensor or a blood oxygen sensor.

[0029] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person). Camera 42 captures, for example, the user's facial expressions and movements.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0032] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. The storage 32 stores a specific processing program 56. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58. The data generation model 58 is used by the specific processing unit 290. The storage 32 also includes a data storage unit 62.

[0034] In the necklace-type terminal 14, data acquisition processing is performed by the processor 46. The storage 50 stores the data acquisition program 60. The processor 46 reads the data acquisition program 60 from the storage 50 and executes the read data acquisition program 60 on the RAM 48. The data acquisition processing is realized by the processor 46 operating as a control unit 46A according to the data acquisition program 60 executed on the RAM 48.

[0035] As shown in Figures 3 and 4, the necklace-type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. Figures 3 and 4 show an example where two microphones 38 are positioned in front of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two sensors 39 are positioned to the right and left of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two speakers 40 are positioned to the right rear and left rear of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two cameras 42 are positioned to the right front and left front of the user 20 when the user 20 wears the necklace-type terminal 14. Finally, they show an example where two sensors 39 are positioned inside the necklace-type terminal 14 so as to contact the user 20's neck when the user 20 wears the necklace-type terminal 14.

[0036] Next, we will explain the processing of the control unit 46A when the necklace-type terminal 14 performs data collection processing to collect data.

[0037] In this basic configuration, data collection processes collect the user's biometric data in real time. Furthermore, it collects not only biometric data but also all surrounding environment data. This makes it possible to detect early signs of conditions such as Alzheimer's disease and dementia. It also allows for monitoring of the user's health status (e.g., heart disease).

[0038] As shown in Figure 5, the control unit 46A includes a data acquisition unit 100 and a communication unit 102.

[0039] The data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42, respectively.

[0040] The communication unit 102 transmits the outputs of the microphone 38, sensor 39, and camera 42, which are collected by the data acquisition unit 100, to the data processing unit 12.

[0041] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs specific processing to obtain a response corresponding to a user utterance.

[0042] In this basic configuration, the specific processing involves obtaining a response corresponding to the user's utterance picked up by the microphone 38 of the necklace-type terminal 14 using the data generation model 58.

[0043] As shown in Figure 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0044] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, received from the necklace-type terminal 14, in the data storage unit 62. Alternatively, the outputs may be stored in the database 24 instead of, or in addition to, the data storage unit 62.

[0045] The input unit 292 acquires user utterances received by the necklace-type terminal 14. Specifically, it acquires user utterances picked up by the microphone 38 of the necklace-type terminal 14.

[0046] The processing unit 294 performs specific processing using the data generation model 58. Specifically, it inputs a prompt including user utterances to the data generation model 58 and obtains a generation result. At this time, the outputs of the sensor 39 and camera 42 collected by the data acquisition unit 100 may also be included in the prompt.

[0047] The output unit 296 transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the specific processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the specific processing. The control unit 46A transmits the audio data indicating the user utterance acquired by the microphone 38 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the user utterance.

[0048] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0049] The outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 62 are used, for example, to diagnose the health status of user 20. In this case, the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 62 may be transmitted to a terminal on the medical institution's side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 62 to diagnose the health status of user 20.

[0050] Next, the operation of the data processing system 10 will be explained.

[0051] First, let's explain an example of the data collection process flow.

[0052] When user 20 is wearing the necklace-type terminal 14, the data acquisition unit 100 sequentially collects the outputs of the microphone 38, sensor 39, and camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, sensor 39, and camera 42 collected by the data acquisition unit 100 to the data processing unit 12.

[0053] Next, an example of the flow of a specific process will be explained with reference to Figure 7. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 62.

[0054] In step S300, the processing unit 294 determines whether a predetermined trigger condition is met. Specifically, the trigger condition may be that the user utterance picked up by the microphone 38 contains a specific word (for example, the name of the agent installed in the necklace-type terminal 14) or a phrase (for example, "Hi! XX" (where XX is the name of the agent)).

[0055] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 terminates the specific processing.

[0056] In step S301, the processing unit 294 generates a prompt by adding an instruction to obtain the result of a specific process to the text representing the user utterance picked up by the microphone 38.

[0057] For example, a prompt such as "The user is speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech) can be generated. Alternatively, the outputs of sensor 39 and camera 42 can be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and video data representing the user's surroundings. The user is also speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech)

[0058] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58 and obtains the result of a specific process based on the output of the data generation model 58.

[0059] In step S304, the output unit 296 outputs the result of the specific processing to the necklace-type terminal 14 and terminates the specific processing.

[0060] (Embodiment) Next, the data processing system according to this embodiment will be described. Figure 8 shows an example of the configuration of the data processing system 11 according to this embodiment.

[0061] The data processing system 11 according to this embodiment is configured to further include a display device 16 compared to the basic form.

[0062] The display device 16 has the function of displaying images transmitted from an external device such as a necklace-type terminal 14 or a data processing device 12. For example, smart glasses or a mobile terminal such as a smartphone may be used, or other devices such as a personal computer may be used.

[0063] Furthermore, as shown in Figure 8, the display device 16 may be connected to the necklace-type terminal 14 and the data processing device 12 via the network 54, or it may be directly wirelessly connected to the necklace-type terminal 14 and / or the data processing device 12 via wireless communication such as Wi-Fi or Bluetooth®.

[0064] Furthermore, the display device 16 may be configured to include a camera and be capable of applying AR (Augmented Reality) technology. That is, the display device 16 may display images captured by the camera and be capable of overlaying digital information (augmented reality content) onto the displayed images.

[0065] In this embodiment, the necklace-type terminal 14, compared to the basic form, further includes a supply unit 104 in the control unit 46A, as shown in Figure 9.

[0066] The provision unit 104 provides learning plans and augmented reality content generated by a generative AI based on data collected by the data collection unit 100, as well as a discussion function with at least one of other students and teachers.

[0067] In this embodiment, the data processing device 12 performs a specific process to propose an optimal learning plan using data collected from the necklace-type terminal 14 worn by the student. Examples of proposed learning plans include reviewing areas where understanding is weak, and suggesting additional learning materials based on the student's interests. For example, the specific process involves generating a learning plan based on the data collected by the data collection unit 100 of the necklace-type terminal 14, using a generative AI that corresponds to the student's stress level and cognitive characteristics.

[0068] Furthermore, in this embodiment, the data processing device 12 has multiple functions, including a function to provide interactive learning using augmented reality (AR), a function to promote community learning, a function to manage student health, a multilingual support function, and a skill development function.

[0069] The interactive learning function, for example, utilizes AR technology through the necklace-type terminal 14 and display device 16 to enable students to integrate the physical world with digital information in their learning. Specifically, in a history lesson, historical places and events can be recreated using AR on the display device 16, allowing students to learn as if they were actually there. This deepens their understanding of the learning material and makes them more interested in the subject matter.

[0070] Features that promote community learning include, for example, the ability to connect with other students and teachers in real time via a necklace-type device 14, allowing them to collaborate on projects and engage in discussions. This feature fosters social skills and cooperation as students learn from and exchange ideas with each other. AI analyzes dialogue and activities within the community and suggests optimal group formations and role assignments.

[0071] The student health management function monitors not only students' learning activities but also their health status. For example, if students study for extended periods, the necklace-type device 14 may prompt them to take a break or suggest simple exercises. It also integrates data on diet and sleep to help students maintain a balanced lifestyle. This data may also be shared with parents and medical professionals.

[0072] The multilingual support feature aims to cultivate students with a global perspective by providing AI-powered multilingual capabilities. For example, it will offer lesson content in multiple languages ​​and provide content to foster intercultural understanding and international awareness. This will allow students to learn while being exposed to the cultures and languages ​​of other countries.

[0073] The skills development function provides programs to cultivate skills that will be in demand in the future (e.g., critical thinking, problem-solving ability, creativity, etc.). For example, AI analyzes students' learning history and performance and creates special training plans to strengthen the skills they will need in the future. This allows students to efficiently acquire the skills they will need in the future.

[0074] Next, an example of the flow of a specific process in this embodiment will be described. Figure 10 is a sequence diagram showing the flow of a specific process in this embodiment.

[0075] The necklace-type terminal 14 worn by the student acquires the student's biometric data, captured images, and audio.

[0076] The acquired data is transmitted from the communication unit 102 of the necklace-type terminal 14 to the data storage unit 62 of the data processing device 12 and stored therein.

[0077] The specific processing unit 390 generates a learning plan and augmented reality content using the data stored in the data storage unit 62. Specifically, the processing unit 294 generates a prompt requesting the generation of a learning plan and augmented reality content, including the collected data, inputs it to the data generation model 58, and generates the learning plan and augmented reality content by obtaining the generation result. For example, the generation of a learning plan and augmented reality content involves generating a prompt requesting the generation of a learning plan and augmented reality content according to the learning content and inputting it to the generative AI. When generating augmented reality content, the generative AI may also be configured to generate augmented reality content that matches the captured image by inputting an image captured by the camera of the display device 16. As a prompt requesting the generation of a learning plan, it is conceivable to input the collected data and use a prompt such as, "This data consists of the student's biometric data, captured images, and voice. Please generate a learning plan to improve this student's motivation to learn." At this time, the generative AI generates a learning plan that shows the learning materials to be studied for each period. Furthermore, a possible prompt for requesting the generation of augmented reality content could be one in which the collected data is input, along with the prompt: "This data consists of the student's biometric data, captured images, and audio. Please generate augmented reality content to improve this student's motivation to learn." In this case, a generative AI would generate augmented reality content that depicts an image of the student achieving their learning goals, or a vision of the student's future after achieving those goals.

[0078] The learning plan and augmented reality content generated by the specific processing unit 390 are transmitted to the necklace-type terminal 14.

[0079] In the necklace-type terminal 14, the communication unit 102 receives the generated learning plan and augmented reality content, and the provision unit 104 provides the learning plan and augmented reality content to the student. For example, the learning plan and augmented reality content are provided to the student by being transmitted to and displayed on the display device 16. Specifically, in a history lesson, historical places and events are recreated on the display device 16 using augmented reality content. This allows students to learn as if they were actually there, deepening their understanding of the learning content and making them more interested in the subject matter.

[0080] Furthermore, when providing students with learning plans and augmented reality content, the microphone 38 and speaker 40 of the necklace-type terminal 14 allow students to communicate and discuss with other students and teachers wearing necklace-type terminals 14. This enables students to learn from each other, exchange opinions, and cultivate social skills and cooperation. Additionally, by analyzing dialogue and activities within the community using AI, it becomes possible to suggest optimal group formations and role assignments.

[0081] Next, we will describe the specific processing performed in each part of the data processing system 11 according to this embodiment.

[0082] First, the processing performed in the necklace-type terminal 14 will be described. Figure 11 is a flowchart showing an example of the processing flow performed in the necklace-type terminal 14 of the data processing system 11 according to this embodiment. Note that the processing in Figure 11 starts, for example, when the power supply (not shown) of the necklace-type terminal 14 is turned on.

[0083] In step S400, the data acquisition unit 100 acquires biometric data, captured images, and audio data. Specifically, the data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42.

[0084] In step S402, the communication unit 102 transmits the acquired data (biometric data, captured image data, and audio data) to the data processing unit 12.

[0085] In step S404, the communication unit 102 determines whether or not it has received the learning plan and augmented reality content from the data processing unit 12. If it has received them (step S404; Yes), it proceeds to step S406. On the other hand, if it has not received them in step S404 (step S404; No), it proceeds to step S406.

[0086] In step S406, the provision unit 104 displays the received learning plan and augmented reality content on the display device 16. That is, the display device 16 can display the learning plan or overlay augmented reality content on an image captured by the camera. This allows for the provision of an optimal learning plan to students. Furthermore, displaying augmented reality content deepens understanding of the learning material and encourages students to engage with it with interest. Note that the image overlaid with augmented reality content may be an image provided by a computer operated by the teacher instead of an image captured by the camera.

[0087] In step S408, the control unit 46A determines whether or not to terminate the process. This determination is made, for example, by determining whether or not a power supply (not shown) has been turned off. If the determination is negative (S408; No), the process returns to step S400 and the above process is repeated. On the other hand, if the determination is positive (S408; Yes), the process of the necklace-type terminal 14 is terminated.

[0088] Next, a specific process performed by the data processing device 12 of this embodiment will be described. Figure 12 is a flowchart showing an example of the processing flow performed by the data processing device 12 of the data processing system 11 according to this embodiment.

[0089] In step S500, the input unit 292 determines whether or not data has been received from the necklace-type terminal 14. If data has been received (S500; Yes), the process proceeds to step S502. On the other hand, if data has not been received (S500; No), the process proceeds to step S504.

[0090] In step S502, the input unit 292 stores the biometric data, captured image, and audio data in the data storage unit 62.

[0091] In step S504, the processing unit 294 determines whether or not to generate a learning plan and augmented reality content. This determination may be made, for example, by determining whether the amount of data collected from the necklace-type terminal 14 has reached a predetermined threshold, or by determining whether or not a predetermined amount of time has elapsed since the last generation of a learning plan and augmented reality content. Alternatively, it may be determined whether or not a request has been received from the necklace-type terminal 14. If a learning plan and augmented reality content are to be generated (S504; Yes), the process proceeds to step S506. On the other hand, if they are not to be generated (S504; No), the process returns to step S500 and the above process is repeated.

[0092] In step S506, the processing unit 294 retrieves each of the stored data (biological data, captured image data, and audio data) from the data storage unit 62.

[0093] In step S508, the processing unit 294 generates a learning plan and augmented reality content using the data. Specifically, the processing unit 294 generates a prompt requesting the generation of a learning plan and augmented reality content, including the collected data, inputs it to the data generation model 58, and generates the learning plan and augmented reality content by obtaining the generation result. In addition to the collected data, other information such as images captured by the camera of the display device 16 may also be input to the generative AI to generate the learning plan and augmented reality content. For example, by inputting captured images to the generative AI to generate augmented reality content, augmented reality content can be generated that matches the captured images.

[0094] In step S510, the output unit 296 transmits the generated learning plan and augmented reality content to the necklace-type terminal 14, and the process returns to step S500 to repeat the above steps.

[0095] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0096] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0097] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0098] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0099] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0100] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0101] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0102] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0103] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0104] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0105] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference. [Explanation of Symbols]

[0106] 10 Data Processing Systems 12 Data Processing Devices 14 Necklace-type terminal 38 Microphones 39 Sensors 40 speakers 42 cameras 46A Control Unit 100 Data Acquisition Unit 102 Communications Department 290 Specific Processing Unit 292 Input section 294 Processing Unit 296 Output section< / url:>

Claims

1. A sensor that detects the biometric data of the student wearing the device, A camera that captures the facial expressions and movements of the aforementioned student, Microphone and, A data acquisition unit that collects the outputs of the sensor, the camera, and the microphone, A providing unit that provides a learning plan generated by a generative AI based on the output collected by the data collection unit, A necklace-type terminal that includes this device.

2. The necklace-type terminal according to claim 1, further comprising a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the data acquisition unit to a data processing unit and receives the learning plan from the data processing unit.

3. The necklace-type terminal according to claim 1, wherein the providing unit provides the learning plan by transmitting and displaying the learning plan on a display device.

4. The necklace-type terminal described in claim 1, A data processing device that generates the learning plan using the generative AI based on the output collected by the data collection unit, A display device that displays the learning plan provided by the aforementioned supply unit, A data processing system that includes this.

5. The data processing system according to claim 4, wherein the data processing device generates a prompt requesting the generation of a learning plan, including the collected output, inputs it to the generation system AI, and generates a learning plan by obtaining a generation result.

Images