Data processing system, data processing device, data processing method, necklace-type terminal, and interactive jewelry feedback system

Abstract

Provided is a data processing system capable of detecting that a user may have initial symptoms of dementia.　The data processing system comprises: a necklace-type terminal including a microphone that collects a user utterance of a wearer, a collection unit that collects an output of the microphone, and a communication unit that transmits the output of the microphone collected by the collection unit to an external device; and a data processing device. The data processing device includes: an input unit that receives the user utterance collected by the microphone; and a processing unit that, when a remark indicating a sign of dementia is included in the user utterance, notifies a preset notification destination that the wearer may have dementia.

Description

Data processing system, data processing device, data processing method, necklace-type terminal, and interactive jewelry feedback system

[0001] The technology of the present disclosure relates to a data processing system, a data processing device, a data processing method, a necklace-type terminal, and an interactive jewelry feedback system.

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

[0003] Japanese Patent Application Laid-Open No. 2022-180282

[0004] In such known data processing systems, it is difficult to properly interpret a user's speech and output data, making it difficult to communicate effectively. It is also difficult to determine important information from a user's dialogue, etc., and then perform document creation, data entry, and data analysis, or to properly output the results of the data analysis. Furthermore, in such known data processing systems, there is room for improvement in collecting biometric data of the user and data about the user's surroundings and accurately providing the user with the information they need or want.

[0005] A first aspect (1) of the technology of the present disclosure is a data processing system comprising: a necklace-type terminal including a microphone that picks up user utterances from a wearer, a collection unit that collects the output of the microphone, and a communication unit that transmits the microphone output collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes: an input unit that accepts the user utterance picked up by the microphone; and a processing unit that, if it is determined that there are signs of dementia based on the user utterance, notifies a pre-set notification destination that the wearer may have dementia.

[0006] A second aspect (1) of the technology disclosed herein is a data processing system of the first aspect, in which the processing unit determines that there are signs of dementia when there is a lack of consistency between multiple statements made by the wearer during the user's speech, and notifies a pre-set notification destination that the wearer may have dementia.

[0007] A third aspect (1) of the technology disclosed herein is a data processing system of the first aspect, in which the processing unit determines whether or not there are signs of dementia based on the frequency of occurrence of specific words in the user's speech.

[0008] A fourth aspect (1) of the technique of the present disclosure is the data processing system of the third aspect, wherein the specific word is a pronoun.

[0009] A fifth aspect (1) of the technology disclosed herein is a data processing system according to the first aspect, wherein the processing unit counts the number of times that the user's utterances are judged to be signs of dementia, and when the count value within a predetermined period of time reaches or exceeds a threshold value, notifies a predetermined notification destination that the wearer may have dementia.

[0010] A sixth aspect (1) of the technology of the present disclosure is a data processing system according to the first aspect, wherein the necklace-type terminal further includes a speaker, and the processing unit outputs a question of pre-prepared content as audio from the speaker, determines whether the wearer is likely to have signs of dementia based on the answer to the question picked up by the microphone, and notifies a pre-set notification destination that the wearer may have dementia if the determined possibility is equal to or greater than a pre-set value.

[0011] A seventh aspect (1) of the technology of the present disclosure is the data processing system of the first aspect, wherein the processing unit determines whether or not there are signs of dementia based on a pattern of changes in the wearer's emotions estimated from the user's speech, and if it is determined that there are signs of dementia, notifies a pre-set notification destination that the wearer may have dementia.

[0012] An eighth aspect (1) of the technology of the present disclosure is a data processing system according to any one of the first to seventh aspects, wherein the necklace-type terminal further includes a heart rate sensor that detects the wearer's heart rate, and the processing unit determines the type of dementia by performing frequency analysis of heart rate fluctuations in the wearer's heart rate data detected by the heart rate sensor, and when notifying the wearer that there is a possibility that they have dementia, it also notifies information about the determined type of dementia.

[0013] A ninth aspect of the technology of the present disclosure is a data processing system comprising: a necklace-type terminal including a microphone that picks up user utterances from the wearer, a sensor that detects biometric data of the wearer, a collection unit that collects the output of the microphone and the output of the sensor, and a communication unit that transmits the microphone output and the sensor output collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes an input unit that accepts the user utterance picked up by the microphone, and a processing unit that determines whether the wearer has signs of heart disease based on a pattern of changes in the wearer's emotions estimated from the user utterance and the biometric data, and if it is determined that the wearer has signs of heart disease, notifies a pre-set notification destination that the wearer has signs of heart disease.

[0014] A tenth aspect of the technology of the present disclosure is the data processing system of the ninth aspect, wherein the processing unit, each time collected user utterances are obtained, estimates the wearer's emotions based on the collected user utterances using an emotion engine that estimates the wearer's emotions, and determines whether or not there are signs of heart disease based on time-series changes in the estimated wearer's emotions and the biometric data.

[0015] An eleventh aspect of the technology of the present disclosure is a data processing system of the ninth or tenth aspect, wherein the sensor detects either one or both of the wearer's heart rate data and blood pressure data as biometric data.

[0016] A twelfth aspect of the technology of the present disclosure is a data processing system including: a necklace-type terminal including a sensor that detects biometric data of a wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into audio data; a collection unit that collects the outputs of the sensor and the microphone; and a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes: an input unit that accepts the biometric data collected by the sensor and the audio data picked up by the microphone; a processing unit that acquires medically important information from the audio data based on the amplitude of the wearer's emotions estimated from the biometric data; and an output unit that outputs the important information to a predetermined output destination.

[0017] A thirteenth aspect of the technology of the present disclosure is the data processing system of the twelfth aspect, wherein the processing unit inputs the amplitude of the wearer's emotion and a prompt including the voice data into a data generation model, and obtains the important information based on the output of the data generation model.

[0018] A fourteenth aspect of the technology of the present disclosure is the data processing system of the thirteenth aspect, wherein when the amplitude of the wearer's emotion satisfies a predetermined trigger condition, the processing unit inputs a prompt including the amplitude of the wearer's emotion into the data generation model, and obtains the important information based on the output of the data generation model.

[0019] A fifteenth aspect of the technology disclosed herein is a data processing system according to the thirteenth or fourteenth aspect, wherein when the voice data satisfies a predetermined trigger condition, the processing unit inputs a prompt including the voice data into the data generation model, and obtains the important information based on the output of the data generation model.

[0020] A sixteenth aspect of the technology of the present disclosure is the data processing system of the twelfth aspect, wherein the processing unit detects the content of the voice data at that time from the amplitude of the wearer's emotion estimated from the biometric data, and assigns a label indicating the type of emotion to the detected voice data.

[0021] A seventeenth aspect of the technique of the present disclosure is the data processing system of the sixteenth aspect, wherein the processing unit obtains medically important information from the audio data based on the label.

[0022] An eighteenth aspect of the technology of the present disclosure is a data processing system including: a necklace-type terminal including a sensor that detects biometric data of a wearer who is a medical professional; a microphone that picks up a conversation between the wearer and a patient and converts the conversation into audio data; a collection unit that collects the outputs of the sensor and the microphone; and a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes: an input unit that accepts the biometric data collected by the sensor and the audio data picked up by the microphone; a processing unit that acquires medically important information from the audio data based on changes in the wearer's emotions estimated from the biometric data, creates a summary that summarizes the content of the conversation so as to prioritize the important information, and records the summary in a recording unit; and an output unit that outputs the summary to a predetermined output destination.

[0023] A nineteenth aspect of the technique of the present disclosure is the data processing system of the eighteenth aspect, wherein the processing unit creates the summary in a manner that makes the important information identifiable.

[0024] A twentieth aspect of the technology of the present disclosure is the data processing system of the eighteenth aspect, wherein the processing unit inputs the wearer's emotional change pattern and a prompt including the voice data into a data generation model, and obtains the important information based on the output of the data generation model.

[0025] A 21st aspect of the technology of the present disclosure is the data processing system of the 20th aspect, wherein when the pattern of change in the wearer's emotions satisfies a predetermined trigger condition, the processing unit inputs a prompt including the pattern of change in the wearer's emotions into the data generation model, and obtains the important information based on the output of the data generation model.

[0026] A 22nd aspect of the technology of the present disclosure is a data processing system according to the 20th or 21st aspect, wherein when the voice data during the dialogue satisfies a predetermined trigger condition, the processing unit inputs a prompt including the voice data into the data generation model, and obtains the important information based on the output of the data generation model.

[0027] A 23rd aspect of the technology of the present disclosure is the data processing system of the 18th aspect, wherein the processing unit detects the content of the voice data at that time from a pattern of changes in the wearer's emotions estimated from the biometric data, and determines the importance of the detected voice data in the voice data during the conversation.

[0028] A twenty-fourth aspect of the technique of the present disclosure is the data processing system of the twenty-third aspect, wherein the processing unit creates the summary based on the importance.

[0029] A 25th aspect of the technology of the present disclosure is a data processing system including: a necklace-type terminal including a sensor that detects biometric data of a wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into audio data; a collection unit that collects the outputs of the sensor and the microphone; and a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device, wherein the data processing device includes: an input unit that accepts the biometric data collected by the sensor and the audio data picked up by the microphone; a processing unit that acquires and infers medically important information from the audio data based on a pattern of changes in the wearer's emotions estimated from at least one of the biometric data and the audio data, and acquires the inference result as diagnostic support information; and an output unit that outputs the diagnostic support information acquired by the processing unit to a predetermined output destination.

[0030] A 26th aspect of the technology of the present disclosure is a data processing system according to the 25th aspect, wherein the processing unit inputs a prompt including a pattern of change in the wearer's emotions and the voice data into a data generation model, and obtains the diagnostic assistance information based on the output of the data generation model.

[0031] A 27th aspect of the technology of the present disclosure is the data processing system of the 26th aspect, wherein the processing unit, each time the collected dialogue is obtained, estimates the emotion of the wearer based on the biometric data using an emotion engine that estimates the emotion of the wearer, and estimates the time-series change in the estimated emotion of the wearer as a change pattern of the emotion of the wearer.

[0032] A 28th aspect of the technology disclosed herein is a data processing system of any one of the 25th to 27th aspects, in which the diagnostic assistance information includes one or more types of disease and an evaluation result for each of the diseases.

[0033] A 29th aspect of the technique of the present disclosure is the data processing system of the 28th aspect, wherein one or more treatment methods are acquired for each of the diseases as the diagnostic support information.

[0034] A thirtieth aspect of the data processing device according to the disclosed technology includes an emotion recognition unit that recognizes the emotional state of a customer based on audio data including the customer's spoken voice and image data including the customer's facial expressions; a stress level derivation unit that derives the stress level of the sales representative based on biometric data of the sales representative; and a feedback generation unit that uses a data generation model to generate effective feedback for successfully conducting sales activities toward the customer based on the recognized emotional state of the customer and the derived stress level of the sales representative.

[0035] The feedback generation unit may use the data generation model to generate, as the feedback, advice on how to proceed with a business negotiation based on the recognized emotional state of the customer.

[0036] When the derived stress level of the salesperson is equal to or greater than a threshold, the feedback generation unit may use the data generation model to generate, as the feedback, a method of dealing with the stress level to reduce the stress level.

[0037] If the correlation between the recognized emotional state of the customer and the derived stress level of the sales representative is above a certain level, the feedback generation unit may use the data generation model to generate a warning to the sales representative as the feedback.

[0038] The data processing device may further include a report generation unit that uses the data generation model to generate a report including improvements and recommended approaches for the next sales negotiation based on a sales negotiation record including the content of the conversation between the customer and the sales representative during the sales negotiation, the emotional state of the customer during the sales negotiation, the stress level of the sales representative during the sales negotiation, and the content of the feedback.

[0039] A thirty-first aspect of the technology of the present disclosure is a data processing device that includes an input unit that acquires biometric data transmitted from a necklace-type terminal worn by a user, a processing unit that inputs the biometric data and a prompt that instructs the user to generate an action plan for health management based on the biometric data into a data generation model and acquires the action plan output from the data generation model, and an output unit that outputs the action plan to the necklace-type terminal.

[0040] A 32nd aspect of the technology of the present disclosure is a data processing device of the 31st aspect, wherein the input unit acquires a user utterance acquired by the necklace-type terminal for the action plan, and when the user utterance indicates a request to change the action plan, the processing unit inputs a prompt to the data generation model that instructs the change of the action plan based on the user utterance, acquires a new action plan output from the data generation model, and the output unit outputs the new action plan to the necklace-type terminal.

[0041] A thirty-third aspect of the technology of the present disclosure is a data processing device of the thirty-second aspect, wherein the prompt instructing a change to the action plan also includes a sentence instructing an analysis of whether the user utterance indicates a prompt for a change to the action plan.

[0042] A thirty-fourth aspect of the technology of the present disclosure is a data processing system including a data processing device of any one of the thirty-first to thirty-third aspects and a necklace-type terminal, wherein the necklace-type terminal includes a sensor that detects biometric data of the user, a microphone, a data collection unit that collects the outputs of the sensor and the microphone, and a communication unit that transmits the outputs of the sensor and the microphone collected by the data collection unit to the data processing device.

[0043] A thirty-fifth aspect of the technology of the present disclosure is a data processing system including a wearable device having a sensor that detects biometric data of a wearer and surrounding environmental data, and a data processing device that receives output data from the sensor and a prompt that requests a response regarding the wearer's health condition based on the output data, wherein the data processing device has a proactive intervention processing unit that analyzes the output data of the sensor to predict a risk score regarding the wearer's health condition and set alert data according to the degree of risk, and a providing unit that provides the alert data set by the proactive intervention processing unit to the wearer via the wearable device in response to the prompt.

[0044] A 36th aspect of the technology of the present disclosure is characterized in that, in the data processing system of the 35th aspect, the sensor that detects the biometric data includes at least one of a heart rate sensor, a blood pressure sensor, a blood oxygen sensor, a body temperature sensor, and a breathing pattern sensor, and the sensor that detects the environmental data includes at least one of a position sensor, a temperature sensor, and a humidity sensor.

[0045] A 37th aspect of the technology of the present disclosure is characterized in that, in the data processing system of the 35th or 36th aspect, the data processing device further has a database that stores past data including biometric data and behavioral pattern data of the wearer, and the proactive intervention processing unit obtains a response regarding the health condition of the wearer by using the output data from the sensor and the past data stored in the database in combination.

[0046] A thirty-eighth aspect of the technology of the present disclosure is characterized in that, in the data processing system of any one of the thirty-fifth to thirty-seventh aspects, the prompt is generated as a result of execution of a predictive mode identification process based on voice data of the wearer input from a microphone provided in the wearable device or a numerical determination of output data of the sensor.

[0047] A thirty-ninth aspect of the technology of the present disclosure is characterized in that, in the data processing system of any one of the thirty-fifth to thirty-eighth aspects, the wearable device further includes a speaker that outputs the response provided by the providing unit.

[0048] A fortieth aspect of the technology of the present disclosure is characterized in that in the data processing system of any one of the thirty-fifth to thirty-sixth aspects, if the response indicates an abnormality in health status, all related data is automatically saved and further analyzed or provided to a medical institution.

[0049] A data processing system according to a forty-first aspect of the present disclosure comprises a teacher terminal worn by a teacher, student terminals worn by each of a plurality of students, and a data processing device communicatively connected to the teacher terminal and the student terminal, wherein the data processing device comprises a processor that collects data output from at least one of a microphone, a sensor, and a camera provided on the student terminal, analyzes the data to estimate the status of the plurality of students, and provides feedback to the teacher terminal based on the estimated results.

[0050] A data processing system according to a 42nd aspect of the present disclosure is the data processing system according to the 41st aspect, wherein the processor analyzes biometric data detected by the sensor to estimate the mental states of the multiple students and provides feedback to the teacher's terminal based on the mental states.

[0051] A data processing system according to a 43rd aspect of the present disclosure is a data processing system according to the 41st or 42nd aspect, wherein the processor analyzes at least one of the audio data collected by the microphone and the image data captured by the camera to estimate the emotional states of the students, and provides feedback to the teacher's terminal based on the emotional states.

[0052] A data processing system according to a 44th aspect of the present disclosure is a data processing system according to any one of the 41st to 43rd aspects, wherein the processor provides feedback to the teacher's terminal based on the results of statistical processing of the status of the multiple students.

[0053] A data processing system according to a 45th aspect of the present disclosure is a data processing system according to any one of the 41st to 44th aspects, wherein the processor provides feedback to the student terminal based on the estimated result.

[0054] A data processing system according to a 46th aspect of the present disclosure is a data processing system according to any one of the 41st to 45th aspects, wherein the processor acquires learning data indicating the learning history of each of the plurality of students, analyzes the learning data to generate a curriculum for each of the plurality of students, and transmits a message according to the curriculum to the teacher's terminal.

[0055] A data processing system according to a 47th aspect of the present disclosure is the data processing system according to the 46th aspect, wherein the processor analyzes the audio data collected by the microphone to accept questions from each of the plurality of students, and transmits answers to the questions to the student terminal worn by the student who asked the question.

[0056] A data processing system according to a 48th aspect of the present disclosure is the data processing system according to any one of the 41st to 47th aspects, wherein the student terminal is a necklace-type terminal.

[0057] A data processing system according to a 49th aspect of the present disclosure is a data processing system according to the 48th aspect, in which the teacher's terminal is also the necklace-type terminal, and the processor registers whether the necklace-type terminal is the teacher's terminal or the student's terminal.

[0058] A data processing method according to a 50th aspect of the present disclosure is a data processing system comprising a teacher terminal worn by a teacher, student terminals worn by each of a plurality of students, and a data processing device communicatively connected to the teacher terminal and the student terminal, the data processing device collecting data output from at least one of a microphone, a sensor, and a camera provided on the student terminal, analyzing the data to estimate the status of the plurality of students, and providing feedback to the teacher terminal based on the estimated results.

[0059] A fifty-first aspect of the technology of the present disclosure is a necklace-type terminal including a sensor that detects biometric data of the student who is wearing the terminal, a camera that captures the facial expressions and movements of the student, a microphone, a data collection unit that collects the outputs of the sensor, the camera, and the microphone, and a provision unit that provides a learning plan generated by a generative AI based on the outputs collected by the data collection unit.

[0060] A fifty-second aspect of the technology of the present disclosure is a necklace-type terminal of the fifty-first aspect, further including a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the data collection unit to a data processing device and receives the learning plan from the data processing device.

[0061] A 53rd aspect of the technology disclosed herein is a necklace-type terminal of the 51st aspect, wherein the provision unit provides the study plan by transmitting the study plan to a display device and displaying it.

[0062] A 54th aspect of the technology of the present disclosure includes the necklace-type terminal of the 51st aspect, a data processing device that generates the study plan using the generation system AI based on the output collected by the data collection unit, and a display device that displays the study plan provided by the provision unit.

[0063] A 55th aspect of the technology of the present disclosure is a data processing system of the 54th aspect, in which the data processing device generates a prompt requesting the generation of a learning plan, including the collected output, and inputs this into the generation system AI to obtain a generation result, thereby generating a learning plan.

[0064] A fifty-sixth aspect of the data processing system according to the technology of the present disclosure comprises a necklace-type terminal including a camera that captures images of the wearer's surroundings, a sensor that detects biometric data of the wearer, a microphone, a collection unit that collects the outputs of the camera, the sensor, and the microphone, a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the collection unit to a data processing device, and a speaker that outputs a response corresponding to user utterances picked up by the microphone; and the data processing device including: an input unit that acquires the user utterances, an acquisition unit that acquires from a recipe database recipe data corresponding to a recipe selected by the wearer indicated by the user utterances, a first output unit that outputs the recipe data acquired by the acquisition unit to the necklace-type terminal, a processing unit that, when the state of the wearer identified based on the biometric data transmitted from the necklace-type terminal satisfies a predetermined condition, inputs a first prompt including the biometric data into a data generation model and uses the output of the data generation model to acquire suggestions for cooking the recipe, and a second output unit that outputs the suggestions for cooking the recipe acquired by the processing unit to the necklace-type terminal.

[0065] In a fifty-sixth aspect of the data processing system, the input unit acquires user utterances picked up by a microphone. The acquisition unit acquires recipe data corresponding to a recipe selected by the wearer indicated by the user utterance from a recipe database. The first output unit outputs the recipe data acquired by the acquisition unit to the necklace-type device. When the wearer's condition identified based on the biometric data transmitted from the necklace-type device satisfies a predetermined condition, the processing unit inputs a first prompt including the biometric data into a data generation model and acquires a cooking suggestion for the recipe using the output of the data generation model. The second output unit outputs the cooking suggestion for the recipe acquired by the processing unit to the necklace-type device. With the above configuration, the necklace-type device outputs, from the speaker, a cooking suggestion for a recipe suited to the wearer's condition identified from the wearer's biometric data. In this way, the data processing system can make cooking suggestions suited to the wearer's condition identified from the wearer's biometric data.

[0066] A 57th aspect of the data processing system relating to the technology of the present disclosure is the 56th aspect of the data processing system, wherein the processing unit adds an instruction sentence to the first prompt for outputting a suggestion to simplify the cooking steps of the recipe when the stress level of the wearer identified based on the biometric data is above a threshold.

[0067] In a fifty-seventh aspect of the data processing system, if the stress level of the wearer determined based on the biometric data is equal to or greater than a threshold, the processing unit adds an instruction sentence to the first prompt to output a suggestion to simplify the cooking steps of the recipe. By inputting the first prompt into the data generation model, the necklace-type device outputs a suggestion to simplify the cooking steps of the recipe from the speaker. Therefore, the data processing system can suggest cooking steps appropriate for the wearer's stress level.

[0068] A 58th aspect of the data processing system relating to the technology of the present disclosure is a data processing system of the 56th or 57th aspect, in which, when the value of the biometric data is equal to or greater than a threshold value, the processing unit adds an instruction sentence to the first prompt for outputting cooking suggestions that will lead to a decrease in the value of the biometric data.

[0069] In the data processing system of the fifty-eighth aspect, when the value of the biometric data is equal to or greater than a threshold, the processing unit adds an instruction to the first prompt to output recipe suggestions that will lead to a decrease in the value of the biometric data. By inputting the first prompt into the data generation model, the necklace-type device outputs recipe suggestions that will lead to a decrease in the value of the biometric data from the speaker. Therefore, the data processing system can suggest recipes that will improve the wearer's health.

[0070] A fifty-ninth aspect of the data processing system according to the technology of the present disclosure is the data processing system of the fifty-sixth, fifty-seventh, or fifty-eighth aspect, wherein, when ingredients to be used in the recipe identified based on the image of the wearer's surroundings transmitted from the necklace-type terminal include a specific ingredient for which the wearer has at least one of a food allergy and a food intolerance registered in a characteristic database indicating the wearer's characteristics related to ingredients, the processing unit inputs a second prompt including the wearer's characteristics related to the ingredients into the data generation model and uses the output of the data generation model to obtain a suggestion to exclude the specific ingredient from the recipe, and the second output unit outputs the suggestion to exclude the specific ingredient from the recipe obtained by the processing unit to the necklace-type terminal.

[0071] In a fifty-ninth aspect of the data processing system, when a specific ingredient is included in the ingredients used in a recipe, the processing unit inputs a second prompt including the wearer's characteristics related to the ingredient into the data generation model, and uses the output of the data generation model to obtain a suggestion to exclude the specific ingredient from the recipe. The second output unit outputs the suggestion to exclude the specific ingredient from the recipe obtained by the processing unit to the necklace-type device. By inputting the second prompt into the data generation model, the necklace-type device outputs the suggestion to exclude the specific ingredient from the recipe. Therefore, this data processing system can reduce the risk of the wearer consuming ingredients that may be harmful to their health.

[0072] The necklace-type terminal of the sixtieth aspect is a necklace-type terminal equipped with a sensor that detects environmental data representing the living environment, and includes a data collection unit that collects the environmental data detected by the sensor around the wearer of the necklace-type terminal, and an operation unit that activates a cleaning function around the wearer based on the collected environmental data.

[0073] A necklace-type terminal according to a sixty-first aspect is the necklace-type terminal according to the sixtieth aspect, wherein the activation unit activates the cleaning function when the environmental data exceeds a predetermined threshold value.

[0074] The necklace-type terminal of the 62nd aspect is the necklace-type terminal of the 60th aspect, and is provided with a communication unit that transmits the collected environmental data, and the operating unit activates the cleaning function in accordance with instructions corresponding to the transmitted environmental data.

[0075] A data processing system according to the 63rd aspect is a data processing system comprising the necklace-type terminal according to the 62nd aspect and a data processing device, wherein the data processing device comprises an input unit that accepts the environmental data, a processing unit that uses output from the data generation model obtained by inputting a prompt including the accepted environmental data into the data generation model to obtain an instruction to activate the cleaning function corresponding to the environmental data, and an output unit that outputs the instruction to the necklace-type terminal.

[0076] A data processing system according to a 64th aspect is a data processing system according to the 63rd aspect, wherein when the environmental data satisfies a predetermined trigger condition, the processing unit obtains an instruction to activate the cleaning function corresponding to the environmental data using output from the data generation model obtained by inputting a prompt including the environmental data into the data generation model.

[0077] A data processing system according to a 65th aspect is a data processing system according to the 63rd aspect, wherein the processing unit obtains the instruction using output from the data generation model obtained by inputting a prompt including the wearer's behavior pattern into the data generation model.

[0078] The necklace-type terminal of the 66th aspect is a necklace-type terminal equipped with a cleaning mechanism that cleans a predetermined position on the terminal itself, and is equipped with a data collection unit that collects behavioral data of the wearer of the necklace-type terminal, and an operation unit that activates the cleaning mechanism based on the collected behavioral data.

[0079] The necklace-type terminal of the 67th aspect is the necklace-type terminal of the 66th aspect, which is provided with a communication unit that transmits the behavioral data, and the operating unit operates the cleaning mechanism in accordance with instructions corresponding to the transmitted behavioral data.

[0080] A data processing system according to a 68th aspect is a data processing system comprising the necklace-type terminal according to the 67th aspect and a data processing device, wherein the data processing device comprises an input unit that accepts the behavioral data, a processing unit that uses output from the data generation model obtained by inputting a prompt including the accepted behavioral data into the data generation model to obtain an instruction to operate the cleaning mechanism corresponding to the behavioral data, and an output unit that outputs the instruction to the necklace-type terminal.

[0081] A data processing system according to a 69th aspect is the data processing system according to the 68th aspect, wherein when the behavioral data satisfies a predetermined trigger condition, the processing unit obtains an instruction to operate the cleaning mechanism corresponding to the behavioral data using output from the data generation model obtained by inputting a prompt including the behavioral data into the data generation model.

[0082] A data processing system according to a seventieth aspect is a data processing system according to a sixty-eighth aspect, wherein the processing unit obtains the instruction using output from the data generation model obtained by inputting a prompt including the wearer's schedule into the data generation model.

[0083] The necklace-type terminal of the 71st aspect is a necklace-type terminal formed to contain a self-repairing material, and is equipped with a data collection unit that collects damage data representing damage to the necklace-type terminal, and an operation unit that activates a self-repairing mechanism that acts on the self-repairing material based on the collected damage data.

[0084] A necklace-type terminal according to a seventy-second aspect is the necklace-type terminal according to the seventy-first aspect, in which at least one of heat and pressure is applied to the self-repairing material.

[0085] The necklace-type terminal of the 73rd aspect is the necklace-type terminal of the 71st or 72nd aspect, and is provided with a communication unit that transmits the damage data, and the operating unit activates the self-repair mechanism in accordance with instructions corresponding to the transmitted damage data.

[0086] A data processing system according to a 74th aspect is a data processing system comprising: a necklace-type terminal according to the 73rd aspect; and a data processing device, wherein the data processing device comprises: an input unit that accepts the damage data; a processing unit that uses output from the data generation model obtained by inputting a prompt including the accepted damage data into the data generation model to obtain an instruction to activate the self-repair mechanism corresponding to the damage data; and an output unit that outputs the instruction to the necklace-type terminal.

[0087] A data processing system according to a 75th aspect is a data processing system according to the 74th aspect, wherein when the damage data satisfies a predetermined trigger condition, the processing unit obtains an instruction to activate the self-repair mechanism corresponding to the damage data using output from the data generation model obtained by inputting a prompt including the damage data into the data generation model.

[0088] A 76th aspect of the technology of the present disclosure is a necklace-type terminal including a camera that captures images of the wearer's surroundings, a sensor that detects biometric data of the wearer, and a collection unit that collects outputs from at least one of a microphone, and an execution unit that uses the collected outputs to execute convenience processing to improve convenience when the wearer is traveling.

[0089] A 77th aspect of the technology of the present disclosure is the necklace-type terminal of the 76th aspect, wherein the convenience processing is processing that executes at least one of the following functions: a local information presentation function that, when the collection unit collects the output of the camera, identifies a travel destination using images captured by the camera and acquires and presents local information about the identified travel destination; a translation and playback function that, when the collection unit collects the output of the microphone, translates the language of speech of local people at the travel destination picked up by the microphone into the language spoken by the wearer and plays it back; and a notification function that, when the collection unit collects the output of the sensor, notifies an emergency contact if the biometric data detected by the sensor indicates an abnormality in the wearer.

[0090] A 78th aspect of the technology of the present disclosure is a necklace-type terminal of the 77th aspect, in which, when the execution unit executes the local information presentation function as the convenience processing, the collection unit further collects the output of a position detection unit that detects the position of the wearer, and the local information presentation function identifies the travel destination using the captured image and the position detected by the position detection unit.

[0091] A 79th aspect of the technology of the present disclosure is a necklace-type terminal of the 77th or 78th aspect, wherein the translation playback function is at least one of a function of translating the language spoken by local people at the travel destination into a language spoken by the wearer and playing it back, and a function of translating the language spoken by the wearer picked up by the microphone into the language of the local people and playing it back.

[0092] An 80th aspect of the technology of the present disclosure is a necklace-type terminal of the 76th or 77th aspect, further including a communication unit that transmits the output collected by the collection unit to a data processing device, and the execution unit executes the convenience processing using a response from the data processing device corresponding to the transmitted output.

[0093] An 81st aspect of the technology of the present disclosure is a data processing system including the necklace-type terminal of the 80th aspect and a data processing device, wherein the data processing device includes an input unit that accepts the output transmitted by the communication unit, a processing unit that inputs a prompt including information corresponding to the accepted output into a data generation model and obtains a response for executing the convenience processing using the output of the data generation model, and an output unit that outputs the obtained response to the necklace-type terminal.

[0094] An 82nd aspect of the technology of the present disclosure is a data processing system of the 81st aspect, in which, when the input unit receives the biometric data, the processing unit directly notifies an emergency contact as the response.

[0095] An 83rd aspect of the technology of the present disclosure is a necklace-type terminal including an acceleration sensor that detects the movement of a wearer, an electrodermal response sensor that detects biometric data of the wearer, an environmental sensor that acquires external environmental data of the wearer, a current position acquisition module that acquires the current position of the wearer, a collection unit that collects data from the acceleration sensor, the electrodermal response sensor, the environmental sensor, and the current position acquisition module while the wearer is training, and a control unit that acquires the analysis results of the data collected by the collection unit and provides feedback regarding the training to the wearer during the training.

[0096] An 84th aspect of the technology of the present disclosure is a necklace-type terminal of the 83rd aspect, further comprising a communication unit that transmits the data collected by the collection unit to a data processing device and receives a training plan generated by the data processing device based on the data.

[0097] An 85th aspect of the technology disclosed herein is a data processing system comprising the necklace-type terminal of the 83rd aspect and a data processing device that receives and stores the data collected by the collection unit.

[0098] An 86th aspect of the technology of the present disclosure is a data processing system of the 86th aspect, wherein the data processing device further includes a processing unit that analyzes and optimizes a training plan for the wearer based on the stored data.

[0099] An 87th aspect of the technology of the present disclosure is a necklace-type terminal including an environmental sensor that detects environmental data around the wearer and a collection unit that collects the output of each of the biosensors that detect biometric data of the wearer, and an execution unit that uses the collected output to execute an environmental improvement process that includes a process of presenting suggested information for dynamically improving the environment around the wearer.

[0100] A 88th aspect of the technology of the present disclosure is a necklace-type terminal of the 87th aspect, in which the environmental data is at least one of temperature, humidity, air quality, light intensity, and sound pressure.

[0101] An 89th aspect of the technology of the present disclosure is a necklace-type terminal of the 87th or 88th aspect, wherein the environmental improvement process is a process of dynamically improving at least one of the temperature, humidity, air quality, and brightness in the environment surrounding the wearer.

[0102] A 90th aspect of the technology of the present disclosure is a necklace-type terminal of the 87th or 88th aspect, further including a communication unit that transmits the output collected by the collection unit to a data processing device, and the execution unit executes the environmental improvement process using a response from the data processing device corresponding to the transmitted output.

[0103] A 91st aspect of the technology of the present disclosure is a data processing system including the necklace-type terminal of the 90th aspect and a data processing device, wherein the data processing device includes an input unit that accepts the output transmitted by the communication unit, a processing unit that inputs a prompt including information corresponding to the accepted output into a data generation model and obtains a response for executing the environmental improvement process using the output of the data generation model, and an output unit that outputs the obtained response to the necklace-type terminal.

[0104] A 92nd aspect of the technology of the present disclosure is the data processing system of the 91st aspect, wherein the processing unit directly executes control when the output of the data generation model is an output that controls an environmental device for improving the environment around the wearer.

[0105] A 93rd aspect of the technology of the present disclosure is a data processing system of the 91st or 92nd aspect, wherein the output unit further outputs the environmental data in the output received by the input unit to a user other than the wearer.

[0106] A ninety-fourth aspect of the technique of the present disclosure is the data processing system of the ninety-first or ninety-second aspect, wherein the input unit further accepts profile information of the wearer.

[0107] A ninety-fifth aspect of the technology of the present disclosure is a necklace-type terminal including: a collection unit that collects the output of a microphone that picks up the wearer's speech; and an execution unit that uses the collected output to execute a health problem suppression process that presents suggestion information for suppressing the occurrence of health problems in at least one of the wearer's mental health and physical health.

[0108] A 96th aspect of the technology of the present disclosure is a necklace-type terminal of the 95th aspect, wherein the mental health problem is an increased sense of loneliness and the physical health problem is a decline in cognitive function.

[0109] A 97th aspect of the technology of the present disclosure is a necklace-type terminal of the 95th or 96th aspect, wherein the collection unit further collects the output of a biosensor that detects biometric data of the wearer, and the execution unit uses the collected biometric data to execute, as the health problem suppression process, a process of presenting suggested information for suppressing the occurrence of physical health problems in the wearer.

[0110] A 98th aspect of the technology disclosed herein is a necklace-type terminal of the 97th aspect, wherein the biometric data is at least one of heart rate, blood oxygen concentration, body temperature, and sleep pattern.

[0111] A 99th aspect of the technology of the present disclosure is a necklace-type terminal of the 95th or 96th aspect, wherein the collection unit further collects outputs from at least one of a position detection unit that detects the position of the wearer and a fall sensor that detects the wearer's state of fall, and the execution unit uses at least one of the collected position and state of fall to execute, as the health problem prevention process, a process of presenting suggested information for preventing the wearer from developing physical health problems.

[0112] A hundredth aspect of the technology of the present disclosure is a necklace-type terminal of the ninety-fifth or ninety-sixth aspect, in which the wearer is an elderly person above a predetermined age.

[0113] A 101st aspect of the technology of the present disclosure is a necklace-type terminal of the 95th or 96th aspect, further including a communication unit that transmits the output collected by the collection unit to a data processing device, and the execution unit executes the health problem suppression process using a response from the data processing device corresponding to the transmitted output.

[0114] A 102nd aspect of the technology of the present disclosure is a data processing system including the necklace-type terminal of the 101st aspect and a data processing device, wherein the data processing device includes an input unit that accepts the output transmitted by the communication unit, a processing unit that inputs a prompt including information corresponding to the accepted output into a data generation model and obtains a response for executing the health problem suppression process using the output of the data generation model, and an output unit that outputs the obtained response to the necklace-type terminal.

[0115] A 103rd aspect of the technology of the present disclosure is the data processing system of the 102nd aspect, wherein, when the output of the data generation model indicates that there is a problem with the wearer's health, the output unit outputs information indicating that there is a problem to a person related to the wearer.

[0116] A 104th aspect of the technology of the present disclosure is a data processing system of the 102nd or 1039th aspect, wherein the output unit outputs information to the necklace-type terminal indicating that there is a problem with cognitive function when the output of the data generation model is an output related to a decline in the cognitive function of the wearer.

[0117] A 105th aspect of the technology of the present disclosure is a data processing system of the 102nd or 103rd aspect, wherein the input unit further receives profile information of the wearer, and the processing unit inputs information corresponding to the received output and a prompt including the profile information to the data generation model, and obtains a response for executing the health problem suppression process using the output of the data generation model.

[0118] A 106th aspect of the technology of the present disclosure is a necklace-type terminal comprising: a data collection unit that collects biometric data of a wearer and environmental data including the temperature and humidity around the wearer; a processing unit that performs specific processing using the biometric data, the environmental data, and a specific algorithm that evaluates the wearer's comfort level in an air-conditioned environment; and a communication unit that transmits the results of the specific processing to an air conditioning device that conditions the air of an indoor space in which the wearer is present, wherein the processing unit performs the specific processing by calculating air conditioning settings that take into account the wearer's physical condition, the temperature, and the humidity, and the communication unit transmits a control signal corresponding to the air conditioning setting to the air conditioning device as a result of the specific processing.

[0119] A 107th aspect of the technology of the present disclosure is a necklace-type terminal of the 106th aspect, wherein the processing unit calculates the air conditioning settings as the specific processing by inputting a prompt sentence to a generative AI model that instructs the processing unit to calculate air conditioning settings taking into account the wearer's physical condition, the temperature, and the humidity based on the biometric data and the environmental data.

[0120] A 108th aspect of the technology of the present disclosure is a necklace-type terminal of the 106th aspect, which is equipped with an emotion engine that estimates emotions contained in speech about the air-conditioning environment uttered by the wearer, and the processing unit calculates the air-conditioning settings taking into account the emotions, the physical condition, the temperature, and the humidity.

[0121] A 109th aspect of the technology of the present disclosure is the necklace-type terminal of the 108th aspect, wherein the processing unit determines the speech content to suggest the air conditioning setting suitable for the physical condition of the wearer by inputting a prompt sentence to a generative AI model based on the biometric data and the environmental data, the prompt sentence instructing the model to suggest the air conditioning setting suitable for the physical condition of the wearer, and the communication unit generates information to play back the determined speech content.

[0122] A 110th aspect of the technology of the present disclosure is a necklace-type terminal of the 109th aspect, wherein the processing unit uses a generative AI model to adjust the speech content based on the emotions contained in the speech about the air-conditioning environment uttered by the wearer after the speech content is played back, and the communication unit generates information to play back the adjusted speech content.

[0123] A 111th aspect of the technology of the present disclosure is a data processing system comprising: an input unit that inputs biometric data of a wearer wearing a necklace-type terminal detected by a sensor included in the necklace-type terminal and environmental data including the temperature and humidity around the wearer; a processing unit that performs specific processing using the biometric data, the environmental data, and a specific algorithm that evaluates the wearer's comfort level in an air-conditioned environment; and an output unit that transmits the results of the specific processing to an air conditioning device that conditions the air of an indoor space where the wearer is present, wherein the processing unit performs the specific processing by calculating air conditioning settings that take into account the wearer's physical condition, the temperature, and the humidity, and the output unit transmits a control signal corresponding to the air conditioning setting to the air conditioning device as a result of the specific processing.

[0124] A 112th aspect related to the technology of the present disclosure is the data processing system of the 111th aspect, wherein the processing unit calculates, as the specific processing, the air conditioning setting by inputting a prompt sentence to a generative AI model, the prompt sentence instructing the generation AI model to calculate the air conditioning setting taking into account the physical condition of the wearer, the temperature, and the humidity, based on the biometric data and the environmental data.

[0125] A 113th aspect of the technology of the present disclosure is the data processing system of the 111th aspect, comprising an emotion engine that estimates emotions contained in speech about the air-conditioned environment uttered by the wearer, and the processing unit calculates the air-conditioning settings taking into account the emotions, the physical condition, the temperature, and the humidity.

[0126] A 114th aspect related to the technology of the present disclosure is the data processing system of the 113th aspect, wherein the processing unit determines the speech content to suggest the air conditioning setting suitable for the physical condition of the wearer by inputting a prompt sentence to a generative AI model based on the biometric data and the environmental data, the prompt sentence instructing the model to suggest the air conditioning setting suitable for the physical condition of the wearer, and the output unit plays back the determined speech content.

[0127] A 115th aspect of the technology of the present disclosure is the data processing system of the 114th aspect, wherein the processing unit uses a generative AI model to adjust the speech content based on the emotion contained in the speech about the air-conditioning environment uttered by the wearer after the speech content is played back, and the output unit plays back the adjusted speech content.

[0128] A 116th aspect of the technology of the present disclosure is a necklace-type terminal comprising: a data collection unit that collects biometric data of a wearer and environmental data including the temperature, light intensity, and color tone of the wearer's surroundings; a processing unit that performs specific processing using the biometric data, the environmental data, and a specific algorithm that evaluates the wearer's level of comfort with respect to the surrounding environment including the air conditioning and brightness around the wearer; and a communication unit that transmits the results of the specific processing to an air conditioning device that conditions the air in an indoor space where the wearer is present and to a lighting device that illuminates the indoor space, wherein the processing unit performs the specific processing by calculating air conditioning settings and lighting settings taking into account the wearer's physical condition, the temperature, the light intensity, and the color tone, and the communication unit transmits control signals corresponding to the air conditioning settings and the lighting settings to the air conditioning device and the lighting device as the results of the specific processing.

[0129] A 117th aspect of the technology of the present disclosure is the necklace-type terminal of the 116th aspect, wherein the processing unit calculates the air conditioning settings and the lighting settings as the specific processing by inputting a prompt sentence to a generative AI model that instructs the calculation of the air conditioning settings and the lighting settings taking into account the wearer's physical condition, the temperature, the light intensity, and the color tone based on the biometric data and the environmental data.

[0130] A 118th aspect of the technology of the present disclosure is a necklace-type terminal of the 116th aspect, which is equipped with an emotion engine that estimates emotions contained in utterances made by the wearer to the surrounding environment, and the processing unit calculates the air conditioning settings and the lighting settings taking into account the emotions, the physical condition, the temperature, the light intensity, and the color tone.

[0131] A 119th aspect of the technology of the present disclosure is the necklace-type terminal of the 118th aspect, wherein the processing unit determines the speech content that suggests the air conditioning settings and the lighting settings that are suitable for the physical condition of the wearer by inputting a prompt sentence that instructs the wearer to suggest the air conditioning settings and the lighting settings that are suitable for the physical condition of the wearer into a generation AI model based on the biometric data and the environmental data, and the communication unit generates information to play back the determined speech content.

[0132] A fifth aspect of the technology of the present disclosure is a necklace-type terminal of the 120th aspect, in which the processing unit uses a generative AI model to adjust the speech content based on the emotions contained in the speech regarding the air conditioning settings and the lighting settings uttered by the wearer after the speech content is played back, and the communication unit generates information to play back the adjusted speech content.

[0133] A 121st aspect of the technology of the present disclosure is a data processing system comprising: an input unit that inputs biometric data of a wearer wearing a necklace-type terminal detected by a sensor included in the necklace-type terminal, and environmental data including the temperature, light intensity, and color tone of the wearer's surroundings; a processing unit that performs specific processing using the biometric data, the environmental data, and a specific algorithm that evaluates the wearer's level of comfort with respect to the surrounding environment including the air conditioning and brightness around the wearer; and an output unit that transmits the results of the specific processing to an air conditioning device that conditions the air in an indoor space where the wearer is present and to a lighting device that illuminates the indoor space, wherein the processing unit performs the specific processing by calculating air conditioning settings and lighting settings taking into account the wearer's physical condition, the temperature, the light intensity, and the color tone, and the output unit transmits control signals corresponding to the air conditioning settings and the lighting settings to the air conditioning device and the lighting device as the results of the specific processing.

[0134] A 122nd aspect of the technology of the present disclosure is the data processing system of the 121st aspect, wherein the processing unit calculates the air conditioning settings and the lighting settings as the specific processing by inputting a prompt sentence to a generative AI model that instructs the model to calculate the air conditioning settings and the lighting settings taking into account the wearer's physical condition, the temperature, the light intensity, and the color tone based on the biometric data and the environmental data.

[0135] A 123rd aspect of the technology of the present disclosure is the data processing system of the 121st aspect, comprising an emotion engine that estimates emotions contained in utterances made by the wearer toward the surrounding environment, and the processing unit calculates the air conditioning settings and the lighting settings taking into consideration the emotions, the physical condition, the temperature, the light intensity, and the color tone.

[0136] A 124th aspect of the technology of the present disclosure is the data processing system of the 123rd aspect, wherein the processing unit determines the speech content to suggest the air conditioning settings and the lighting settings that are suitable for the physical condition of the wearer by inputting a prompt sentence to a generative AI model based on the biometric data and the environmental data, the prompt sentence instructing the model to suggest the air conditioning settings and the lighting settings that are suitable for the physical condition of the wearer, and the output unit generates information to play back the determined speech content.

[0137] A 125th aspect of the technology of the present disclosure is the data processing system of the 124th aspect, wherein the processing unit uses a generative AI model to adjust the speech content based on the emotions contained in the speech regarding the air conditioning settings and the lighting settings uttered by the wearer after the speech content is played back, and the output unit generates information to play back the adjusted speech content.

[0138] A data processing system according to a 126th aspect of the present disclosure is a data processing system comprising a terminal worn by a user and a data processing device communicatively connected to the terminal, wherein the data processing device comprises a processor, which acquires images of the user's meal taken by a camera provided on the terminal, analyzes the images to generate meal data indicating details of the meal, acquires biometric data of the user detected by a sensor provided on the terminal, integrates and analyzes the meal data and the biometric data, and outputs information according to the results of the analysis.

[0139] A data processing system according to a 127th aspect of the present disclosure is a data processing system according to the 126th aspect, wherein the processor acquires historical data indicating the history of the user's dietary data and biometric data, predicts the user's future health risks based on the historical data, and outputs information according to the predicted results.

[0140] A data processing system according to a 128th aspect of the present disclosure is a data processing system according to a 127th aspect, wherein the processor sends a signal to the terminal to warn the user when the health risk exceeds a predetermined standard.

[0141] A data processing system according to a 129th aspect of the present disclosure is a data processing system according to any one of aspects 126 to 128, wherein the processor sends a message to the terminal indicating a meal recommended to the user based on the results of the analysis.

[0142] A data processing system according to a 130th aspect of the present disclosure is a data processing system according to a 129th aspect, wherein the processor acquires other data indicating the dietary data and biometric data of other users, and determines the recommended diet based on the other data.

[0143] A data processing system according to a 131st aspect of the present disclosure is a data processing system according to any one of the 126th to 130th aspects, wherein the terminal is a necklace-type terminal.

[0144] A data processing method according to a 132nd aspect of the present disclosure is a data processing system including a terminal worn by a user and a data processing device communicatively connected to the terminal, the data processing device acquiring images of the user's meal taken by a camera provided on the terminal, analyzing the images to generate meal data indicating details of the meal, acquiring biometric data of the user detected by a sensor provided on the terminal, integrating and analyzing the meal data and the biometric data, and outputting information according to the results of the analysis.

[0145] The necklace-type terminal of the 133rd aspect of the technology disclosed herein is a necklace-type terminal that includes a camera that captures the work environment of the wearer, an engineer, a sensor that detects the engineer's biometric data, a microphone, a virtual display, and a processor, wherein the processor collects the outputs of the camera, the sensor, and the microphone, transmits the collected outputs of the camera, the sensor, and the microphone to a data processing device, and displays the information received from the data processing device on the virtual display.

[0146] The necklace-type terminal of the 134th aspect relating to the technology of the present disclosure is the necklace-type terminal of the first aspect, further comprising a speaker, and the processor outputs audio through the speaker in accordance with the information received from the data processing device.

[0147] A 135th aspect of the technique of the present disclosure is a data processing system including the necklace type terminal of the 133rd or 134th aspect and a data processing device, wherein the data processing device is provided with a processor, and the processor inputs a prompt including at least one of outputs from the camera, the sensor, and the microphone, task information acquired from a management tool for a project performed by the engineer, and the code acquired from a code repository that stores the code created by the engineer into a data generation model; a first process deriving a progress status of the work by the engineer based on the task information acquired from the management tool for the project performed by the engineer and transmitting the progress status to the necklace type terminal; a second process deriving error information of the code based on the code acquired from the code repository that stores the code created by the engineer; a third process deriving information required by the engineer based on the output from the necklace type terminal and transmitting the derived information required by the engineer to the necklace type terminal; A specific process including at least one of a fourth process of deriving relaxation information corresponding to the stress level of the engineer based on the output from the necklace-type terminal, and a fifth process of deriving progress information representing the progress of work of each of the multiple engineers based on task information of the multiple engineers obtained from a management tool for the project performed by the engineers is performed, and the results of the specific process are output to the necklace-type terminal.

[0148] A 136th aspect of the technology of the present disclosure is an interactive jewelry feedback system, comprising: a necklace-type device worn by a wearer and having a jewelry part, the necklace-type device including a sensor and an output unit; and a data processing device that determines feedback to the jewelry part according to data obtained by the sensor, the necklace-type device outputting information related to the feedback to the wearer via the output unit according to the feedback. The output unit may be at least one of a light emitting unit, a vibrator, or an audio output unit. The light emitting unit may be configured to change the properties of the light it emits according to data obtained by the sensor. Furthermore, the sensor may be a biosensor that measures biometric data of the wearer.

[0149] Additionally, the necklace type terminal may include a camera that captures images of the wearer's surroundings, a microphone, and a collection unit that collects outputs from the camera, the sensor, and the microphone. The necklace type terminal may further include a speaker that outputs a response corresponding to user utterances picked up by the microphone.

[0150] A 137th aspect of the data processing system relating to the technology of the present disclosure comprises a necklace-type terminal including a camera that captures images of the wearer's surroundings, a sensor that detects biometric data of the wearer, a microphone, a collection unit that collects the outputs of the camera, the sensor, and the microphone, a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the collection unit to a data processing device, and a speaker that outputs a response corresponding to user utterances picked up by the microphone; and the data processing device including an input unit that acquires the biometric data and user learning data, an analysis unit that analyzes the user state indicated by the biometric data, a processing unit that inputs a prompt including the user state and the user learning data into a data generation model and uses the output of the data generation model to acquire a learning plan corresponding to the user state and the user learning data, and an output unit that outputs suggestions based on the learning plan to the necklace-type terminal.

[0151] In a 138th aspect of the data processing system relating to the technology of the present disclosure, the analysis unit analyzes the level of concentration as the user state, the processing unit adds instructions to the prompt to suggest tasks of a learning difficulty level corresponding to the level of concentration of the user state, obtains the learning plan including the suggested tasks of the difficulty level, and the output unit outputs the suggested tasks of the difficulty level as the learning plan.

[0152] In a 139th aspect of the data processing system relating to the technology of the present disclosure, the analysis unit analyzes the degree of fatigue and stress level as the user state, the processing unit adds an instruction sentence to suggest a break to the prompt when at least one of the degree of fatigue and stress level of the user state is high, obtains the study plan including the break suggestion, and the output unit outputs the break suggestion as the study plan.

[0153] In a 140th aspect of the data processing system relating to the technology of the present disclosure, the processing unit adds an instruction sentence including the learning progress of the user learning data to the prompt, obtains the learning plan including review timing, and the output unit outputs a suggestion of review timing as the learning plan.

[0154] The data processing system of the 141st aspect related to the technology of the present disclosure comprises a necklace-type terminal including a collection unit that collects the output of a sensor that detects biometric data of animals present in the wearer's vicinity and a communication unit that transmits the biometric data of the animal collected by the collection unit to a data processing device, a processing unit that inputs a prompt to a data generation model instructing the estimation of information regarding the health condition of the animal based on the biometric data of the animal received from the necklace-type terminal, and acquires information regarding the health condition of the animal using the output of the data generation model, and an output unit that outputs information regarding the health condition of the animal.

[0155] A 142nd aspect of the technology of the present disclosure is the data processing system of the 141st aspect, wherein the necklace-type terminal further includes a camera that photographs the animal, the collection unit further collects images of the animal photographed by the camera, the communication unit further transmits the images of the animal, the data processing device inputs a prompt to the data generation model instructing the processing unit to estimate information regarding the emotional state of the animal based on the images of the animal received from the necklace-type terminal, and uses the output of the data generation model to further obtain information regarding the emotional state of the animal, and the output unit further outputs information regarding the emotional state of the animal.

[0156] A 143rd aspect of the technology of the present disclosure is the data processing system of the 142nd aspect, wherein the necklace-type terminal further collects environmental data around the animal through the collection unit and further transmits the environmental data through the communication unit, and the data processing device further obtains an evaluation result of the environmental state around the animal using the output of the data generation model by inputting a prompt to the data generation model instructing the processing unit to evaluate the environmental state around the animal based on the environmental data received from the necklace-type terminal, and further outputs the evaluation result of the environmental state through the output of the data generation model.

[0157] A 144th aspect of the technology of the present disclosure is a data processing system of the 143rd aspect, wherein the output unit of the data processing device outputs, as the evaluation result, instructions to control the environment surrounding the animal to a control device that controls the environment surrounding the animal.

[0158] A 145th aspect of the technology of the present disclosure is the data processing system of the 144th aspect, wherein the necklace-type terminal further includes a sensor for detecting biometric data of the wearer, the collection unit further collects biometric data of the wearer, the communication unit further transmits the biometric data of the wearer, and the data processing device inputs a prompt to the data generation model instructing the processing unit to estimate information regarding the health condition of the animal based on the biometric data of the animal received from the necklace-type terminal and the biometric data of the wearer, and obtains information regarding the health condition of the animal using the output of the data generation model.

[0159] A 146th aspect of the present disclosure is a necklace-type terminal configured to be worn by a mobile living organism and to collect data in an area where plants are grown, the necklace-type terminal comprising: a biological sensor device configured to detect biological data of the living organism; an environmental sensor device configured to detect environmental data outside the necklace-type terminal; an acoustic collector configured to detect acoustic vibrations including sound waves from outside the necklace-type terminal and sound waves emitted by the living organism; a data collection unit configured to collect data signals from the biological sensor device, the environmental sensor device, and the acoustic collector and generate respective output data; a first communication unit configured to receive the output data from the data collection unit and capable of communicating with a data processing device arranged outside the necklace-type terminal; and an electro-acoustic conversion device that converts electrical signals from the first communication unit into acoustic vibrations, wherein the environmental sensor device includes a light sensor device that detects light at the position of the necklace-type terminal.

[0160] A 147th aspect of the present disclosure is a data processing device that is provided in an area where plants are grown, the data processing device comprising: a second communication unit capable of communicating with the necklace-type terminal described in the first aspect; an input unit that receives the output data from the necklace-type terminal via the second communication unit; an output unit that sends a response to the necklace-type terminal that is generated based on the received output data to the second communication unit; a processor connected to the input unit and the output unit; and a memory that is configured to store program code and is connected to the processor, wherein the program code, when executed by the processor, causes the processor to input a prompt that includes text indicating the content of the utterance generated from the acoustic data of the output data of the necklace-type terminal as a user utterance into a data generation model, and obtain a response to the utterance using the output data of the data generation model.

[0161] A 148th aspect of the present disclosure is a data processing system, comprising a necklace-type terminal described in the first aspect and a data processing device described in the second aspect.

[0162] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a necklace-type terminal according to the first embodiment. FIG. 3 is a side view showing the configuration of the necklace-type terminal according to the first embodiment. FIG. 4 is a top view showing the configuration of the necklace-type terminal according to the first embodiment. FIG. 5 is a schematic diagram showing the functional configuration of a control unit of the necklace-type terminal according to the first embodiment. FIG. 6 is a schematic diagram showing the functional configuration of an identification processing unit of the data processing device according to the first embodiment. FIG. 7 is a schematic diagram showing an example of the operation flow of identification processing by the data processing device according to the first embodiment. FIG. 8 is a flowchart showing the operation when the processing unit 294 detects early symptoms of dementia from the user utterance of the user 20 using the data generation model 58 in the data processing device according to the second embodiment. FIG. 9 is a flowchart showing the operation when updating the data generation model 58 in the data processing device according to the third embodiment. 10 is a flowchart showing the operation of the processing unit 294 when detecting signs of cardiac disease from the user utterance and biometric data of the user 20 using the data generation model 58 in the fourth embodiment. FIG. 11 is a flowchart for explaining the operation when updating the data generation model 58 in the fourth embodiment. FIG. 12 is a flowchart showing the operation of the processing unit 294 when detecting signs of cardiac disease from the user utterance and biometric data of the user 20 using the data generation model 58 in the fifth embodiment. FIG. 13 is a flowchart for explaining the operation when updating the data generation model 58 in the fifth embodiment. FIG. 14 is a flowchart showing the operation of the processing unit 294 when detecting signs of cardiac disease from the user utterance and biometric data of the user 20 using the data generation model 58 in the sixth embodiment. FIG. 15 is a flowchart for explaining the operation when updating the data generation model 58 in the sixth embodiment.23 。 FIG. 24 is a diagram schematically showing the flow of various data transmitted and received between the necklace type terminal and the data processing device in the seventh embodiment. FIG. 25 is a diagram schematically showing the functional configuration of the identification processing unit of the data processing device in the seventh embodiment. FIG. 26 is a diagram schematically showing an example of the operation flow of the identification processing by the data processing device in the seventh embodiment. FIG. 27 is a diagram schematically showing an example of the operation flow of the identification processing by the data processing device according to the eighth embodiment. FIG. 28 is a diagram schematically showing the functional configuration of the identification processing unit of the data processing device according to the ninth embodiment. FIG. 29 is a subroutine flowchart showing details of step S303 of FIG. 23 in the ninth embodiment. FIG. 29 is a conceptual diagram showing an example of the configuration of a data processing system according to the tenth embodiment. FIG. 29 is a diagram schematically showing an example of a first operation flow of the identification processing by the data processing device according to the tenth embodiment. FIG. 29 is a conceptual diagram showing an example of the configuration of a data processing system according to the eleventh embodiment. FIG. 29 is a diagram schematically showing the functional configuration of the control unit of the necklace type terminal according to the eleventh embodiment. FIG. 29 is a sequence diagram showing the flow of the identification processing in the eleventh embodiment. 13 is a flowchart showing an example of the flow of processing performed in a necklace-type terminal of a data processing system according to an eleventh embodiment. 14 is a flowchart showing an example of the flow of processing performed in a data processing device of a data processing system according to an eleventh embodiment. 15 is a flowchart showing an example of the flow of processing performed in a data processing device of a data processing system according to an eleventh embodiment. 16 is a schematic diagram showing the functional configuration of a specification processing unit of a data processing device according to a twelfth embodiment. 17 is a schematic diagram showing the storage configuration of a data processing device according to an twelfth embodiment. 18 is a schematic diagram showing an example of a first operation flow of proposal processing by a data processing device according to an twelfth embodiment. 19 is a conceptual diagram showing an example of the configuration of a data processing system according to a thirteenth embodiment. 20 is a schematic diagram showing the functional configuration of a control unit of a necklace-type terminal according to an thirteenth embodiment. 21 is a sequence diagram showing an example of the flow of processing in a data processing system according to an thirteenth embodiment. 22 is a schematic diagram showing an example of the operation flow of specification processing by a data processing device according to an thirteenth embodiment.16 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourteenth embodiment. FIG. 17 is a sequence diagram showing an example of the processing flow of the data processing system according to the fourteenth embodiment. FIG. 18 is a conceptual diagram showing an example of the configuration of a data processing system according to a fifteenth embodiment. FIG. 19 is a sequence diagram showing an example of the processing flow of the data processing system according to the fifteenth embodiment. FIG. 20 is a conceptual diagram showing an example of the configuration of a data processing system according to a sixteenth embodiment. FIG. 21 is a sequence diagram showing an example of the processing flow of the data processing system according to the sixteenth embodiment. FIG. 22 is a conceptual diagram showing an example of the configuration of a data processing system according to a sixteenth embodiment. FIG. 23 is a sequence diagram showing the overall processing flow of the data processing system according to the sixteenth embodiment. FIG. 24 is a diagram showing an example of the processing flow of the necklace type terminal according to the sixteenth embodiment. FIG. 25 is a diagram showing an example of the configuration of a sensor provided in a necklace type terminal according to a seventeenth embodiment. 17 is a sequence diagram showing the operation of a data processing system according to a seventeenth embodiment. FIG. 18 is a flowchart showing the operation of a necklace type terminal according to a seventeenth embodiment. FIG. 19 is a flowchart showing the operation of a data processing device according to a seventeenth embodiment. FIG. 20 is a conceptual diagram showing an example of the configuration of a data processing system according to an eighteenth embodiment. FIG. 21 is a sequence diagram showing the flow of a first identification process by a data processing system according to an eighteenth embodiment. FIG. 22 is a sequence diagram showing the flow of a second identification process by a data processing system according to an eighteenth embodiment. FIG. 23 is a schematic diagram showing an example of the operation flow of processing by a necklace type terminal according to an eighteenth embodiment. FIG. 24 is a schematic diagram showing an example of the operation flow of first identification processing by a data processing device according to an eighteenth embodiment. FIG. 25 is a schematic diagram showing an example of the operation flow of second identification processing by a data processing device according to an eighteenth embodiment.19 is a conceptual diagram showing an example of the configuration of a data processing system according to a 19th embodiment. FIG. 19 is a schematic diagram showing the functional configuration of a control unit of a necklace type terminal according to a 19th embodiment. FIG. 20 is a sequence diagram showing the flow of a first identification process by a data processing system according to a 19th embodiment. FIG. 21 is a sequence diagram showing the flow of a second identification process by a data processing system according to a 19th embodiment. FIG. 22 is a schematic diagram showing an example of the operation flow of processing by a necklace type terminal according to a 19th embodiment. FIG. 23 is a conceptual diagram showing an example of the configuration of a data processing system according to a 20th embodiment. FIG. 24 is a conceptual diagram showing an example of the main functions of a data processing device and a necklace type terminal according to a 20th embodiment. FIG. 25 is a diagram showing the functional configuration of a control unit of a necklace type terminal according to a 20th embodiment. FIG. 26 is a diagram showing a sequence chart of identification processing by a data processing device according to a 20th embodiment. FIG. 27 is a conceptual diagram showing an example of the configuration of a data processing system according to a 21st embodiment. FIG. 28 is a conceptual diagram showing an example of the main functions of a data processing device and a necklace type terminal according to a 21st embodiment. 21. A diagram showing an outline of the functional configuration of a control unit of a necklace type terminal according to a 21st embodiment. A diagram showing a sequence chart of identification processing by a data processing device according to the 21st embodiment. A conceptual diagram showing an example of the configuration of a data processing system according to a 22nd embodiment. A conceptual diagram showing an example of main functions of a data processing device and a necklace type terminal according to the 22nd embodiment. A diagram showing an outline of the functional configuration of a control unit of a necklace type terminal according to the 22nd embodiment. A diagram showing a sequence chart of identification processing by a data processing device according to the 22nd embodiment. A conceptual diagram showing an example of the configuration of a data processing system according to a 23rd embodiment. A conceptual diagram showing an example of main functions of a data processing device and a necklace type terminal according to the 23rd embodiment. A diagram showing an outline of the functional configuration of a control unit of a necklace type terminal according to the 23rd embodiment. A diagram showing a sequence chart of identification processing by a data processing device according to the 23rd embodiment.27. A conceptual diagram showing an example of the configuration of a data processing system according to a twenty-fourth embodiment. A conceptual diagram showing an example of the operational flow of a specific process by a data processing device according to a twenty-fourth embodiment. A schematic diagram showing an example of the operational flow of a specific process by a data processing device according to a twenty-fifth embodiment. A conceptual diagram showing an example of the configuration of a data processing system according to a twenty-sixth embodiment. A flowchart explaining the operation of a data processing system according to a twenty-sixth embodiment. A sequence diagram explaining the operation of a data processing system according to a twenty-sixth embodiment. A schematic diagram showing the functional configuration of a specific processing unit of a data processing device according to a twenty-seventh embodiment. A schematic diagram showing the storage configuration of a data processing device according to a twenty-seventh embodiment. A schematic diagram showing an example of the operational flow of a study plan proposal process by a data processing device according to a twenty-seventh embodiment. A conceptual diagram showing an example of the configuration of a data processing system according to a twenty-eighth embodiment. A schematic diagram showing an example of the operational flow of a specific process by a data processing device according to a twenty-eighth embodiment. A schematic diagram showing an exemplary data processing system according to a twenty-ninth embodiment. A schematic diagram showing main functions of an exemplary data processing device and an exemplary necklace-type terminal according to a twenty-ninth embodiment. A schematic diagram showing an exemplary necklace-type terminal according to a twenty-ninth embodiment. 103。FIG. 103 is a diagram showing an exemplary user lifestyle model and a development model, as well as the use of these models. FIG ...29 is a diagram schematically illustrating an example of an operational flow of a specific process performed by a data processing device according to a twenty-ninth embodiment.

[0163] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technique of the present disclosure will be described with reference to the accompanying drawings.

[0164] First, the terms used in the following description will be explained.

[0165] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), or an accelerated processing unit (APU).

[0166] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by a processor.

[0167] In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0168] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0169] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0170] First Embodiment FIG. 1 shows an example of the configuration of a data processing system 10 according to a first embodiment of the present disclosure.

[0171] 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure, and the necklace-type terminal 14 is an example of a "necklace-type terminal" according to the technology of the present disclosure.

[0172] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0173] The necklace-type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 38, the speaker 40, and the camera 42 are also connected to the bus 52.

[0174] The user 20 who wears the necklace-type terminal 14 may be, for example, a patient whose health condition is to be diagnosed, or may be a normal user 20.

[0175] The microphone 38 picks up the voice uttered by the user 20 who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions and the like from the user 20 by receiving the voice uttered by the user 20. The microphone 38 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio in accordance with instructions from the processor 46. The speaker 40 is, for example, a directional speaker, and outputs audio toward the ears of the user 20.

[0176] The sensor 39 is a sensor that detects biological data of the user 20 who is wearing the necklace-type terminal. For example, the sensor 39 is a heart rate sensor, an SPO2 (arterial oxygen saturation) sensor, or a blood oxygen sensor. For example, the sensor 39 is a sensor that measures the blood pressure, blood sugar level, body temperature, glucose, respiration, etc. of the user 20 as biological data.

[0177] The camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of the user 20 (for example, an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0178] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0179] FIG. 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0180] 2 , in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The processor 28 reads the specific process program 56 from the storage 32 and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0181] The storage 32 stores a data generation model 58. The data generation model 58 is used by the specification processing unit 290. The storage 32 also includes a data accumulation unit 57.

[0182] In the necklace-type terminal 14, the data collection process is performed by the processor 46. A data collection program 60 is stored in the storage 50. The processor 46 reads the data collection program 60 from the storage 50 and executes the read data collection program 60 on the RAM 48. The data collection process is realized by the processor 46 operating as the control unit 46A in accordance with the data collection program 60 executed on the RAM 48.

[0183] As shown in Figures 3 and 4, the necklace type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. Figures 3 and 4 show an example in which two microphones 38 are arranged so as to be located in front of the user 20 when the user 20 wears the necklace type terminal 14. Figures 3 and 4 also show an example in which two sensors 39 are arranged so as to be located on the right and left sides of the user 20 when the user 20 wears the necklace type terminal 14. Figures 3 and 4 also show an example in which two speakers 40 are arranged so as to be located on the right and left rear sides of the user 20 when the user 20 wears the necklace type terminal 14. Figures 3 and 4 also show an example in which two cameras 42 are arranged so as to be located on the right and left front sides of the user 20 when the user 20 wears the necklace type terminal 14. Figures 3 and 4 also show an example in which two sensors 39 are arranged inside the necklace type terminal 14 so as to come into contact with the neck of the user 20 when the user 20 wears the necklace type terminal 14.

[0184] Next, the processing of the control unit 46A when the necklace-type terminal 14 performs a data collection process for collecting data will be described.

[0185] In the data collection process of this embodiment, biometric data of the user is collected in real time. Furthermore, not only biometric data but also all situational data surrounding the user is collected. This makes it possible to detect early signs of, for example, Alzheimer's disease and dementia. It also makes it possible to monitor the user's health condition (e.g., heart disease).

[0186] As shown in FIG. 5, the control unit 46A includes a data collection unit 100 and a communication unit 102.

[0187] The data collection unit 100 collects the output of each of the microphone 38 , the sensor 39 , and the camera 42 .

[0188] The communication unit 102 transmits the outputs of the microphone 38 , the sensor 39 , and the camera 42 collected by the data collection unit 100 to the data processing device 12 .

[0189] Next, the processing of the identification processing unit 290 when the data processing device 12 performs identification processing to acquire a response corresponding to a user utterance will be described.

[0190] In the identification process of this embodiment, a response corresponding to a user utterance picked up by the microphone 38 of the necklace-type terminal 14 is acquired using the data generation model 58 .

[0191] As shown in FIG. 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0192] The input unit 292 stores the outputs of the microphone 38 , the sensor 39 , and the camera 42 received from the necklace-type terminal 14 in the data storage unit 57 .

[0193] The input unit 292 acquires the user's utterance received by the necklace type terminal 14. Specifically, the input unit 292 acquires the user's utterance picked up by the microphone 38 of the necklace type terminal 14.

[0194] The processing unit 294 performs a specific process using the data generation model 58. Specifically, a prompt including a user utterance is input to the data generation model 58 to obtain a generation result. At this time, the prompt may further include outputs from the sensor 39 and the camera 42 collected by the data collection unit 100.

[0195] The output unit 296 transmits the result of the identification process to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A causes the speaker 40 to output the result of the identification process. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the identification process. The control unit 46A transmits audio data indicating the user utterance acquired by the microphone 38 to the data processing device 12. In the data processing device 12, the identification processing unit 290 acquires the user utterance.

[0196] The data generation model 58 is a so-called generative AI (artificial intelligence). Examples of the data generation model 58 include generative AI such as ChatGPT (registered trademark: Internet search <URL: https: / / openai.com / blog / chatgpt>) and Gemini (registered trademark: Internet search <URL: https: / / gemini.google.com / ?hl=ja>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0197] The outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 are used, for example, to diagnose the health condition of the user 20. In this case, the outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 may be transmitted to a terminal on the medical institution side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, the sensor 39, and the camera 42 stored in the data storage unit 57 to diagnose the health condition of the user 20.

[0198] Next, the operation of the data processing system 10 will be described.

[0199] First, an example of the flow of the data collection process will be described.

[0200] When the user 20 is wearing the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38, the sensor 39, and the camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, the sensor 39, and the camera 42 collected by the data collection unit 100 to the data processing device 12.

[0201] Next, an example of the flow of the identification process will be described with reference to Fig. 7. Here, it is assumed that the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace-type terminal 14 and stores them in the data accumulation unit 57.

[0202] In step S300, the processing unit 294 determines whether a predetermined trigger condition is satisfied. Specifically, the trigger condition may be that a specific word (e.g., the name of an agent installed in the necklace-type terminal 14) or phrase (e.g., "Hi! XXX" (XXX is the name of the agent)) is included in the user utterance picked up by the microphone 38.

[0203] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 ends the identification process.

[0204] In step S301, the processing unit 294 generates a prompt by adding an instruction sentence for obtaining a result of a specific process to text representing a user utterance picked up by the microphone 38.

[0205] For example, a prompt such as "The user is saying the following: XXX. Please respond as an agent" (XXX is the user utterance) can be generated. Alternatively, the outputs of the sensor 39 and the camera 42 can be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and video data representing the user's surroundings. The user is also saying the following: XXX. Please respond as an agent" (XXX is the user utterance).

[0206] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58, and obtains the result of the specific process based on the output of the data generation model 58.

[0207] In step S304, the output unit 296 outputs the result of the identification process to the necklace-type terminal 14, and the identification process ends.

[0208] Second Embodiment Next, a second embodiment will be described in which a process for detecting early symptoms of dementia in a user is executed as the identification process described in the first embodiment. Note that the second embodiment will also be described using the configuration and reference numerals of the data processing system 10 described in the first embodiment.

[0209] In this embodiment, a user 20 who may have dementia is asked to wear the necklace-type terminal 14. As described above, the necklace-type terminal 14 includes a microphone 38 that picks up user utterances from the wearer, the user 20, a data collection unit 100 that collects the output of the microphone 38, and a communication unit 102 that transmits the output of the microphone 38 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0210] The data processing device 12 in this embodiment includes an input unit 292 that accepts user utterances picked up by the microphone 38, and a processing unit 294 that, if it is determined that there are signs of dementia based on the user utterances, notifies a pre-set notification destination that the user 20 who is wearing the device may have dementia.

[0211] Here, the preset notification destination may be, for example, a medical professional such as a doctor or nurse, or may be a caregiver for user 20, a care worker at the facility where user 20 is staying, etc. The medical professional, caregiver, care worker, etc. who receives this notification can understand that user 20 has early symptoms of dementia, and can take appropriate measures or responses.

[0212] Here, if there is no consistency between multiple statements made by the wearer, user 20, while the user is speaking, the processing unit 294 determines that there are signs of dementia and notifies a pre-set notification destination that the user 20 may have dementia.

[0213] For example, if a user utterance includes a statement such as "I have never been to XX" and a statement such as "I have been to XX many times," the processing unit 294 determines that there is no consistency between the multiple utterances. Also, for example, if a user utterance includes a statement such as "I don't have any siblings" and a statement such as "I have a younger brother and sister," the processing unit 294 determines that there is no consistency between the multiple utterances.

[0214] Furthermore, when the same utterance content is repeatedly included in the user's utterance, the processing unit 294 may notify a preset notification destination that the user 20 may have dementia. For example, when the user's utterance repeatedly includes the utterance "Did I have breakfast?", the processing unit 294 may determine that the user 20 may have dementia.

[0215] Furthermore, the processing unit 294 may be configured to determine whether or not there are signs of dementia based on the frequency of appearance of specific words in the user's utterances. Pronouns such as "that," "it," and "this" may be frequently included in the utterances of a patient with dementia. Specifically, the utterances of a patient with dementia may contain an extremely high frequency of pronouns, such as "I'm going there with that, with that." Therefore, the processing unit 294 may be configured to determine whether or not the user 20 has dementia based on the frequency of appearance of pronouns in the user's utterances, using specific words as pronouns.

[0216] In addition, the processing unit 294 may not determine that the user 20 may have dementia if the user's speech contains even one statement indicating a sign of dementia, but may count the number of times that the user's speech is determined to contain signs of dementia, and may determine that the user 20 may have dementia if the count value within a predetermined period, for example, one day, exceeds a threshold value.

[0217] Furthermore, the processing unit 294 may not only detect early symptoms of dementia from user utterances in the daily life of the user 20, but also actively check whether or not the user 20 has early symptoms of dementia. When performing such a check, the processing unit 294 outputs a question sentence with a prepared content as audio from the speaker 40, determines whether the user 20 has a sign of dementia based on the answer to the question sentence picked up by the microphone 38, and if the determined possibility is equal to or greater than a predetermined value, notifies a predetermined notification destination that the user 20 may have dementia.

[0218] For example, the processing unit 294 outputs questions such as "What year, month, and day is it today?", "What day of the week is it today?", and "Where are you currently?" as audio from the speaker 40, and determines the possibility of dementia of the user 20 based on the accuracy rate of the answers to these questions.

[0219] There are various types of dementia depending on the cause. The most common type of dementia is Alzheimer's disease, followed by dementia with Lewy bodies. Since treatment methods vary depending on the type of dementia, it is possible to take appropriate measures if it is possible to not only determine whether a person has dementia, but also what type of dementia they may have early on.

[0220] As described above, the necklace-type terminal 14 includes a heart rate sensor that detects the heart rate of the user 20 as one of the sensors 39 .

[0221] Therefore, the processing unit 294 may determine the type of dementia by performing frequency analysis of heart rate fluctuations in the heart rate data of the user 20 detected by the heart rate sensor, and when notifying the user 20 that there is a possibility that the user 20 has dementia, the processing unit 294 may also notify information about the determined type of dementia.

[0222] For example, the processing unit 294 performs frequency analysis of the heart rate fluctuations in the detected heart rate data to obtain a power spectrum, calculates the high frequency component HF and the low frequency component LF of this power spectrum, and calculates LF / HF. This LF / HF value indicates the ratio of the activity levels of the sympathetic and parasympathetic nerves, which are autonomic nerves, and it is possible to determine that if this LF / HF value is equal to or greater than a reference value, there is a high possibility of Alzheimer's disease, and if this LF / HF value is less than the reference value, there is a high possibility of Lewy body dementia.

[0223] Note that the data generation model 58 is generated in advance as a learning model for dementia assessment by inputting user utterances of a user with dementia and information that the user has dementia into the data generation model 58 as training data and performing machine learning. The processing unit 294 then inputs the user utterances into the data generation model 58 and receives from the data generation model 58 a determination result as to whether or not the user 20 is likely to have dementia, thereby executing the above-described processing.

[0224] Next, the operation of the data processing system 10 according to this embodiment will be described in detail with reference to the drawings.

[0225] First, the operation of the processing unit 294 when detecting early symptoms of dementia from the user utterance of the user 20 using the data generation model 58 is shown in the flowchart of Fig. 8. Note that in the flowchart of Fig. 8, as an example, a case will be described in which a user is determined to have the possibility of dementia if there are 10 or more utterances indicating signs of dementia in the user's utterances in one day.

[0226] First, the processing unit 294 clears the count value A used when determining that the user 20 has dementia (step S401). Next, the processing unit 294 determines whether one day, which is a preset period, has passed since the start of the determination (step S402). Here, if one day has not passed since the start of the determination (no in step S402), the processing unit 294 monitors the conversation of the user 20 who is the wearer (step S403).

[0227] The processing unit 294 then inputs the monitored user utterances of the user 20 into the data generation model 58 to check whether or not there are any utterances indicating signs of dementia in the conversation of the user utterances (step S404).The processing unit 294 then determines whether or not there are any utterances indicating signs of dementia based on the determination result output from the data generation model 58 (step S405).

[0228] If it is determined in step S405 that the user's speech contains a utterance indicating a symptom of dementia (yes in step S405), the processing unit 294 adds 1 to the count value A (step S406). If it is determined in step S405 that the user's speech contains no utterance indicating a symptom of dementia (no in step S405), the processing unit 294 returns to the process of step S402.

[0229] Then, when count value A is incremented by 1 in step S406, the processing unit 294 determines whether count value A is equal to or greater than a threshold value of 10 (step S407). If it is determined in step S407 that count value A is equal to or greater than 10, the processing unit 294 notifies the pre-set notification destinations that user 20 may have dementia (step S408). If it is determined in step S407 that count value A is not equal to or greater than 10, the processing unit 294 returns to the processing of step S402.

[0230] By performing the above-described processing, if the processing unit 294 determines that the user's speech in a day contains 10 or more statements indicating signs of dementia, it will notify the pre-set notification destination that the user 20 may have dementia.

[0231] When the previously set notification destination is notified by the process described above that the user 20 may have dementia, a medical professional or the like will diagnose the user 20 to confirm whether or not the user 20 actually has dementia. However, if the determination accuracy of the data generation model 58 is not high, even if it is determined that the user 20 may have dementia, the user may not actually have dementia. Therefore, by feeding back the actual diagnosis result of the user 20 to the data generation model 58, it is expected that the data generation model 58 will be updated and the determination accuracy will be improved.

[0232] The operation for updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0233] When the control unit 294 notifies a preset notification destination that the user 20 may have dementia, the control unit 294 receives a feedback result regarding the validity of the notification (step S501). Then, the control unit 294 evaluates the determination accuracy of the data generation model 58 based on the received feedback result (step S502). Finally, the control unit 294 performs deep learning based on the evaluation result to update the data generation model 58 (step S503).

[0234] By performing the processing as described above, the data generation model 58 performs machine learning again based on the feedback results each time it is determined that the user 20 may have dementia, thereby improving the accuracy of the determination.

[0235] The processing unit 294 may determine whether or not there are signs of dementia based on a pattern of changes in the wearer's emotions estimated from the user's utterances, and if it is determined that there are signs of dementia, may notify a pre-set notification destination that the wearer may have dementia. For example, each time a collected user utterance is obtained, the processing unit 294 estimates the wearer's emotions using an emotion engine that estimates the speaker's emotions based on the collected user utterance. Then, the processing unit 294 may determine whether or not there are signs of dementia based on the time-series changes in the estimated wearer's emotions using the data generation model 58. For example, the processing unit 294 may add the estimated time-series changes in the wearer's emotions to a prompt, generating a prompt such as "These are time-series changes in the user's emotions. Please determine whether or not there are signs of dementia." The generated prompt may be input to the data generation model 58, and a determination result of whether or not there are signs of dementia may be obtained based on the output of the data generation model 58.

[0236] [Third Embodiment] Next, a third embodiment will be described in which a process for detecting signs of a heart disease in a user is executed as the identification process described in the first embodiment. Note that the configuration and symbols of the data processing system 10 in the first embodiment will also be used in the description of this embodiment.

[0237] In this embodiment, a user 20 who is a target for detecting signs of heart disease is asked to wear a necklace-type terminal 14. As described above, this necklace-type terminal 14 includes a microphone 38 that picks up user utterances from the wearer, the user 20, a sensor 39 that detects biometric data of the user 20, a data collection unit 100 that collects the outputs of the microphone 38 and the sensor 39, and a communication unit 102 that transmits the outputs of the microphone 38 and the sensor 39 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0238] Here, the sensor 39 detects either or both of heart rate data and blood pressure data of the user 20 who is the wearer as biometric data.

[0239] The data processing device 12 in this embodiment includes an input unit 292 that receives user utterances picked up by the microphone 38, and a processing unit 294 that determines whether the user 20 has signs of heart disease based on the emotional change pattern of the user 20 estimated from the user utterances and biometric data, and if it is determined that the user 20 has signs of heart disease, notifies a pre-set notification destination that the user 20, who is the wearer, has signs of heart disease.

[0240] Here, the preset notification destination may be, for example, a medical professional such as a doctor or nurse, a family member or caregiver of the user 20, or the user 20 themselves. The medical professional, family member, caregiver, etc. who receives this notification can understand that the user 20 has signs of heart disease and can take appropriate measures or responses. Note that if the user 20 themselves receives a notification that they have signs of heart disease, they can take early action, such as visiting a medical institution.

[0241] Each time collected user speech is obtained, the processing unit 294 estimates the emotions of the user 20 based on the collected user speech using an emotion engine that estimates the emotions of the user 20, and determines whether the user 20 has any signs of heart disease based on the time-series changes in the estimated emotions of the user 20 and the biometric data.

[0242] For example, the processing unit 294 adds the estimated time series changes in the emotions of the user 20 to a prompt, generates a prompt saying, "These are the time series changes in the user's emotions. Please determine whether there are any signs of heart disease.", inputs the generated prompt to the data generation model 58, and obtains a determination result as to whether there are any signs of heart disease based on the output of the data generation model 58.

[0243] Here, changes in emotions may be linked to symptoms of heart disease. Therefore, by detecting changes in emotions, it is possible to determine whether or not the user 20 has signs of heart disease. Furthermore, by using not only changes in the emotions of the user 20 but also the changes in the emotions of the user 20 and the accompanying physical reactions such as changes in heart rate and blood pressure, as in this embodiment, it is possible to detect signs of heart disease more quickly and accurately.

[0244] Note that cardiac diseases that can be detected by the data processing system 10 of this embodiment include, for example, arrhythmias such as atrial fibrillation, heart failure, and ischemic heart diseases such as myocardial infarction and angina pectoris.

[0245] Next, the operation of the data processing system 10 according to this embodiment will be described in detail with reference to the drawings.

[0246] First, the operation of the processing unit 294 when detecting signs of heart disease from the user utterance and biometric data of the user 20 using the data generation model 58 is shown in the flowchart of FIG.

[0247] First, the processing unit 294 monitors the conversation of the user 20 who is the wearer of the necklace-type terminal 14 (step S401-3).

[0248] Then, the processing unit 294 uses the emotion engine to estimate a pattern of changes in the emotion of the user 20 who is the wearer, from the user utterances of the monitored user 20 (step S402-3).

[0249] Then, the processing unit 294 inputs the estimated emotional change pattern of the user 20 and biometric data such as heart rate data and blood pressure data into the data generation model 58 to determine whether the user 20 has any signs of heart disease (step S404-3).

[0250] If it is determined in step S404-3 that the user 20 has a symptom of heart disease (yes in step S404-3), the processing unit 294 notifies a preset notification destination that the user 20 has a symptom of heart disease (step S405-3). On the other hand, if it is determined in step S404-3 that the user 20 does not have a symptom of heart disease (no in step S404-3), the processing unit 294 returns to the processing of step S401-3.

[0251] As a result of the above processing, when the processing unit 294 determines that the user 20 who is wearing the necklace-type terminal 14 has signs of heart disease, it will notify the pre-set notification destination that the user 20 has signs of heart disease.

[0252] Note that when the previously set notification destination is notified by the above-described process that the user 20 may have signs of heart disease, a medical professional or the like will diagnose the user 20 to determine whether or not the user 20 actually has heart disease. However, if the determination accuracy of the data generation model 58 is not high, even if it is determined that the user 20 has signs of heart disease, the user may not actually have heart disease. Therefore, by feeding back the actual diagnosis results of the user 20 to the data generation model 58, the data generation model 58 can be updated, and the determination accuracy is expected to improve.

[0253] The operation for updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0254] When the processing unit 294 notifies a preset notification destination that the user 20 has symptoms of heart disease, the processing unit 294 receives feedback results regarding the effectiveness of the notification (step S501-3). Then, the processing unit 294 evaluates the determination accuracy of the data generation model 58 based on the received feedback results (step S502-3). Finally, the processing unit 294 performs deep learning based on the evaluation results to update the data generation model 58 (step S503-3).

[0255] By performing the processing described above, the data generation model 58 performs machine learning again based on the feedback results each time it is determined that the user 20 has signs of heart disease, thereby improving the accuracy of the determination.

[0256] This embodiment focuses on the possibility that changes in a user's emotions may be linked to symptoms of heart disease, and by detecting these changes in emotions, it is possible to monitor the patient's condition. In particular, changes in a user's emotions and the accompanying physical reactions, such as heart rate and blood pressure, can be important signals for the early detection of heart disease. This embodiment allows the user 20 to monitor their own heart disease-related condition in real time at home, enabling them to respond immediately to any unforeseen circumstances. This allows medical professionals and others to be notified of the condition immediately.

[0257] [Fourth Embodiment] Next, a fourth embodiment will be described. The fourth embodiment relates to a case in which, as a specific process, a process is executed in a medical setting, for example, by recording a conversation between a medical professional and a patient during a medical interview in real time, and acquiring medically important information for appropriate diagnosis and treatment from the patient's biometric data during the medical interview and audio data including the content of the recorded conversation. Here, a medical professional refers to a person engaged in medical care, such as a doctor or nurse. Furthermore, "medically important information" refers to information indicating symptoms, such as a physical condition or change, that a patient consciously or unconsciously feels, and is information linked to a specific disease. Note that this embodiment will also be described using the configuration and symbols of the data processing system 10 in the first embodiment described above.

[0258] In this embodiment, the user 20, who is a patient, wears the necklace-type terminal 14. As described above, the necklace-type terminal 14 includes a sensor 39 that detects biometric data of the user 20, who is a patient and wears the necklace-type terminal 14, in real time, a microphone 38 that picks up the conversation between the user 20 and a medical professional and converts it into audio data, a data collection unit 100 that collects the outputs of the sensor 39 and the microphone 38, and a communication unit 102 that transmits the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0259] The data processing device 12 in this embodiment includes an input unit 292 that accepts biometric data collected by the sensor 39 and voice data picked up by the microphone 38, a processing unit 294 that acquires medically important information from the voice data during the conversation based on the amplitude of emotions estimated from the biometric data of the user 20, and an output unit 296 that outputs the acquired medically important information to a pre-set output destination.

[0260] The sensor 39 measures biometric data, such as the heart rate, pulse rate, respiratory rate, blood pressure, and sweating, of the user 20 wearing the necklace-type device 14 in real time to estimate the emotional amplitude of the user 20, which is the range of emotional changes. The emotional amplitude may be an absolute emotional value representing the intensity of an emotion, an index value representing one of joy, anger, sadness, or pleasure, or the amount of change in an emotional value over a predetermined period of time. The emotional amplitude serves as an index for estimating the user 20's feelings, such as tension, relaxation, agitation, fatigue, excitement, relief, anxiety, anger, or sadness. In other words, when a patient and a healthcare professional communicate, not only words but also the information conveyed by the patient's emotions is important. Furthermore, emotional changes may reveal the patient's pain or stress and provide important health information.

[0261] The preset output destination may be, for example, a terminal of a medical professional or a terminal of a medical institution. The medical professional can access his / her own terminal or the terminal of the medical institution to check medically important information during the interview with the patient and select an appropriate diagnosis and treatment method.

[0262] The processing unit 294 inputs the amplitude of emotion estimated from the biometric data of the user 20 and a prompt including voice data during a conversation with a healthcare professional into the data generation model 58, and obtains a generation result. That is, the prompt includes the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100.

[0263] Furthermore, when the amplitude of the emotion of the user 20 satisfies a predetermined trigger condition, the processing unit 294 inputs a prompt including the amplitude of the emotion of the user 20 to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. Furthermore, when the voice data during the dialogue with the medical professional satisfies a predetermined trigger condition, the processing unit 294 inputs a prompt including the voice data during the dialogue with the medical professional to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. Note that the trigger condition may not be required.

[0264] The processing unit 294 also detects the content of the voice data at this time from the amplitude of emotion estimated from the biometric data of the user 20, and assigns a label indicating the type of emotion to the detected voice data.The processing unit 294 then acquires medically important information from the voice data during the conversation with the medical professional based on the assigned label.The processing unit 294 also determines the medical importance of the medically important information based on the assigned label.

[0265] The processing unit 294 then inputs the emotional amplitude of the user 20 (the patient) and a prompt including voice data during the conversation with the healthcare professional to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. The data generation model 58 infers the input emotional amplitude of the user 20 and the voice data during the conversation with the healthcare professional in accordance with the instructions indicated by the prompt, and outputs the inference result, medically important information during the conversation, in the form of voice data and / or text data, etc.

[0266] The processing unit 294 may also determine whether the voice data contains medically important information based on a pattern of changes in the user 20's emotions during a conversation with a healthcare professional, and if the voice data contains medically important information, output a message indicating that the voice data may contain medically important information to a preset output destination. For example, each time collected voice data is obtained, an emotion engine that estimates emotions based on the collected voice data may be used to estimate the user 20's emotions. Then, based on the estimated time-series changes in the user 20's emotions, the data generation model 58 may be used to determine whether the conversation contains medically important information. For example, the estimated time-series changes in the user 20's emotions may be added to a prompt to generate a prompt such as, "These are time-series changes in the user's emotions. Please determine whether medically important information is contained." The generated prompt may be input to the data generation model 58, and a determination result as to whether the voice data contains medically important information may be obtained based on the output of the data generation model 58. In addition, the output of each of the sensor 39 and microphone 38 may be added to the prompt to generate a prompt such as, "This is biometric data representing the user's heart rate and audio data including a conversation with a medical professional. Please summarize the contents of the interview in a way that identifies any medically important information."

[0267] The processing unit 294 may also acquire, as medically important information, utterances from the voice data during a conversation with a medical professional, such as when there is a change in the emotion of the user 20 who is a patient and wears the necklace-type terminal 14, or when the amplitude of the emotion of the user 20 is greater than a preset threshold. The processing unit 294 may also determine the medical importance of medically important information using the frequency of utterances associated with physical changes during a conversation with a medical professional. The medical importance may also be determined according to the severity of the corresponding disease.

[0268] For example, when user 20, who is a patient and wears necklace-type terminal 14, utters "I can't sleep" during a medical interview with a medical professional, if there is a change in user 20's emotions or if the amplitude of the emotions becomes larger than a preset threshold, processing unit 294 acquires user 20's utterance of "I can't sleep" at that time as medically important information.

[0269] Furthermore, the processing unit 294 may estimate changes in the emotions of the user 20 based on the speaking speed, intonation, tone of voice, etc. in the voice data collected by the microphone 38. Specifically, when the user 20 utters "My lower abdomen hurts" during a medical interview with a medical professional, if the tone of the user's voice increases, the processing unit 294 may acquire the utterance "My lower abdomen hurts" as medically important information.

[0270] Next, an example of the operation of the specific processing unit 290 in this embodiment will be described with reference to FIG.

[0271] Here, when the patient user 20 wears the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38 and the sensor 39. The communication unit 102 sequentially transmits the outputs of the microphone 38 and the sensor 39 collected by the data collection unit 100 to the data processing device 12.

[0272] In step S401-4, the processing unit 294 receives, via the input unit 292, the biometric data collected by the sensor 39 and the voice data including the content of the conversation picked up by the microphone .

[0273] In step S402-4, the processing unit 294 converts the voice data into text. The text voice data is then associated with the amplitude of the emotion of the user 20 estimated based on the biometric data and stored (recorded) in the data storage unit 57. As the amplitude of the emotion, for example, an absolute value of an emotion value indicating the strength of the emotion, an index value indicating any of "joy, anger, sadness, or pleasure," or the amount of change in the emotion value for a predetermined period of time is used.

[0274] In step S403-4, the processing unit 294 detects the content of the currently converted voice data from the amplitude of the emotion estimated from the biometric data of the user 20, and assigns a label indicating the type of emotion to the detected voice data. Specifically, for example, if the absolute value of the emotion value suddenly increases or if the amount of change in the emotion value over a certain period of time is large, the processing unit 294 assigns a label of "agitation" to the utterance that is the content of the voice data at that time. Furthermore, if the absolute value of the emotion value gradually decreases or if the amount of change in the emotion value over a certain period of time is small, the processing unit 294 assigns a label of "fatigue" to the utterance that is the content of the voice data at that time. Furthermore, the processing unit 294 assigns a label of "sadness" to the utterance that is the content of the voice data whose index value indicating "joy, anger, sadness, and happiness" is a value of "sadness." In other words, the index value estimated based on the biometric data is used as a label indicating the type of emotion.

[0275] In step S404-4, the processing unit 294 determines the medical importance of the medically important information based on the assigned labels. Specifically, for example, the processing unit 294 determines that an utterance labeled "tension" is more important than an utterance labeled "relaxed." Furthermore, the processing unit 294 determines that an utterance labeled "anxiety" is more important than an utterance labeled "relieved." In this way, the processing unit 294 acquires medically important information from the voice data during the conversation with the medical professional based on the assigned labels.

[0276] In step S405-4, the processing unit 294 outputs the medically important information in the dialogue to a predetermined output destination in order of estimated importance using the output unit 296, and then ends the identification process. Specifically, the processing unit 294 outputs the medically important information in the interview or a summary including the medically important information to a predetermined terminal of a medical professional or a terminal at the medical site, and then ends the identification process. At this time, the summary is output so that the medically important information can be distinguished from other information.

[0277] In other words, by checking the medically important information contained in the output dialogue during the medical interview, doctors, as medical professionals, can make the most of the information obtained during the short interview with the patient and select more appropriate diagnoses and treatment methods. It also reduces the burden on patients of having to fill out and input information on the medical interview form. It also enables accurate medical interviews even when patients are unable to explain their physical condition well.

[0278] When medically important information or a summary including medically important information is output to a predetermined output destination by the processing described above, a medical professional, i.e., a doctor, will check the output content and select a diagnosis result and treatment method for the user 20. However, if the determination accuracy of the data generation model 58 is not high, even if medically important information or a summary including medically important information is output during a conversation, the part determined to be medically important information may not actually be important. Therefore, it is expected that the data generation model 58 will be updated and the determination accuracy will be improved by actually feeding back the diagnosis result of the user 20 by the medical professional to the data generation model 58.

[0279] The operation for updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0280] When the processing unit 294 outputs medically important information or a summary including medically important information to a predetermined output destination during a medical interview with the user 20, the processing unit 294 receives a feedback result regarding the validity of the medically important information (step S501-4). Then, the processing unit 294 evaluates the determination accuracy of the data generation model 58 based on the received feedback result (step S502-4). Finally, the processing unit 294 performs deep learning based on the evaluation result to update the data generation model 58 (step S503-4).

[0281] By performing the processing as described above, the data generation model 58 performs machine learning again based on the feedback results each time it determines that the information is medically important during the medical interview with the user 20, thereby improving the accuracy of the determination.

[0282] Fifth Embodiment Next, a fifth embodiment will be described. In the fifth embodiment, as the specific processing described in the first embodiment, a conversation between a medical professional and a patient during a medical interview is recorded in real time in a medical setting. Then, from the biometric data of the medical professional during the medical interview and audio data including the content of the recorded conversation, medically important information for appropriate diagnosis and treatment is obtained, and a summary of the medical interview is created. Here, "medical professional" refers to a person engaged in medical care, such as a doctor or nurse. Furthermore, "medically important information" refers to information indicating symptoms, such as a physical condition or change, that a patient consciously or unconsciously feels, and is information linked to a specific disease. Note that this embodiment will also be described using the configuration and symbols of the data processing system 10 described in the first embodiment.

[0283] In this embodiment, the user 20, who is a medical professional, wears the necklace-type terminal 14. As described above, the necklace-type terminal 14 includes a sensor 39 that detects biometric data of the user 20, who is a medical professional and wears the necklace-type terminal 14, in real time, a microphone 38 that picks up the conversation between the user 20 and a patient and converts it into audio data, a data collection unit 100 that collects the outputs of the sensor 39 and the microphone 38, and a communication unit 102 that transmits the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0284] The data processing device 12 in this embodiment includes an input unit 292 that receives biometric data collected by the sensor 39 and voice data picked up by the microphone 38, a processing unit 294 that acquires medically important information from the voice data during the conversation based on the pattern of emotional changes estimated from the biometric data of the user 20, creates a summary that summarizes the content of the conversation so that the medically important information is included preferentially, and stores (records) the summary in the data accumulation unit 57 that serves as a recording unit, and an output unit 296 that outputs the summary including the medically important information to a predetermined output destination.

[0285] Here, the "preset output destination" may be, for example, a terminal of a medical professional or a terminal of a medical institution. The medical professional can access their own terminal or the terminal of the medical institution to check the summary including medically important information.

[0286] The sensor 39 estimates the emotional change pattern of the user 20 by measuring in real time biological data such as the heart rate, pulse rate, respiratory rate, blood pressure, and perspiration of the user 20 wearing the necklace-type terminal 14. Here, the emotional change pattern may be an absolute emotional value representing the intensity of an emotion, an index value representing any of joy, anger, sadness, and pleasure, or the amount of change in an emotional value within a predetermined period of time. The emotional change pattern serves as an index for estimating the emotion of the user 20, such as whether the user 20 is tense, relaxed, agitated, tired, excited, relieved, anxious, angry, sad, etc.

[0287] Here, when conversing with a patient, not only the medical professional's tone of voice but also the information conveyed by the medical professional's emotions is important. For this reason, the processing unit 294 detects patterns of emotional changes in the user 20, who is a medical professional, using voice data collected by the microphone 38 and biometric data of the user 20 collected by the sensor 39. As a result, the processing unit 294 acquires medically important information in the conversation with the patient and creates a summary that summarizes the content of the conversation by prioritizing the acquired medically important information. At this time, the processing unit 294 captures subtle changes in the user 20's emotions using the biometric data and, based on the pattern of changes in the user 20's emotions during the conversation, evaluates what is important and what is not important in the conversation with the patient and determines the level of importance.

[0288] The processing unit 294 automatically converts the voice data collected by the microphone 38 into a text format to generate text. The processing unit 294 also recognizes the voice of the user 20 from the voice data collected by the microphone 38 and automatically converts it into text.

[0289] The processing unit 294 inputs the emotion change pattern estimated from the biometric data of the user 20 and a prompt including voice data during a conversation with the patient into the data generation model 58, and obtains a generation result. That is, the prompt includes the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100.

[0290] Furthermore, when the pattern of changes in the emotions of the user 20 satisfies a predetermined trigger condition, the processing unit 294 inputs a prompt including the pattern of changes in the emotions of the user 20 to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. Furthermore, when voice data during a dialogue with the patient satisfies a predetermined trigger condition, the processing unit 294 inputs a prompt including the voice data during a dialogue with the patient to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. Note that the trigger condition may not be required.

[0291] The processing unit 294 then inputs prompts including the emotional change pattern of the user 20, who is a medical professional, and voice data during the conversation with the patient to the data generation model 58, and acquires medically important information based on the output of the data generation model 58. The data generation model 58 infers the input emotional change pattern of the user 20 and the voice data during the conversation with the patient in accordance with the instructions given by the prompts, and outputs the inference result, medically important information during the conversation, in the form of voice data and / or text data, etc.

[0292] In addition, the processing unit 294 detects the content of the voice data at this time from the pattern of emotional changes estimated from the biometric data of the user 20, and labels the detected voice data by assigning a label indicating the type of emotion.

[0293] The processing unit 294 then determines the medical importance of the converted voice data based on the assigned label. That is, the processing unit 294 acquires medically important information from the voice data during the conversation with the patient based on the assigned label.

[0294] The processing unit 294 then creates a summary of the dialogue with the patient based on the determined medical importance. At this time, the processing unit 294 may create the summary in such a way that the medically important information is distinguishable from other information. Here, "distinguishable" means that the medically important information is displayed in a manner that makes it distinguishable from other information, for example, by underlining the medically important information, adding a color to it, or making it bolder than other information.

[0295] Furthermore, the processing unit 294 may estimate a change in the emotion of the user 20 based on the speaking speed, intonation, tone, and other aspects of the user's speaking style in the audio data collected by the microphone 38. Specifically, for example, during a medical interview with a patient, if the user 20, who is a medical professional and wears the necklace-type terminal 14, utters "Do you have pain in your lower abdomen?" and the user 20 speaks with a stronger tone, the processing unit 294 acquires the patient's response to the user's utterance of "Do you have pain in your lower abdomen?", that is, "I don't have pain in my lower abdomen," as medically important information.

[0296] That is, the processing unit 294 may determine the medical importance of the dialogue with the patient based on the change pattern of the user 20's tone of voice and emotions during the dialogue with the patient, and create a summary that prioritizes information of high medical importance and medical significance. For example, each time collected voice data is obtained, an emotion engine that estimates emotions based on the collected voice data may be used to estimate the emotion of the user 20. Then, the data generation model 58 may be used to determine the medical importance of the voice data based on the time-series changes in the estimated emotion of the user 20. For example, the outputs of the sensor 39 and the microphone 38 may be added to a prompt to generate a prompt such as, "This is voice data including biometric data representing the user's heart rate and the dialogue with the patient. Please summarize the contents of the medical interview in a way that identifies medically significant information."

[0297] Furthermore, the processing unit 294 may acquire, from the voice data during a conversation with a patient, utterances made when there is a change in the emotions of the user 20, who is a medical professional and wears the necklace-type terminal 14, as medically important information. The processing unit 294 may also determine the medical importance of the medically important information using the frequency of utterances associated with physical changes during a conversation with a patient. The medical importance may also be determined according to the severity of the patient's illness during the conversation. Furthermore, when creating a summary, the patient may wear the necklace-type terminal 14, and biometric data of the patient who is wearing the necklace-type terminal 14 may be detected in real time to acquire, as medically important information, utterances made when there is a change in the patient's emotions.

[0298] Next, an example of the operation of the specific processing unit 290 in this embodiment will be described with reference to FIG.

[0299] Here, when the user 20, who is a medical professional, wears the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38 and the sensor 39. The communication unit 102 sequentially transmits the outputs of the microphone 38 and the sensor 39 collected by the data collection unit 100 to the data processing device 12.

[0300] In step S401-5, the processing unit 294 receives, via the input unit 292, the biometric data collected by the sensor 39 and the voice data including the content of the conversation picked up by the microphone .

[0301] In step S402-5, the processing unit 294 automatically converts the voice data into text format.

[0302] In step S403-5, the processing unit 294 determines the medical importance of the voice data that has been converted into text. Specifically, the processing unit 294 detects the content of the voice data that has been converted into text at this time from the pattern of changes in emotion estimated from the biometric data of the user 20, and assigns a label indicating the type of emotion to the detected voice data. Then, the processing unit 294 determines the medical importance of the voice data based on the assigned label.

[0303] The emotion change pattern may be, for example, an absolute value of an emotion value that indicates the intensity of an emotion, an index value that indicates any of "joy, anger, sadness, or pleasure," or the amount of change in the emotion value for each preset period. Furthermore, the processing unit 294 may detect the content of the voice data converted to text at that time from the tone of voice of the user 20 estimated from the voice data of the user 20, and may label the detected voice data by assigning a label indicating the type of emotion.

[0304] Specifically, for example, if the absolute value of the emotion value suddenly increases, if the amount of change in the emotion value over a certain period of time is large, or if the user 20 speaks forcefully, the processing unit 294 labels the utterance, which is the content of the voice data at that time, as "tension." Furthermore, if the absolute value of the emotion value gradually decreases, if the amount of change in the emotion value over a certain period of time is small, or if the user 20 speaks calmly, the processing unit 294 labels the utterance, which is the content of the voice data at that time, as "relief." Furthermore, the processing unit 294 labels the utterance, which is the content of the voice data, as "sad" if the index value indicating "joy, anger, sadness, and happiness" is a value of "sad." At this time, an index value estimated based on biometric data is used as the label indicating the type of emotion.

[0305] Then, the processing unit 294 determines, for example, that an utterance labeled with "tension" is more important than an utterance labeled with "relaxation." Also, the processing unit 294 determines that an utterance labeled with "anxiety" is more important than an utterance labeled with "relief."

[0306] In step S404-5, the processing unit 294 creates a summary including medically important information based on the medical importance of the converted audio data, and stores (records) the summary in the data storage unit 57. For example, the processing unit 294 creates a summary including utterances labeled with "tension," "anxiety," etc., which have been determined to have a high level of medical importance, and stores the summary in the data storage unit 57.

[0307] Specifically, for example, when user 20, who is a medical professional and wears necklace-type terminal 14, utters "Are you getting enough sleep?" during a medical interview with a patient, if there is a large change in the emotional value of user 20, processing unit 294 labels the patient's utterance "I'm not getting enough sleep" in response to user 20's utterance "Are you getting enough sleep?" as "tension" and creates a summary sentence including the information "I'm not getting enough sleep" as being of high medical importance.

[0308] Furthermore, for example, when the patient utters "I have no appetite" during a medical interview, and the user 20, who is a medical professional and wears the necklace-type terminal 14, utters "It's okay even if you have no appetite as long as you stay hydrated," if there is a small change in the emotional value of the user 20, the processing unit 294 will label the patient's utterance "I have no appetite" as "reassuring" and create a summary sentence as being of low medical importance.

[0309] In step S405-5, the processing unit 294 outputs the created summary to a predetermined output destination via the output unit 296, and the identification process ends. Specifically, the processing unit 294 outputs the summary including the medically important information to a predetermined terminal of a medical professional or a terminal of a medical institution, and the identification process ends. At this time, the summary may be output so that the medically important information can be distinguished from other information.

[0310] In other words, it reduces the time medical professionals spend on preparing documents such as medical certificates and entering data such as electronic medical records. As a result, medical professionals can spend more time caring for patients, such as through medical examinations and rehabilitation, which also reduces their stress. In addition, by automatically converting information obtained from conversations with patients into text and determining its medical importance, medical professionals can reduce the time they spend laboriously entering and searching information. In addition, by creating summaries through conversations with patients, more appropriate diagnoses and treatment methods can be selected. It also reduces the burden on patients of having to fill out and enter information into medical questionnaires. Furthermore, it enables accurate medical interviews even when patients are unable to explain their physical condition well.

[0311] In addition, when a summary containing medically important information is output to a pre-set output destination by the process described above, a medical professional, a doctor, will check the output content and select a diagnosis result and treatment method for user 20.

[0312] However, if the determination accuracy of the data generation model 58 is not high, even if a summary including medically important information is output, the portion determined to be medically important may not actually be important. Therefore, it is expected that the determination accuracy will improve if a medical professional actually provides feedback to the data generation model 58 regarding the effectiveness of the summary, thereby updating the data generation model 58.

[0313] The operation for updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0314] When the processing unit 294 outputs a summary containing medically important information to a predetermined output destination through a dialogue with a patient, the processing unit 294 receives feedback regarding the effectiveness of the summary containing the medically important information (step S501-5).The processing unit 294 then evaluates the determination accuracy of the data generation model 58 based on the received feedback (step S502-5).Finally, the processing unit 294 performs deep learning based on the evaluation result to update the data generation model 58 (step S503-5).

[0315] By performing the processing described above, the data generation model 58 undergoes machine learning again based on the feedback results each time it outputs a summary sentence containing medically important information through dialogue with a patient, thereby improving the accuracy of judgment.

[0316] [Sixth Embodiment] Next, a sixth embodiment will be described. In the sixth embodiment, as the specific processing described in the first embodiment, for example, a conversation between a medical professional and a patient during a medical interview is recorded in real time in a medical setting. Then, in order to efficiently determine an appropriate diagnosis and treatment method from the biometric data of the patient during the medical interview and audio data including the recorded conversation, medically important information is acquired and inferred, and the inference result is acquired as diagnostic support information. Here, a medical professional refers to a person engaged in medical care, such as a doctor or a nurse. Note that in this embodiment, "inference" also refers to, for example, analysis, classification, prediction, and / or summarization.

[0317] "Diagnostic support information" is information that can assist medical professionals in making diagnoses and determining treatment methods. It refers to medically important information obtained during interviews between medical professionals and patients, organized and evaluated to make it easier for doctors to understand. Specifically, for example, it is information such as one or more types of diseases that the patient may be suffering from, evaluation results for each disease, one or more treatment methods for each disease, and evaluation results for each treatment method. Here, "evaluation results" include, for example, values ​​indicating the possibility of the disease and values ​​indicating the effectiveness of the treatment methods. Furthermore, "medically important information" is information indicating symptoms such as physical conditions and changes that the patient consciously or unconsciously feels, and is information linked to a specific disease. Note that this embodiment will also be described using the configuration and symbols of the data processing system 10 in the first embodiment described above.

[0318] In this embodiment, the user 20, who is a patient, wears the necklace-type terminal 14. As described above, the necklace-type terminal 14 includes a sensor 39 that detects biometric data of the user 20, who is a patient and wears the necklace-type terminal 14, in real time, a microphone 38 that picks up the conversation between the user 20 and a medical professional and converts it into audio data, a data collection unit 100 that collects the outputs of the sensor 39 and the microphone 38, and a communication unit 102 that transmits the outputs of the sensor 39 and the microphone 38 collected by the data collection unit 100 to the data processing device 12, which is an external device.

[0319] Here, the sensor 39 detects data such as the heart rate, pulse rate, respiratory rate, blood pressure, and perspiration of the user 20 wearing the necklace-type terminal 14 as biometric data.

[0320] The data processing device 12 in this embodiment includes an input unit 292 that accepts biometric data collected by the sensor 39 and audio data picked up by the microphone 38, a processing unit 294 that acquires and infers medically important information from the audio data during the conversation based on a pattern of emotional changes estimated from at least one of the biometric data and audio data of the user 20, and acquires the inference results as diagnostic support information, and an output unit 296 that outputs the acquired diagnostic support information to a pre-set output destination.

[0321] Here, the preset output destination may be, for example, a medical professional's terminal or a terminal of a medical institution. By accessing their own terminal or a terminal of a medical institution, medical professionals can check the diagnostic support information, which is the inference result of inferring medically important information. This allows medical professionals to efficiently select appropriate diagnosis and treatment methods by checking the diagnostic support information.

[0322] In order for healthcare professionals to select appropriate diagnoses and treatment methods, not only words but also the information conveyed by the patient's emotions during dialogue between the patient and healthcare professional are important. Therefore, the pattern of changes in a patient's emotions and the content of the dialogue are medically important information, and analyzing this information can enable more appropriate diagnoses and treatments for individual patients. In other words, the information healthcare professionals need may be obtained through fluctuations in a patient's emotions, and analyzing these fluctuations may provide important insights for healthcare professionals to determine appropriate diagnoses and treatment methods.

[0323] Therefore, the processing unit 294 estimates the emotional change pattern of the user 20 based on at least one of the biometric data of the user 20 wearing the necklace-type terminal 14 output from the sensor 39 and the audio data captured from the conversation between the user 20 and a medical professional. Here, the emotional change pattern may be a value indicating a change in the absolute value of an emotional value that represents the intensity of an emotion, a value indicating a change in an index value that represents any of joy, anger, sadness, and pleasure, or a value indicating a trajectory of the amount of change in an emotional value that changes within a predetermined period of time. The emotional change pattern serves as an index for estimating changes in the emotion of the user 20, such as tension, relaxation, agitation, fatigue, excitement, relief, anxiety, anger, sadness, etc.

[0324] As an example, the processing unit 294 estimates the emotion of the user 20 based on at least one of the collected dialogue and biometric data using an emotion engine that estimates the emotion of the user 20, and estimates a time-series change in the estimated emotion of the user 20 as an emotion change pattern. The processing unit 294 can capture subtle changes in emotion from "human speech and behavior," "changes in pulse rate," "sweating patterns," etc.

[0325] The processing unit 294 inputs the estimated emotion change pattern and a prompt including voice data during the conversation between the user 20 and the healthcare professional into the data generation model 58, and acquires diagnostic assistance information based on the output of the data generation model 58. The data generation model 58 infers the input emotion change pattern of the user 20 and the voice data during the conversation between the user 20 and the healthcare professional in accordance with the instructions given by the prompt, and outputs the diagnostic assistance information that is the inference result in a data format such as voice data and / or text data.

[0326] For example, the processing unit 294 may add the estimated emotional change pattern of the user 20 and the collected voice data to a prompt, generate a prompt saying, "This is voice data including the user's emotional change pattern and a conversation with a medical professional. Please extract medically important information, organize it so that it is easy to understand, and evaluate it," input the generated prompt to the data generation model 58, and acquire, as diagnostic assistance information, an evaluation result in which the medically important information has been organized and evaluated based on the output of the data generation model 58. Alternatively, the processing unit 294 may add the outputs of the sensor 39 and the microphone 38 to the prompt, and generate a prompt saying, "This is voice data including the user's biometric data and a conversation with a medical professional. Please extract medically important information, organize it so that it is easy to understand, and evaluate it."

[0327] The processing unit 294 may acquire, in the voice data during a conversation with a medical professional, an utterance such as when there is a change in the emotion of the user 20 who is a patient and wears the necklace-type terminal 14, or when the pattern of change in the emotion of the user 20 is greater than a preset threshold, as medically important information. Specifically, for example, when the user 20 who is a patient and wears the necklace-type terminal 14 utters "I can't sleep" during a medical interview with a medical professional, if there is a change in the emotion of the user 20 or if the pattern of change in emotion becomes greater than a preset threshold, the processing unit 294 acquires the utterance of the user 20 at that time, "I can't sleep," as medically important information.

[0328] Furthermore, the processing unit 294 may estimate changes in the emotions of the user 20 based on the speaking speed, intonation, tone of voice, etc. in the voice data collected by the microphone 38. Specifically, when the user 20 utters "My lower abdomen hurts" during a medical interview with a medical professional, if the tone of the user's voice increases, the processing unit 294 may acquire the utterance "My lower abdomen hurts" as medically important information.

[0329] In this embodiment, when the processing unit 294 acquires, as medically important information, a plurality of symptoms including, for example, "My lower abdomen hurts," it acquires, as diagnostic support information, one or more types of diseases that may cause the plurality of symptoms including, for example, "My lower abdomen hurts." Specifically, as an example, it acquires, as diagnostic support information, an inference result such as, "Because I have pain in the right lower abdomen, I may have appendicitis, pancreatitis, etc." At this time, the processing unit 294 may acquire, for each disease, an evaluation result that evaluates the possibility of the disease. For example, the evaluation result can be displayed using labels such as 00% or large, medium, small, etc.

[0330] Furthermore, when a medical professional determines a diagnosis for a patient who is the user 20 from the output diagnostic support information, the processing unit 294 may generate a prompt such as "Please suggest some optimal treatment methods for △△ (△△ is the name of the disease)," input the generated prompt to the data generation model 58, and acquire multiple treatment methods as diagnostic support information based on the output of the data generation model 58. In this case, the processing unit 294 acquires multiple treatment methods via the data generation model 58 by referring to the patient's past medical records and the latest research results, etc., which are stored in advance in the data accumulation unit 57, for example.

[0331] Next, an example of the operation of the specific processing unit 290 in this embodiment will be described with reference to FIG.

[0332] Here, when the patient user 20 wears the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38 and the sensor 39. The communication unit 102 sequentially transmits the outputs of the microphone 38 and the sensor 39 collected by the data collection unit 100 to the data processing device 12.

[0333] In step S401-6, the processing unit 294 receives, via the input unit 292, the biometric data collected by the sensor 39 and the voice data including the content of the conversation picked up by the microphone .

[0334] In step S402-6, the processing unit 294 estimates a pattern of change in the emotion of the user 20 based on at least one of the collected dialogue and the biometric data using the emotion engine.

[0335] In step S403-6, the processing unit 294 extracts medically important information from the emotional change pattern estimated from the biometric data of the user 20 and the voice data during the conversation between the user 20 and the medical staff.

[0336] In step S404-6, the processing unit 294 analyzes the extracted medically important information, organizes it into an easily understandable format for medical professionals, and evaluates it. Specifically, the processing unit 294 infers the specific type of disease, the evaluation result of the possibility of the disease, and the like.

[0337] In step S405-6, the processing unit 294 determines the inference result as the specific type of disease, the evaluation result assessing the possibility of the disease, etc., and outputs the inference result as one or more diagnostic support information to a pre-set output destination.

[0338] In step S406-6, the processing unit 294 accepts input from the medical professional via the input unit 292. As an example, the input unit 292 accepts input via a keyboard (not shown) or a mouse (not shown). Note that if the medical professional is wearing the necklace-type terminal 14, speech picked up by the microphone 38 may be accepted. Also, if the medical professional is not wearing the necklace-type terminal 14, speech picked up by an external microphone (not shown) or the like may be accepted.

[0339] In step S407-6, the processing unit 294 determines whether the medical professional has decided on a diagnosis. Specifically, the processing unit 294 determines that the medical professional has decided on a diagnosis when, for example, the medical professional inputs a disease name via the input unit 292.

[0340] If a diagnosis is determined in step S407-6, in step S408-6, the processing unit 294 acquires multiple treatment methods for the disease diagnosed by the medical professional and outputs them to a pre-set output destination, thereby completing the identification process.

[0341] In this way, by checking the output diagnostic support information, medical professionals can save the trouble of processing the large amount of information obtained during patient interviews, allowing them to more efficiently select appropriate diagnostic and treatment methods.

[0342] When the diagnostic assistance information is output to a predetermined output destination by the process described above, a medical professional will check the output content and select a diagnosis result and treatment method for the user 20. However, if the determination accuracy of the data generation model 58 is not high, even if the type of disease, treatment method, etc. are output as diagnostic assistance information, the manually output type of disease and treatment method may not actually be optimal. Therefore, it is expected that the medical professional will actually update the data generation model 58 by feeding back the diagnosis result of the user 20 and the selected treatment method to the data generation model 58, thereby improving the determination accuracy.

[0343] The operation for updating the data generation model 58 will be described with reference to the flowchart of FIG.

[0344] When the processing unit 294 outputs diagnostic assistance information to a predetermined output destination, it receives feedback results regarding the effectiveness of the diagnostic assistance information (step S501-6). Then, the processing unit 294 evaluates the determination accuracy of the data generation model 58 based on the received feedback results (step S502-6). If the determination accuracy is equal to or greater than a predetermined threshold (step S503; YES), the processing unit 294 terminates the process. On the other hand, if the determination accuracy is less than the predetermined threshold (step S503-6; NO), the processing unit 294 performs deep learning based on the evaluation results to update the data generation model 58 (step S504-6).

[0345] By carrying out the processing as described above, the data generation model 58 undergoes machine learning again based on the feedback results every time diagnostic assistance information is output, thereby improving the accuracy of judgment.

[0346] [Seventh embodiment] A seventh embodiment will be described. In sales positions, not only customer service but also stress management for sales representatives is important. A prolonged high stress state can lead to a decline in performance and have a negative impact on the success rate of business negotiations. It is desirable to simultaneously handle customer service and stress management for sales representatives. A data processing system according to a seventh embodiment of the disclosed technology has a function of managing stress for sales representatives while supporting sales activities using a necklace-type terminal. Details of the data processing system according to the seventh embodiment will be described below.

[0347] 18 is a diagram schematically showing the flow of various data transmitted and received between the necklace-type terminal 14 and the data processing device 12. In this embodiment, the necklace-type terminal 14 is worn by a sales representative 401 who conducts sales activities with a customer 400. The necklace-type terminal 14 is an example of a "wearable terminal device" in the disclosed technology.

[0348] The camera 42 included in the necklace-type terminal 14 photographs the customer 400 and generates image data including the facial expressions of the customer 400. The microphone 38 included in the necklace-type terminal 14 picks up the speech of the customer 400 and generates voice data. The sensor 39 included in the necklace-type terminal 14 generates biometric data including at least one of the heart rate, blood pressure, skin temperature, and skin potential of the sales representative 401.

[0349] The image data, audio data, and biometric data are collected by the data collection unit 100 of the necklace-type terminal 14. The collected image data, audio data, and biometric data are transmitted to the data processing device 12 by the communication unit 102 of the necklace-type terminal 14.

[0350] The processor 28 of the data processing device 12 operates as a specific processing unit 290A by executing the specific processing program 56 on the RAM 30. As shown in Fig. 19 , the specific processing unit 290A includes an acquisition unit 501, an emotion recognition unit 502, a stress level derivation unit 503, a feedback generation unit 504, a report generation unit 505, and a communication unit 506.

[0351] The acquisition unit 501 acquires the image data, voice data, and biometric data transmitted from the necklace-type terminal 14 and stores this data in the data storage unit 59 .

[0352] The emotion recognition unit 502 recognizes the emotional state of the customer 400 based on voice data including the speech of the customer 400 and image data including the facial expression of the customer 400. The emotion recognition unit 502 recognizes the emotional state of the customer 400 using, for example, an acoustic analysis model and an image analysis model trained by machine learning.

[0353] The acoustic analysis model is a model that performs acoustic analysis on voice data including the speech of the customer 400, analyzing the volume, intonation, and the like of the voice, and captures changes in voice features and frequency to determine the emotional state of the customer 400. The image analysis model is a model that determines the emotional state of the customer 400 from image data including the facial expressions of the customer 400, such as movements of mimetic muscles, movements of the eyes, and pupil dilation.

[0354] The emotion recognition unit 502 integrates the analysis results of the acoustic analysis model and the image analysis model to output a recognition result of the emotional state of the customer 400. The emotion recognition unit 502 may output, as the recognition result of the emotional state of the customer 400, data that quantifies the intensity of each emotional element, including, for example, "interest," "dissatisfaction," "surprise," and "anger."

[0355] The stress level derivation unit 503 derives the stress level of the sales representative 401 based on the biometric data acquired about the sales representative 401. The stress level is reflected in the heart rate, blood pressure, skin temperature, and skin potential. The stress level derivation unit 503 outputs data that quantifies the stress level of the sales representative 401 based on the biometric data including at least one of the heart rate, blood pressure, skin temperature, and skin potential.

[0356] The feedback generation unit 504 generates effective feedback for successfully conducting sales activities with the customer 400, based on the emotional state of the customer 400 recognized by the emotion recognition unit 502 and the stress level of the salesperson 401 derived by the stress level derivation unit 503. The feedback generation unit 504 generates feedback by inputting a prompt, including the recognized emotional state of the customer 400 and the derived stress level of the salesperson 401, into the data generation model 58, which instructs the generation of feedback effective for successfully conducting sales activities with the customer 400. For example, the prompt "The customer's emotional state is 'dissatisfied.' The salesperson's stress level is '8.' Please generate effective feedback for the salesperson to successfully conduct sales activities with this customer" is input into the data generation model 58. At this time, feedback such as "Please get support from another salesperson" is generated in real time.

[0357] The feedback generation unit 504 generates advice on how to proceed with the business negotiation as feedback by inputting a prompt to the data generation model 58 instructing the generation of advice on how to proceed with the business negotiation based on the recognition result of the emotional state of the customer 400. For example, if the recognized emotional state of the customer 400 is "dissatisfied," the feedback generation unit 504 inputs a prompt saying, "The customer's emotional state is 'dissatisfied'. Please generate advice on how to proceed with the business negotiation" to the data generation model 58. At this time, the data generation model 58 generates, as feedback, voice data including content such as, "The customer seems dissatisfied. Let's change the topic."

[0358] Furthermore, when the derived stress level of the sales representative 401 is equal to or higher than a threshold, the feedback generation unit 504 inputs a prompt to the data generation model 58 instructing the data generation model 58 to generate a coping method for reducing the stress level, thereby generating the coping method as feedback. For example, the prompt "The stress level of the sales representative is '8'. Please generate a coping method for reducing the stress level" is input to the data generation model 58. At this time, the feedback generation unit 504 generates audio data including a breathing technique guide or relaxation audio as feedback.

[0359] The feedback generation unit 504 evaluates the correlation between the recognized emotional state of the customer 400 and the derived stress level of the salesperson 401. For example, if the stress level of the salesperson 401 increases as the customer 400's negative emotions increase, the correlation between the emotional state of the customer 400 and the stress level of the salesperson 401 is evaluated as being high. If the correlation between the emotional state of the customer 400 and the stress level of the salesperson 401 is equal to or greater than a certain level, the feedback generation unit 504 inputs a prompt to the data generation model 58 instructing the data generation model 58 to generate a warning for the salesperson, thereby generating a warning for the salesperson 401 as feedback. For example, if the correlation is equal to or greater than a certain level, the feedback generation unit 504 inputs a prompt to the data generation model 58 saying, "The stress level of the salesperson is increasing as the customer's negative emotions increase. Please generate a warning for this salesperson." At this time, the data generation model 58 generates voice data as feedback including content such as, "The customer's emotions and your stress level have a mutual influence. By relaxing, you can change the customer's emotions to a positive one."

[0360] A sales negotiation record including the content of the conversation between the customer 400 and the sales representative 401 during the sales negotiation, the emotional state of the customer 400, the stress level of the sales representative 401, and the content of the feedback generated by the feedback generation unit 504 is stored in the storage 32 of the data processing device 12. After the sales negotiation is completed, the report generation unit 505 generates a report including improvements to be made and a recommended approach for the next sales negotiation based on the sales negotiation record stored in the storage 32 by inputting a prompt to the data generation model 58 instructing the generation of a report including improvements to be made and a recommended approach for the next sales negotiation. The report generation unit 505 generates the report by inputting the sales negotiation record and the prompt into the data generation model 58. The report generation unit 505 may generate the report using customer information including past sales negotiation history, customer preferences, and past purchase history that can be obtained from a database stored in the storage 32.

[0361] The communication unit 506 transmits the voice data including the feedback content generated by the feedback generation unit 504 and the voice data including the content of the report generated by the report generation unit 505 to the necklace-type terminal 14 .

[0362] FIG. 20 is a flowchart showing an example of the flow of the specification process carried out by the specification processing unit 290A of the data processing device 12.

[0363] In step S310, the acquisition unit 501 acquires image data including the facial expression of the customer 400, voice data including the spoken voice of the customer 400, and biometric data of the sales representative 401 transmitted from the necklace-type terminal 14, and stores this data in the data storage unit 59.

[0364] In step S311, the emotion recognition unit 502 recognizes the emotional state of the customer 400 based on the voice data and image data acquired in step S310.

[0365] In step S312, the stress level deriving unit 503 derives the stress level of the sales representative 401 based on the biometric data acquired in step S310.

[0366] In step S313, the feedback generation unit 504 uses the data generation model 58 to generate effective feedback for successfully conducting sales activities toward the customer 400, based on the emotional state of the customer 400 recognized in step S311 and the stress level of the sales representative 401 derived in step S312.

[0367] In step S314, the communication unit 506 transmits the voice data including the content of the feedback generated in step S313 to the necklace type terminal 14. The speaker 40 of the necklace type terminal 14 outputs voice including the content of the feedback.

[0368] In step S315, the report generation unit 505 determines whether the business negotiation has ended. The report generation unit 505 may determine whether the business negotiation has ended based on the conversation between the customer 400 and the sales representative 401 transmitted through the microphone of the necklace-type terminal 14, for example. If it is determined that the business negotiation has not ended, the process returns to step S310. If it is determined that the business negotiation has ended, the process proceeds to step S316. A business negotiation record including the content of the conversation between the customer 400 and the sales representative 401 during the business negotiation, the emotional state of the customer 400 during the business negotiation, the stress level of the sales representative 401 during the business negotiation, and the content of the feedback generated by the feedback generation unit 504 is stored in the storage 32 of the data processing device 12.

[0369] In step S316, the report generation unit 505 generates a report including improvements and recommended approaches for the next negotiation using the data generation model 58 based on the negotiation records stored in the storage 32. The report generation unit 505 may generate the report using customer information including past negotiation history, customer preferences, and past purchase history, which can be obtained from the database stored in the storage 32.

[0370] In step S317, the communication unit 506 transmits audio data including the contents of the report generated in step S316 to the necklace type terminal 14. The speaker 40 of the necklace type terminal 14 outputs audio including the contents of the report.

[0371] As described above, the data processing device 12 according to this embodiment includes an emotion recognition unit 502 that recognizes the emotional state of the customer 400 based on audio data including the speech of the customer 400 and image data including the facial expressions of the customer 400, a stress level derivation unit 503 that derives the stress level of the sales representative 401 based on biometric data of the sales representative, and a feedback generation unit 504 that uses a data generation model 58 to generate effective feedback for successfully conducting sales activities toward the customer based on the recognized emotional state of the customer 400 and the derived stress level of the sales representative.

[0372] The feedback generation unit 504 uses the data generation model 58 to generate feedback advice on how to proceed with the negotiation based on the recognized emotional state of the customer 400. For example, if the recognized emotional state of the customer 400 is "dissatisfied," a voice message including content such as "The customer is dissatisfied. Let's change the topic" is provided to the salesperson 401. This allows the salesperson to be informed in real time of the optimal approach to match the progress of the negotiation.

[0373] If the derived stress level of the salesperson is equal to or greater than a threshold, the feedback generation unit 504 uses the data generation model 58 to generate, as feedback, a method of reducing the stress level. For example, if the stress level of the salesperson 401 is high, an intervention using a breathing technique guide or relaxation audio is implemented. This reduces the stress of the salesperson 401 and contributes to the smooth progress of the business negotiations.

[0374] If the correlation between the recognized emotional state of customer 400 and the derived stress level of salesperson 401 is equal to or greater than a certain level, feedback generator 504 uses data generation model 58 to generate a warning for salesperson 401 as feedback. For example, if the correlation is equal to or greater than a certain level, a voice message including content such as, "The customer's emotions and your stress level are mutually influencing each other. By relaxing, you can change the customer's emotions to a positive one" is provided to the salesperson. This makes it possible to guide the emotional state of customer 400 in a positive direction while reducing the stress of salesperson 401.

[0375] According to the data processing system 10 of this embodiment, it is possible to simultaneously handle customer correspondence and manage the stress of sales staff.

[0376] Eighth Embodiment An eighth embodiment of the disclosed technology will be described. Note that the configurations of the data processing system 10, data processing device 12, and necklace-type terminal 14 are the same as those of the first embodiment (see FIGS. 1 to 4), and therefore their description will be omitted. Furthermore, the processing of the control unit 46A when the necklace-type terminal 14 performs a data collection process to collect data is also the same as that of the first embodiment (see FIG. 5), and therefore its description will be omitted.

[0377] Referring to FIG. 21, the process of the identification processing unit 290 when the data processing device 12 according to the eighth embodiment performs identification processing for acquiring a response corresponding to a user utterance will be described.

[0378] The input unit 292 stores the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace type terminal 14 in the data storage unit 54. In other words, the input unit 292 acquires biometric data transmitted from the necklace type terminal 14 worn by the user 20.

[0379] The input unit 292 acquires the user utterance received by the necklace type terminal 14. Specifically, the input unit 292 acquires the user utterance picked up by the microphone 38 of the necklace type terminal 14. That is, the input unit 292 acquires the user utterance acquired by the necklace type terminal 14 for the action plan described below.

[0380] The processing unit 294 performs a specification process using the data generation model 58. Specifically, the processing unit 294 inputs the biometric data acquired by the input unit 292 and a prompt that instructs the data generation model 58 to generate an action plan for health management of the user 20 based on the biometric data. The data generation model 58 outputs an action plan corresponding to the prompt. The processing unit 294 acquires the action plan output from the data generation model 58.

[0381] An example of a prompt is a statement such as "Please suggest an action plan for the user's health management based on the user's biometric data."

[0382] Examples of the action plan include an exercise plan, a meal plan, a plan for improving sleep, and / or daily behavior patterns. For example, if the user 20 has high blood pressure, the action plan may include an exercise menu and a meal menu for resolving the high blood pressure. Also, for example, if the user 20 is sleep-deprived, the action plan may include an exercise menu and a meal menu for improving the sleep deprivation. Furthermore, the action plan may include not only a plan for managing the physical health of the user 20, but also a plan for managing the mental health of the user 20.

[0383] In addition, the processing unit 294 provides feedback to the data generation model 58 using the user utterances acquired by the input unit 292 in response to the action plan acquired as described above and output by the output unit 296 to the necklace-type terminal 14.

[0384] Specifically, the processing unit 294 inputs a prompt to the data generation model 58, the prompt including an instruction to analyze whether the user utterance prompts a change in the action plan and, if the user utterance prompts a change in the action plan, an instruction to change the action plan based on the user utterance. That is, the prompt instructing to change the action plan also includes a sentence instructing to analyze whether the user utterance prompts a change in the action plan. If the user utterance prompts a change in the action plan, a new action plan corresponding to the prompt is output from the data generation model 58. In this case, the processing unit 294 acquires the new action plan output from the data generation model 58. If the user utterance does not prompt a change in the action plan, for example, text indicating that the action plan will not be changed is output from the data generation model 58. Note that the processing unit 294 may analyze whether the user utterance prompts a change in the action plan using a natural language processing model obtained in advance by machine learning, separate from the data generation model 58.

[0385] An example of the prompt is a sentence such as, "Please analyze whether the user's utterance prompts a change in the action plan. If the user's utterance prompts a change in the action plan, please propose a new action plan for the user's health management." For example, if the action plan proposed to the user 20 is an exercise plan and the user's utterance is an utterance indicating a reduction in the intensity of the exercise, such as "Please exercise a little more lightly," the new action plan will be an exercise plan with a lower intensity than the previously proposed action plan.

[0386] The output unit 296 outputs (i.e., transmits) the action plan generated by the processing unit 294 to the necklace type terminal 14 as a result of the specific processing. In the necklace type terminal 14, the control unit 46A causes the speaker 40 to output the result of the specific processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the specific processing. The control unit 46A transmits voice data indicating the user utterance acquired by the microphone 38 to the data processing device 12. The data processing device 12 acquires the user utterance.

[0387] In this way, by interactively communicating between the user 20 and the data processing system 10, the action plan proposed by the data processing system 10 to the user 20 becomes personalized for the user 20. As a result, an appropriate action plan can be proposed to the user 20, enabling more accurate and effective health management.

[0388] The output unit 296 may also output the results of the specific processing to a mobile device such as a smartphone that is set up for linkage with the necklace-type terminal 14, if the output unit 296 is capable of communicating with the mobile device. Furthermore, if the user 20 has previously agreed to data linkage with a specific hospital, the output unit 296 may output the biometric data of the user 20 and the action plan proposed to the user 20 to the system of the specific hospital.

[0389] Furthermore, the processing unit 294 may generate an action plan using lifestyle data of the user 20, such as the amount of daily exercise, dietary content, and sleep patterns, in addition to the biometric data of the user 20.

[0390] Furthermore, the processing unit 294 may evaluate the health condition of the user 20 using the biometric data acquired by the input unit 292. In this case, the output unit 296 may output the evaluation result of the health condition of the user 20 to the necklace type terminal 14. For example, the processing unit 294 may input the biometric data acquired by the input unit 292 and a prompt instructing to evaluate the health condition of the user 20 using the biometric data to the data generation model 58, and acquire the evaluation result output from the data generation model 58.

[0391] Next, we will explain the operation of the data processing system 10. Note that the flow of the data collection process is the same as in the first embodiment, so the explanation will be omitted.

[0392] Next, an example of the flow of the identification process according to the eighth embodiment will be described with reference to Fig. 21. Here, it is assumed that the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 54. Note that steps in Fig. 21 that execute the same processes as those in Fig. 7 are assigned the same reference numerals, and their description will be omitted.

[0393] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S400-7. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 ends the identification process.

[0394] In step S400-7, the processing unit 294 generates text as a prompt instructing the generation of an action plan for health management of the user 20 based on the biometric data, as described above. In step S402-7, the processing unit 294 inputs the biometric data acquired by the input unit 292 and the prompt generated in step S400 to the data generation model 58. The processing unit 294 acquires the action plan output from the data generation model 58 in response to the prompt.

[0395] In step S404-7, the output unit 296 outputs the action plan generated in step S402-7 to the necklace type terminal 14. In step S406-7, the processing unit 294 generates a prompt including an instruction to analyze whether the user utterance acquired by the input unit 292 corresponding to the action plan output in step S404-7 indicates a prompt to change the action plan, and an instruction to change the action plan based on the user utterance if the user utterance indicates a prompt to change the action plan.

[0396] In step S408-7, the processing unit 294 inputs the prompt generated in step S406-7 to the data generation model 58. The processing unit 294 acquires the output of the data generation model 58 corresponding to the prompt. In step S410-7, the processing unit 294 determines whether the output of the data generation model 58 acquired in step S408-7 is a new action plan. If the output of the data generation model 58 is a new action plan in step S410-7 (step S410-7; Yes), the data processing system 10 proceeds to step S412-7. On the other hand, if the output of the data generation model 58 is not a new action plan in step S410-7 (step S410-7; No), the data processing system 10 ends the identification process.

[0397] In step S412-7, the output unit 296 outputs the new action plan acquired in step S410-7 to the necklace-type terminal 14, and the identification process ends.

[0398] [Ninth embodiment] A ninth embodiment according to the present disclosure will be described below. Note that the ninth embodiment has the same system configuration as the data processing system 10 shown in Figures 1 and 2 described as the first embodiment, and therefore Figures 1 and 2 will be used as the data processing system 10 according to the ninth embodiment, and the same components will be described using the same reference numerals.

[0399] In the first embodiment, the configuration is such that biometric data and situation data of the user 20 are collected in real time to detect early signs of Alzheimer's disease, dementia, etc., and to monitor the health condition (e.g., heart disease) of the user 20. In other words, the necklace-type terminal 14 of the first embodiment has a so-called "reactive" function that responds after an abnormality occurs, for example, and monitors the health condition, etc. of the user 20.

[0400] In contrast to this, in the eighth embodiment, the data processing system 10 is configured to have a so-called "proactive intervention" function that predicts risks and intervenes early before abnormalities occur in order to more effectively protect the health of the user 20. Furthermore, the data processing system 10 in the ninth embodiment has a learning function that learns the past data and behavioral patterns of the user 20.

[0401] In the following description, the specific processing performed by the specific processing unit 290 will be distinguished as normal specific processing when a reactive function is performed, and as prediction mode specific processing when a proactive intervention function is performed.

[0402] In the ninth embodiment, the sensor 39 attached to the necklace-type terminal 14 is composed of a sensor that detects biological data and a sensor that detects environmental data around the user.

[0403] The sensors for detecting biological data include at least one of a heart rate sensor, a blood oxygen sensor, a blood pressure sensor, a body temperature sensor, and a breathing pattern detection sensor, and the sensors for detecting environmental data include at least one of a position sensor, a temperature sensor, and a humidity sensor.

[0404] The sensors do not need to be concentrated in one location, and may be placed, if necessary, on other parts of the necklace-type terminal 14, or via a wireless device, on the user's 20 body at a location that is most convenient for collecting the desired data, such as the wrist, ankle, or chest.

[0405] FIG. 22 illustrates in detail the functional configuration of the specific processing unit 290A according to the ninth embodiment.

[0406] The specific processing unit 290A in the ninth embodiment includes an input unit 292A, a processing unit 294A, and an output unit 296A. The processing unit 294 includes a proactive intervention function.

[0407] The input unit 292A receives, in real time, biometric data of the user 20, such as the heart rate, blood pressure, body temperature, and breathing pattern of the user 20, based on the outputs of the microphone 38, the sensor 39, and the camera 42, collected in the data collection process of the control unit 46A of the necklace-type terminal 14. In addition to the biometric data, all situational data around the user 20 is also input.

[0408] Furthermore, past sensor data and behavior pattern data of the user 20 stored in the data storage unit 54 are input to the input unit 292A.

[0409] The processing unit 294A performs a specification process (normal specification process) using the data generation model 58 (see FIG. 2). Specifically, a prompt including a user utterance is input to the data generation model 58 to obtain a generation result. At this time, the prompt may further include outputs from the sensor 39 and the camera 42 collected by the data collection unit 100.

[0410] The processing unit 294A also performs a specific process (prediction specific process) using a proactive intervention function. In the prediction specific process, potential risks to the user's health are predicted (prediction mode is executed) based on the biometric data and situation data of the user 20 collected via the input unit 292A and the user's past sensor data and behavioral pattern data stored in the data accumulation unit 54 (see FIG. 2 ). A prompt instructing the data generation model 58 to analyze a behavioral pattern for early intervention regarding the risk (e.g., advice on risk avoidance behavior, warnings, etc.) is input to the data generation model 58, and the analysis result of the behavioral pattern is obtained. For example, the biometric data and situation data of the user 20 collected via the input unit 292A and the user's past behavioral pattern data stored in the database accumulation unit 54 (see FIG. 2 ) are input to the data generation model 58, and a prompt stating, "This is the user's biometric data, situation data, and past behavioral pattern data. Please predict potential risks to the user's health and generate warnings for early intervention regarding the risk" is input to the data generation model 58. As a result, the data generation model 58 generates a warning message saying, "There is a risk of dementia, so please be mindful of developing behavioral habits to prevent dementia."

[0411] This makes it possible to detect signs of Alzheimer's disease and dementia that are tailored to the individual user's living environment before the onset of the disease (before early symptoms appear). The behavioral patterns analyzed by the processing unit 294A are sent to the output unit 296A.

[0412] Next, the operation of the ninth embodiment will be described. First, an example of the flow of data collection processing will be described.

[0413] When the user 20 is wearing the necklace-type terminal 14, the data collection unit 100 sequentially collects the outputs of the microphone 38, the sensor 39, and the camera 42. At this time, a feature of the ninth embodiment is that it collects various data (heart rate, blood oxygen, body temperature, breathing pattern, etc.) detected by the sensor 39, which is a collection of biosensors including a heart rate sensor, a blood oxygen sensor, a blood pressure sensor, a body temperature sensor, and a breathing pattern detection sensor. The communication unit 102 sequentially transmits the outputs of the microphone 38, the sensor 39, and the camera 42 collected by the data collection unit 100 to the data processing device 12.

[0414] Next, an example of the flow of the identification process will be described with reference to Fig. 23. Here, it is assumed that the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace-type terminal 14 and stores them in the data accumulation unit 54.

[0415] In step S300A, the processing unit 294 determines whether a predetermined trigger condition is satisfied. Specifically, the trigger condition may be that a specific word (e.g., the name of an agent installed in the necklace-type terminal 14) or phrase (e.g., "Hi! XXX" (XXX is the name of the agent)) is included in the user utterance picked up by the microphone 38.

[0416] If the trigger condition is met in step S300A (step S300A; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300A (step S300A; No), the data processing system 10 ends the identification process.

[0417] In step S301A, the processing unit 294 generates a prompt by adding an instruction sentence for obtaining a result of a specific process to text representing a user utterance picked up by the microphone 38.

[0418] For example, a prompt such as "The user is uttering the following: XXX. Please respond as an agent." (XXX is the user utterance) may be generated. Alternatively, the outputs of the sensor 39 and the camera 42 may be added to the prompt to generate a prompt such as "This is the user's biometric data and video data showing the user's surroundings. The user is also uttering the following: XXX. Please respond as an agent." (XXX is the user utterance). In this case, if the user 20 utters, "Are there any signs that my health may be affected in the future?", the emergency trigger condition is met, and the proactive intervention function is executed, enabling a transition from normal identification processing (normal mode) to predicted identification processing (prediction mode).

[0419] In prediction mode, the proactive intervention function is used to predict potential risks to the health of the user 20 based on the collected biometric data and situational data of the user 20, as well as the past sensor data and behavioral pattern data of the user 20 stored in the data storage unit 54.

[0420] Furthermore, as described above, the emergency trigger condition may be established when the collected biometric data and situation data of the user 20 satisfy a predetermined condition, and the prediction mode may be entered regardless of the user's utterance.

[0421] In step S303A, the processing unit 294 inputs the generated prompt into the data generation model 58, and based on the output of the data generation model 58, obtains the results of the normal specific processing (execution of the normal specific mode according to the first embodiment) or the prediction specific processing (execution of the prediction mode).

[0422] In the prediction mode, a prompt is input to the data generation model 58 to analyze behavioral patterns for early intervention regarding risks (e.g., advice on risk avoidance behavior, warnings, etc.), and the analysis results of the behavioral patterns are obtained.

[0423] 24 shows a subroutine flow of the detailed processing of step 303A in FIG. 8. In step 303A, it is determined whether the prompt is a predictive mode. If the determination in step 303A is negative, the process proceeds to step 303B, where the specification processing described in detail in the first embodiment is executed, and then the process returns. If the determination in step 303A is positive, the process proceeds to step 303C, where the predictive mode specification processing (proactive intervention function) according to the ninth embodiment is executed, and then the process returns.

[0424] In step S304A, the output unit 296 outputs the result of the identification process to the necklace-type terminal 14, and the identification process ends.

[0425] As described above, in the ninth embodiment, in addition to the reactive function (normal specific processing) of the first embodiment, in order to more effectively protect the health of the user 20, a proactive intervention function is provided that predicts risks before abnormalities occur and intervenes early, and a learning function that learns the past data and behavioral patterns of the user 20 (prediction mode), making it possible to take measures that are appropriate for each individual user 20.

[0426] In this embodiment, a response is made to the user 20 in the prediction mode identification process, but the data processing device 12 may also be provided with a real-time data log processing function that automatically stores all relevant data when an abnormality in health status is indicated, and performs detailed analysis or provides the data to a medical institution.

[0427] Tenth Embodiment FIG. 25 shows an example of the configuration of a data processing system 10 according to a tenth embodiment. In educational settings, teachers need to accurately grasp the status of each student and provide optimal instruction. However, it is difficult to grasp the status of all students in real time, which places a heavy burden on teachers. This has led to problems such as inability to provide appropriate instruction or adequately manage the entire class. In the tenth embodiment, specific processing by the data processing system 10 is introduced into educational settings to solve these problems.

[0428] A data processing system 10 according to a tenth embodiment includes a data processing device 12 and a plurality of necklace-type terminals 14. In the figure, an example is shown in which the data processing system 10 includes one data processing device 12 and four necklace-type terminals 14_0 to 14_3. However, the number of necklace-type terminals 14 is not limited to this. The data processing system 10 may include more than four necklace-type terminals 14, such as 30 to 40 necklace-type terminals 14.

[0429] As an example, the user 20 wearing the necklace-type terminal 14_0 may be a teacher, and the users 20 wearing the necklace-type terminals 14_1 to 14_3 may be students. Here, it is assumed that the necklace-type terminals 14_1, 14_2, and 14_3 are worn by students A, B, and C, respectively, in the class taught by the teacher.

[0430] That is, in this embodiment, the necklace-type terminal 14_0 is an example of a "teacher-side terminal" according to the present disclosure, and the necklace-type terminals 14_1 to 14_3 are examples of "student-side terminals" according to the present disclosure. In this way, the student-side terminal may be the necklace-type terminal 14, and the teacher-side terminal may also be the necklace-type terminal 14.

[0431] In this case, the specific processing unit 290 may pre-register that the necklace-type terminal 14_0 is the teacher's terminal and the necklace-type terminals 14_1 to 14_3 are student's terminals, more specifically, that the necklace-type terminals 14_1, 14_2, and 14_3 are student's terminals to be worn by student A, student B, and student C, respectively.

[0432] Such registration may be performed manually by the user pressing a button (not shown) provided on the necklace-type terminal 14, or may be performed automatically based on the user's speech. In this way, when the same necklace-type terminal 14 is used as both the teacher terminal and the student terminal, the identification processing unit 290 may register whether the necklace-type terminal 14 is the teacher terminal or the student terminal.

[0433] The necklace-type terminals 14_0 to 14_3 are communicatively connected to the data processing device 12 via the network 54. Therefore, the data processing device 12 can collect the output of each of the microphones 38, sensors 39, and cameras 42 provided in each of the necklace-type terminals 14_0 to 14_3. The data processing device 12 can also cause the speaker 40 provided in each of the necklace-type terminals 14_0 to 14_3 to output the results of specific processing.

[0434] 26 shows an example of a first operational flow of the specific processing by the data processing device 12 according to the tenth embodiment. This flow may be started automatically when the lesson starts, or may be started manually by the teacher pressing a start button or by speaking a command to start the lesson.

[0435] In step S400-10, the processor 28 collects data output from at least one of the microphone 38, the sensor 39, and the camera 42 provided in the student terminal. For example, the processor 28 may collect in real time the output of each of the microphone 38, the sensor 39, and the camera 42 provided in each of the necklace-type terminals 14_1 to 14_3.

[0436] In step S401-10, the processor 28 analyzes the data collected in step S400-10 to estimate the states of the students. For example, the processor 28 may analyze the biological data (e.g., heart rate and blood oxygen concentration) detected by the sensor 39 to estimate the mental states (e.g., stress level and concentration level) of each of the students A, B, and C.

[0437] Alternatively or additionally, the processor 28 may analyze the voice data (e.g., tone of voice) collected by the microphone 38 and the image data (e.g., facial expressions) captured by the camera 42 to estimate the emotional states of each of the students A, B, and C. Such analysis processing itself may be based on existing technology, and therefore a detailed description thereof will be omitted here.

[0438] In step S402-10, the processor 28 determines whether a trigger condition is met based on the result of the estimation in step S401-10. For example, the processor 28 may determine whether the mental state of any of the students satisfies a predetermined condition. More specifically, the processor 28 may determine whether the stress level of any of the students exceeds a predetermined threshold or whether the concentration level of any of the students has fallen below a predetermined threshold.

[0439] Alternatively, or in addition, processor 28 may determine whether the emotional state of any of the students is mapped to a predetermined emotion. More specifically, processor 28 may determine whether the emotion of any of the students is mapped to "difficult" or "distressed."

[0440] If it is determined in step S402-10 that the trigger condition is not satisfied (No), the processor 28 proceeds to step S403-10. In step S403-10, the processor 28 determines whether the end condition is satisfied. For example, the processor 28 may determine whether the lesson has ended. Alternatively, or in addition, the processor 28 may determine whether the teacher has pressed the stop button or uttered an utterance indicating that the lesson is ending.

[0441] If it is determined in step S403-10 that the termination condition is met (Yes), the processor 28 terminates this flow. On the other hand, if it is determined that the termination condition is not met (No), the processor 28 returns the process to step S400-10 and continues the flow.

[0442] If it is determined in step S402-10 that the trigger condition is met (Yes), the processor 28 proceeds to step S404-10, where the processor 28 provides feedback to the teacher terminal based on the result of the estimation made in step S401-10.

[0443] For example, suppose that the concentration level of Student A among Student A, Student B, and Student C is below a predetermined threshold. In this case, the processor 28 generates a prompt instructing the teacher to suggest advice according to the student's concentration level. For example, the processor 28 generates a prompt such as, "Student A's concentration level is declining. Please suggest some advice for the teacher." Next, the processor 28 inputs the generated prompt into the data generation model 58, thereby obtaining from the data generation model 58 a message such as, "Tell Student A, 'Class will be over in five minutes, so concentrate!'"

[0444] The processor 28 then transmits the acquired message to the necklace-type terminal 14_0. In response, the message is output as voice from the speaker 40 provided on the necklace-type terminal 14_0, allowing the teacher to tell Student A, "Class will be over in five minutes, so concentrate!" In this way, the processor 28 can provide feedback to the teacher's terminal based on the mental state of the student.

[0445] Similarly, for example, assume that the emotion of Student B among Student A, Student B, and Student C is mapped to "distressed." In this case, the processor 28 generates a prompt instructing the teacher to suggest advice according to the student's emotion. For example, the processor 28 generates a prompt such as "Student B is feeling distressed. Please suggest some advice for the teacher." Next, the processor 28 inputs the generated prompt into the data generation model 58, thereby obtaining from the data generation model 58 a message such as "Student B's hobby is soccer. Please talk about soccer."

[0446] The processor 28 then transmits the acquired message to the necklace-type terminal 14_0. In response, the message is output as audio from the speaker 40 provided on the necklace-type terminal 14_0, allowing the teacher to switch the topic to soccer. In this way, the processor 28 can also provide feedback to the teacher's terminal based on the emotional state of the students.

[0447] In this way, for example, processor 28 can provide feedback to the teacher's terminal based on the results of estimating the state of the student. Although the above explanation has shown an example of providing feedback to the teacher's terminal, feedback can also be provided to the student's terminal.

[0448] For example, if the concentration level of student A is below a predetermined threshold, the processor 28 may send a vibration command to the necklace-type device 14_1. In response, the vibration function of the necklace-type device 14_1 is turned on, allowing student A to come to his senses and regain his concentration. In this way, the processor 28 can also provide feedback to the student-side device based on the estimated result.

[0449] In the above description, the case where the trigger condition is determined for each student is described as an example, but the present invention is not limited to this. For example, processor 28 may perform statistical processing on the stress levels of student A, student B, and student C.

[0450] Suppose a statistical value (e.g., a total value, an average value, etc.) exceeds a predetermined threshold. In this case, processor 28 generates a prompt stating, "The stress level of all students is rising. Please suggest some advice for the teacher." Next, processor 28 inputs the generated prompt into data generation model 58, thereby obtaining a message from data generation model 58 stating, "The class is feeling stressed. Please take a three-minute break."

[0451] The processor 28 then transmits the acquired message to the necklace-type terminal 14_0. In response, a message is output as audio from the speaker 40 provided on the necklace-type terminal 14_0, allowing the teacher to urge the class to take a break. In this way, the processor 28 can also provide feedback to the teacher's terminal based on the results of statistical processing of the states of multiple students.

[0452] In the above description, a case where whether or not the trigger condition is satisfied is determined based on the state of the student at that time has been described as an example, but the present invention is not limited to this. Processor 28 may also predict the future state of the student from the estimated trend of the student's state, and determine whether or not the trigger condition is satisfied based on the prediction result.

[0453] Thus, the data processing system 10 according to the tenth embodiment includes a teacher terminal worn by a teacher, student terminals worn by each of a plurality of students, and a data processing device 12 communicably connected to the teacher terminal and the student terminals. The data processing device 12 collects data output from at least one of the microphone 38, sensor 39, and camera 42 provided on the student terminals, analyzes the data to estimate the status of the plurality of students, and provides feedback to the teacher terminal based on the estimated results. This reduces the burden on teachers to properly instruct a large number of students at once. This supports the optimization of instruction, thereby maximizing the learning effect of students.

[0454] 27 shows an example of a tenth operational flow of the specific processing by the data processing device 12 according to the tenth embodiment. This flow may be executed prior to or in parallel with the flow of FIG.

[0455] In step S500-10, the processor 28 acquires learning data indicating the learning history of each of the students. For example, the processor 28 may access an external database to acquire learning data including the learning efforts, progress, assignment submission status, grades, etc., of each of the students.

[0456] In step S501-10, the processor 28 analyzes the learning data acquired in step S500-10 and generates a curriculum for each of the multiple students. For example, the processor 28 generates a prompt such as, "Please propose a curriculum for student C." Next, the processor 28 inputs the generated prompt, along with the learning data for student C, into the data generation model 58, thereby acquiring from the data generation model 58 a message such as, "Please have student C study page 10 of textbook X."

[0457] In step S502-10, the processor 28 transmits a message corresponding to the curriculum generated in step S501-10 to the necklace-type terminal 14_0. For example, the processor 28 may transmit the message acquired from the data generation model 58 in step S501-10 to the necklace-type terminal 14_0. In response to this, a message is output as voice from the speaker 40 provided in the necklace-type terminal 14_0, allowing the teacher to instruct Student C to "Study page 10 of Textbook X." The same applies to Student A and Student B.

[0458] In step S503-10, the processor 28 determines whether or not there are any questions. At this time, the processor 28 may analyze the voice data collected by the microphone 38 and accept questions from each of the multiple students.

[0459] If it is determined in step S503-10 that there is no question (No), the processor 28 proceeds to step S504-10. In step S504-10, the processor 28 determines whether or not a termination condition is met. The termination condition may be the same as that in step S403-10, for example, and therefore a duplicated description will be omitted here.

[0460] If it is determined in step S504-10 that the termination condition is met (Yes), the processor 28 terminates this flow. On the other hand, if it is determined that the termination condition is not met (No), the processor 28 returns the process to step S503-10 and continues the flow.

[0461] If it is determined in step S503-10 that a question exists (Yes), the processor 28 proceeds to step S505-10, where the processor 28 transmits the answer to the question to the student terminal worn by the student who asked the question.

[0462] For example, suppose that analysis of the voice data collected by the microphone 38 reveals that student C, among students A, B, and C, utters, "I don't understand the second question." In this case, the processor 28 generates a prompt to the effect, "Please tell me how to solve the second question on page 10 of textbook X." Next, the processor 28 inputs the generated prompt into the data generation model 58, thereby acquiring an explanation for the second question on page 10 of textbook X from the data generation model 58.

[0463] Then, the processor 28 transmits the acquired explanation to the necklace-type terminal 14_3. In response, the explanation is output as audio from the speaker 40 provided on the necklace-type terminal 14_3, allowing the student C to understand how to solve the second question on page 10 of the textbook X. The processor 28 repeatedly receives and answers questions in this manner until it is determined in step S504-10 that the termination condition is satisfied.

[0464] As described above, in the data processing system 10 according to the tenth embodiment, the data processing device 12 acquires learning data indicating the learning history of each of the students, analyzes the learning data to generate a curriculum for each of the students, and transmits messages corresponding to the curriculum to the teacher's terminal. At this time, the data processing device 12 analyzes the audio data collected by the microphone 38 to receive questions from each of the students, and transmits answers to the questions to the student's terminal worn by the student who asked the question. Thus, the data processing system 10 according to the tenth embodiment can provide a more personalized education to each student.

[0465] [Eleventh Embodiment] Next, a data processing system according to an eleventh embodiment will be described. Fig. 28 shows an example of the configuration of a data processing system 11 according to this embodiment.

[0466] The data processing system 11 according to this embodiment is configured to further include a display device 16 in addition to the basic configuration.

[0467] The display device 16 has the function of displaying images transmitted from an external device such as the necklace-type terminal 14 or the data processing device 12, and may be, for example, smart glasses, a mobile terminal such as a smartphone, or another device such as a personal computer.

[0468] Furthermore, the display device 16 may be connected to the necklace-type terminal 14 and the data processing device 12 via a network 54, as shown in FIG. 28, or may be directly wirelessly connected to the necklace-type terminal 14 and / or the data processing device 12 via wireless communication such as Wi-Fi or Bluetooth (registered trademark).

[0469] The display device 16 may also be configured to include a camera and to be capable of applying AR (Augmented Reality) technology. That is, the display device 16 may display an image captured by a camera and overlay digital information (augmented reality content) on the displayed image.

[0470] As shown in FIG. 29, the necklace-type terminal 14 in this embodiment further includes a providing unit 104 in a control unit 46A in addition to the basic configuration.

[0471] The provision unit 104 provides learning plans and augmented reality content generated by generative AI based on data collected by the data collection unit 100, as well as a discussion function with other students and / or teachers.

[0472] In this embodiment, the data processing device 12 performs the identification process by proposing an optimal study plan using data collected from the necklace-type device 14 worn by the student. Examples of the proposed study plan include review focused on areas of shallow understanding and suggestions for additional learning materials based on the student's interests. For example, the identification process involves using generative AI to generate a study plan based on the student's stress level and cognitive characteristics, based on the data collected by the data collection unit 100 of the necklace-type device 14.

[0473] Furthermore, in this embodiment, the data processing device 12 has multiple functions, such as a function to provide interactive learning using augmented reality (AR), a function to promote community learning, a function to manage student health, a multilingual function, and a skill development function.

[0474] The interactive learning function utilizes AR technology through the necklace-type device 14 and the display device 16, for example, to enable students to learn by integrating the physical world with digital information. Specifically, in a history class, historical places and events can be recreated on the display device 16 using AR, allowing students to learn as if they were actually there. This deepens students' understanding of the learning content and encourages them to engage with it with interest.

[0475] The function to promote community learning allows students to connect with other students and teachers in real time via the necklace-type device 14, for example, to work together on projects and hold discussions. This function allows students to learn from each other and exchange opinions, fostering social skills and cooperation. AI analyzes conversations and activities within the community and suggests optimal group formation and role allocation.

[0476] The student health management function monitors students' health status in addition to their learning activities. For example, if a student continues studying for a long period of time, the necklace-type device 14 will prompt them to take a break or suggest simple exercise. It also integrates data on diet and sleep to help students maintain a balanced lifestyle. This data may be shared with parents and medical professionals.

[0477] The multilingual support function uses AI to provide multilingual support, aiming to cultivate students with a global perspective. For example, lesson content can be provided in multiple languages, and content designed to foster intercultural understanding and international awareness can be provided. This allows students to learn while being exposed to the cultures and languages ​​of other countries.

[0478] The skill development function provides programs to foster skills that will be in demand in the future (such as critical thinking, problem-solving ability, and creativity). For example, AI analyzes a student's learning history and performance and creates a special training plan to strengthen the skills they will need for the future. This allows students to efficiently acquire the skills they will need for the future.

[0479] Next, an example of the flow of the identification process in this embodiment will be described. Fig. 30 is a sequence diagram showing the flow of the identification process in this embodiment.

[0480] The necklace-type terminal 14 worn by the student acquires the student's biometric data, photographed images, voice, etc.

[0481] The acquired data is transmitted from the communication unit 102 of the necklace-type terminal 14 to the data storage unit 62 of the data processing device 12 and stored therein.

[0482] The specific processing unit 390 generates a study plan and augmented reality content using data stored in the data storage unit 62. Specifically, the processing unit 294 generates a prompt requesting the generation of a study plan and augmented reality content, including the collected data, and inputs the generated prompt into the data generation model 58 to obtain the generation results, thereby generating the study plan and augmented reality content. For example, the study plan and augmented reality content are generated by generating a prompt requesting the generation of a study plan and augmented reality content according to the study content and inputting the prompt into the generation system AI. Note that when generating augmented reality content, images captured by the camera of the display device 16 may also be input into the generation system AI to generate augmented reality content that matches the captured images. The prompt requesting the generation of a study plan may include inputting the collected data and using a prompt such as, "This data is a collection of the student's biometric data, captured images, and audio. Please generate a study plan to improve this student's motivation to learn." At this time, the generation system AI generates a study plan indicating the learning materials to be studied for each period. Furthermore, a prompt for requesting the generation of augmented reality content could be to input the collected data and use a prompt such as, "This data is a collection of the student's biometric data, photographed images, and audio. Please generate augmented reality content to increase this student's motivation to learn." In this case, the generative AI generates augmented reality content that shows an image of the scene when the learning goal is achieved and a vision of the student's future after achieving the learning goal.

[0483] The study plan and augmented reality content generated by the specific processing unit 390 are transmitted to the necklace-type terminal 14 .

[0484] In the necklace-type device 14, the communication unit 102 receives the generated lesson plan and augmented reality content, and the provision unit 104 provides the lesson plan and augmented reality content to the student. For example, the lesson plan and augmented reality content are provided to the student by transmitting them to the display device 16 and displaying them. Specifically, in a history class, historical places and events are reproduced on the display device 16 using augmented reality content. This allows students to study as if they were actually there, deepening their understanding of the learning content and encouraging them to engage with the material with interest.

[0485] Furthermore, when providing students with learning plans and augmented reality content, the microphone 38 and speaker 40 of the necklace-type device 14 can be used to communicate with other students and teachers wearing necklace-type devices 14 for discussion, allowing students to learn from each other and exchange opinions, fostering social skills and cooperation. Furthermore, by analyzing conversations and activities within the community using AI, it is possible to suggest optimal group formation and role allocation.

[0486] Next, specific processing performed by each unit of the data processing system 11 according to this embodiment will be described.

[0487] First, we will explain the processing performed in the necklace-type terminal 14. Fig. 31 is a flowchart showing an example of the flow of processing performed in the necklace-type terminal 14 of the data processing system 11 according to this embodiment. Note that the processing in Fig. 31 starts, for example, when the power (not shown) of the necklace-type terminal 14 is turned on.

[0488] In step S400-11, the data collection unit 100 acquires biometric data, captured images, and audio data. That is, the data collection unit 100 collects the outputs of the microphone 38, the sensor 39, and the camera 42.

[0489] In step S402-11, the communication unit 102 transmits the acquired data (biometric data, photographed image data, and audio data) to the data processing device 12.

[0490] In step S404-11, the communication unit 102 determines whether or not the learning plan and the augmented reality content have been received from the data processing device 12. If they have been received (step S404-11; Yes), the process proceeds to step S406-11. On the other hand, if they have not been received (step S404-11; No), the process proceeds to step S406-11.

[0491] In step S406-11, the providing unit 104 displays the received learning plan and augmented reality content on the display device 16. That is, the display device 16 can display the learning plan or overlay the augmented reality content on an image captured by a camera. This allows the optimal learning plan to be provided to the student. Furthermore, displaying the augmented reality content deepens the student's understanding of the learning content and encourages them to engage with the learning with interest. Note that the image overlaid with the augmented reality content may be an image provided by a computer operated by a teacher, instead of an image captured by a camera.

[0492] In step S408-11, the control unit 46A determines whether or not to end the process. This determination is made by determining, for example, whether or not a power source (not shown) has been turned off. If the determination is negative (S408-11; No), the process returns to step S400 and the above-described process is repeated. On the other hand, if the determination is positive (S408; Yes), the process of the necklace-type terminal 14 ends.

[0493] Next, a description will be given of the specific processing performed in the data processing device 12 of this embodiment. Fig. 32 is a flowchart showing an example of the flow of processing performed in the data processing device 12 of the data processing system 11 according to this embodiment.

[0494] In step S500-11, the input unit 292 determines whether data has been received from the necklace-type terminal 14. If data has been received (S500-11; Yes), the process proceeds to step S502-11. On the other hand, if data has not been received (S500-11; No), the process proceeds to step S504-11.

[0495] In step S502-11, the input unit 292 stores the biometric data, the captured image, and the audio data in the data storage unit 62.

[0496] In step S504-11, the processing unit 294 determines whether a study plan and augmented reality content are to be generated. This determination may be made, for example, by determining whether the accumulated amount of data collected from the necklace-type terminal 14 has reached a predetermined threshold, or by determining whether a predetermined amount of time has passed since the previous generation of the study plan and augmented reality content. Alternatively, the processing unit 294 may determine whether a request has been received from the necklace-type terminal 14. If a study plan and augmented reality content are to be generated (S504-11; Yes), the processing proceeds to step S506-11. On the other hand, if they are not to be generated (S504-11; No), the processing returns to step S500-1 and the above-described processing is repeated.

[0497] In step S506-11, the processing unit 294 acquires the stored data (biometric data, photographed image data, and audio data) from the data storage unit 62.

[0498] In step S508-11, the processing unit 294 generates a study plan and augmented reality content using each piece of data. Specifically, the processing unit 294 generates a prompt requesting the generation of a study plan and augmented reality content, including the collected data, inputs the generated prompt into the data generation model 58, and obtains the generation result, thereby generating the study plan and augmented reality content. Note that in addition to the collected data, other information, such as images captured by the camera of the display device 16, may also be input into the generative AI to generate the study plan and augmented reality content. For example, by inputting the captured image into the generative AI to generate augmented reality content, augmented reality content can be generated to match the captured image.

[0499] In step S510-11, the output unit 296 transmits the generated study plan and augmented reality content to the necklace type terminal 14, and the process returns to step S500-11 to repeat the above-described processing.

[0500] Twelfth Embodiment Next, a data processing system 10 according to a twelfth embodiment will be described while omitting or simplifying portions that overlap with the above-described embodiments.

[0501] In the data processing system 10 according to the twelfth embodiment, the data processing device 12 executes a proposal process to generate a proposal for cooking a selected recipe for the user 20 who is cooking the recipe based on the biometric data transmitted from the necklace-type terminal 14. The necklace-type terminal 14 outputs the result of the proposal process transmitted from the data processing device 12 from the speaker 40.

[0502] Next, the processing of the specific processing unit 290 when the data processing device 12 performs the proposal processing will be described.

[0503] 33 , a specific processing unit 290 according to the twelfth embodiment includes an input unit 292, an acquisition unit 298, an output unit 296, and a processing unit 294. The output unit 296 is an example of the “first output unit and second output unit” according to the technology of the present disclosure.

[0504] The input unit 292 stores the outputs of the microphone 38 , the sensor 39 , and the camera 42 received from the necklace-type terminal 14 in the data storage unit 57 .

[0505] The input unit 292 acquires the user's utterance received by the necklace type terminal 14. Specifically, the input unit 292 acquires the user's utterance picked up by the microphone 38 of the necklace type terminal 14.

[0506] The acquisition unit 298 acquires recipe data corresponding to the recipe selected by the user 20 indicated by the user utterance received by the necklace-type terminal 14 from the recipe database 70 (see FIG. 34 ).

[0507] The output unit 296 transmits the recipe data acquired by the acquisition unit 298 to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A causes the acquired recipe data to be output from the speaker 40. As a result, a voice explaining the recipe selected by the user 20 is output to the user 20 by the speaker 40.

[0508] The processing unit 294 performs a proposal process using the data generation model 58. Specifically, a first prompt including biometric data of the user 20 transmitted from the necklace-type terminal 14 or a second prompt including characteristics of ingredients of the user 20 is input to the data generation model 58, and a generation result is obtained. Specific examples of the first prompt and the second prompt will be described later.

[0509] The output unit 296 transmits the results of the proposal process to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A causes the speaker 40 to output the results of the proposal process. In this way, a proposal regarding cooking the recipe selected by the user 20 is output to the user 20 by the speaker 40.

[0510] Next, a description will be given of the configuration of the storage 32 of the data processing device 12. Fig. 34 is a block diagram showing the configuration of the storage 32 of the data processing device 12 according to the twelfth embodiment.

[0511] As shown in FIG. 34, the storage 32 stores a specific processing program 56, a data generation model 58, a data accumulation unit 57, a recipe database 70, and a characteristic database 72.

[0512] The proposal process is realized by the processor 28 operating as a specific processing unit 290 shown in Fig. 33 in accordance with a specific processing program 56 executed on the RAM 30. The data generation model 58 and the data storage unit 57 are used by the specific processing unit 290 when the proposal process is executed.

[0513] The recipe database 70 stores recipe data showing, for example, multiple recipes posted by various users on a specific website. Here, a "recipe" is an instruction manual that describes how to make a dish (procedures) and the ingredients used to confirm the procedures and necessary ingredients when making a dish. The recipe data includes the recipe name corresponding to each recipe, the cooking method indicated in the recipe, the ingredients used in the recipe, and nutritional value and allergy information for the ingredients.

[0514] The characteristics database 72 stores, for example, the characteristics of ingredients registered by various users. Here, "characteristics related to ingredients" includes the presence or absence of food allergies and food intolerances. For example, the characteristics database 72 stores ingredients to which the user 20 has food allergies and ingredients to which the user 20 has food intolerances. Hereinafter, ingredients to which the user 20 has at least one of a food allergy and a food intolerance will be referred to as "specific ingredients."

[0515] Next, an example of the flow of the proposal process will be described with reference to Figures 35 and 36. Here, it is assumed that the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, the sensor 39, and the camera 42 received from the necklace-type terminal 14 and stores them in the data accumulation unit 57.

[0516] First, a first operational flow of the suggestion process will be described with reference to Fig. 35. In step S310-12, the input unit 292 acquires a user utterance received by the necklace-type terminal 14. Here, it is assumed that the user utterance includes a specific phrase for selecting a recipe that the user wants to make (for example, "I want to eat XX" (XX is the name of a dish)).

[0517] In step S311-12, the acquisition unit 298 acquires, from the recipe database 70, recipe data corresponding to the recipe selected by the user 20 indicated by the user utterance acquired in step S310-12.

[0518] In step S312-12, the output unit 296 transmits the recipe data acquired in step S311-12 to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A causes the speaker 40 to output the acquired recipe data. As a result, audio explaining the recipe selected by the user 20 is output to the user 20 by the speaker 40. Therefore, the user 20 can proceed with cooking the recipe based on the audio output from the speaker 40. Then, while cooking the recipe, the necklace type terminal 14 automatically sets a timer according to the cooking steps, and causes the speaker 40 to output audio explaining the next step at the appropriate timing.

[0519] In step S313-12, the processing unit 294 determines whether the state of the user 20 identified based on the biometric data satisfies a predetermined condition. Specifically, the processing unit 294 determines that the predetermined condition is met if the stress level of the user 20 identified based on the biometric data is equal to or greater than a threshold, or if the value of the biometric data is equal to or greater than a threshold. If the predetermined condition is met, the process proceeds to step S314-12, and if the predetermined condition is not met, the process proceeds to step S317-12.

[0520] As an example, the processing unit 294 identifies the stress level of the user 20 based on the variability in the heart rate of the user 20 contained in the biometric data. The smaller the variability between heartbeats, the higher the stress level, and the greater the variability between heartbeats, the lower the stress level. The processing unit 294 identifies the stress level of the user 20 on a scale between 0 and 100, with 76 being the stress level threshold.

[0521] The processing unit 294 also uses blood pressure as a value of biological data, and sets "maximum blood pressure 140 mmHg" as the blood pressure threshold.

[0522] In step S314-12, the processing unit 294 generates a first prompt including biometric data of the user 20 and an instruction to output a suggestion to simplify the cooking procedure of the recipe selected by the user 20 or a suggestion of cooking content that will lead to a decrease in the user 20's blood pressure.

[0523] For example, the processing unit 294 generates a first prompt such as, "This is biometric data representing the user's heart rate. The user's stress is increasing, so please act as an agent and suggest ways to simplify the cooking steps of the recipe." Also, the processing unit 294 generates a first prompt such as, "This is biometric data representing the user's blood pressure. The user's blood pressure is high, so please act as an agent and suggest ways to cook that will lead to a decrease in the user's blood pressure."

[0524] In step S315, the processing unit 294 inputs the generated first prompt into the data generation model 58, and obtains the result of the suggestion process based on the output of the data generation model 58.

[0525] In step S316, the output unit 296 outputs the result of the suggestion process to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the suggestion process. For example, the speaker 40 outputs a voice such as "Would you like to simplify the steps and change to a time-saving recipe?" as a suggestion to simplify the cooking steps of the recipe selected by the user 20. In addition, the speaker 40 outputs a voice such as "Reduce the amount of salt to reduce the salt content" as a suggestion to cook food that will lead to a lower blood pressure of the user 20.

[0526] In step S317-12, the processing unit 294 determines whether or not a termination condition for the proposal process is met. Specifically, the processing unit 294 determines that the termination condition is met when a specific word (e.g., "finished") or phrase (e.g., "XX, the cooking is done" (XX is the name of the agent)) is included in the user utterance picked up by the microphone 38. If the termination condition is met, the proposal process ends, and if the termination condition is not met, the process returns to step S313-12.

[0527] Next, a second operation flow of the suggestion process will be described with reference to Fig. 36. In step S320-12, the input unit 292 acquires a user utterance received by the necklace-type terminal 14. Here, it is assumed that the user utterance includes a specific phrase for selecting a recipe that the user wants to make (for example, "I want to eat XX" (XX is the name of a dish)).

[0528] In step S321-12, the acquisition unit 298 acquires, from the recipe database 70, recipe data corresponding to the recipe selected by the user 20 indicated by the user utterance acquired in step S320-12.

[0529] In step S322-12, the output unit 296 transmits the recipe data acquired in step S321 to the necklace type terminal 14. In the necklace type terminal 14, the control unit 46A outputs the acquired recipe data from the speaker 40. This allows the user 20 to proceed with cooking the recipe based on the audio explaining the recipe output from the speaker 40.

[0530] In step S323-12, the processing unit 294 determines whether the ingredients used in the recipe include a specific ingredient based on the image of the user 20's surroundings received from the necklace-type terminal 14. Specifically, the processing unit 294 performs known image processing on the image of the user 20's surroundings to identify the ingredients appearing in the image. The processing unit 294 then determines that the specific ingredient is included if the identified ingredient is an ingredient for which the user 20, registered in the characteristics database 72, has at least one of a food allergy and a food intolerance. If the specific ingredient is included, the process proceeds to step S324-12; if the specific ingredient is not included, the process proceeds to step S327-12.

[0531] In step S324-12, the processing unit 294 generates a second prompt including the characteristics of the ingredient of the user 20 and an instruction to output a suggestion to exclude the particular ingredient from the recipe selected by the user 20.

[0532] For example, the processing unit 294 generates a second prompt saying, "This is the user's ingredient characteristics. The user has a food allergy to XX (XX is the name of a specific ingredient). As an agent, please make a suggestion to remove XX from the recipe."

[0533] In step S325-12, the processing unit 294 inputs the generated second prompt to the data generation model 58, and obtains the result of the suggestion process based on the output of the data generation model 58.

[0534] In step S326-12, the output unit 296 outputs the result of the proposal process to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the proposal process. For example, the speaker 40 outputs a voice message such as "This recipe contains nuts. Would you like to change it to a substitute?" as a proposal to remove a specific ingredient from the recipe selected by the user 20.

[0535] In step S327-12, the processing unit 294 determines whether or not a termination condition for the proposal process is met. Specifically, the processing unit 294 determines that the termination condition is met when a specific word (e.g., "finished") or phrase (e.g., "XX, the cooking is done" (XX is the name of the agent)) is included in the user utterance picked up by the microphone 38. If the termination condition is met, the proposal process ends, and if the termination condition is not met, the process returns to step S323-12.

[0536] As described above, in the data processing system 10, the input unit 292 acquires user utterances picked up by the microphone 38. The acquisition unit 298 acquires recipe data corresponding to a recipe selected by the user 20 indicated by the user utterance from the recipe database 70. The output unit 296 outputs the recipe data acquired by the acquisition unit 298 to the necklace type terminal 14. When the state of the user 20 identified based on the biometric data transmitted from the necklace type terminal 14 satisfies a predetermined condition, the processing unit 294 inputs a first prompt including the biometric data to the data generation model 58 and acquires a cooking suggestion for the recipe using the output of the data generation model 58. The output unit 296 outputs the cooking suggestion for the recipe acquired by the processing unit 294 to the necklace type terminal 14. With the above configuration, the necklace type terminal 14 outputs, from the speaker 40, a cooking suggestion for a recipe suited to the state of the user 20 identified from the biometric data of the user 20. In this way, the data processing system 10 can provide suggestions regarding cooking that is suitable for the condition of the user 20 identified from the biometric data of the user 20.

[0537] Furthermore, in the data processing system 10, when the stress level of the user 20 identified based on the biometric data is equal to or higher than a threshold, the processing unit 294 adds an instruction sentence to the first prompt to output a suggestion to simplify the cooking steps of the recipe. By inputting the first prompt into the data generation model 58, the necklace-type terminal 14 outputs a suggestion to simplify the cooking steps of the recipe from the speaker 40. Therefore, the data processing system 10 can suggest cooking steps that are suitable for the stress level of the user 20.

[0538] Furthermore, in the data processing system 10, when the blood pressure is equal to or higher than a threshold, the processing unit 294 adds an instruction to the first prompt to output recipe suggestions that will lead to a decrease in blood pressure. By inputting the first prompt into the data generation model 58, the necklace-type terminal 14 outputs recipe suggestions that will lead to a decrease in blood pressure from the speaker 40. Therefore, the data processing system 10 can suggest recipes that will improve the health of the user 20.

[0539] Furthermore, in the data processing system 10, when a recipe includes ingredients that are potentially harmful to one's health, the processing unit 294 inputs a second prompt including the user's 20's characteristics related to the ingredients into the data generation model 58, and uses the output of the data generation model 58 to obtain a suggestion to exclude the specific ingredients from the recipe. The output unit 296 outputs the suggestion to exclude the specific ingredients from the recipe obtained by the processing unit 294 to the necklace-type terminal 14. By inputting the second prompt into the data generation model 58, the necklace-type terminal 14 outputs a suggestion to exclude the specific ingredients from the recipe. Therefore, the data processing system 10 can reduce the risk of the user 20 consuming ingredients that may be harmful to their health.

[0540] In the twelfth embodiment, the biological data for determining whether a value is equal to or greater than a threshold value is blood pressure, but the biological data is not limited to blood pressure. For example, the biological data may be heart rate, blood sugar level, etc. instead of or in addition to blood pressure.

[0541] In the twelfth embodiment, when the stress level of the user 20 is equal to or higher than a threshold, the specific processing unit 290 generates a first prompt including an instruction to output a suggestion to simplify the cooking steps of a recipe selected by the user 20. However, the processing in this case is not limited to this. In this case, in addition to generating the first prompt, the specific processing unit 290 may also send an instruction to the necklace type terminal 14 to play music that helps the user 20 to relax. This makes it possible to create an environment in which cooking can be performed while supporting the mental health of the user 20.

[0542] In the twelfth embodiment, the identification processing unit 290 may adjust ingredients used in a recipe based on a comparison result between the calorie expenditure of the user 20 identified based on biometric data (e.g., heart rate) and the target calorie intake of the user 20 stored in the storage 32. For example, if the value obtained by subtracting the calorie expenditure from the target calorie intake of the user 20 is equal to or less than a predetermined value, the identification processing unit 290 may input a prompt including an instruction to output a suggestion to add a high-calorie ingredient to the recipe to the data generation model 58. Furthermore, if the value obtained by subtracting the calorie expenditure from the target calorie intake of the user 20 is equal to or greater than a specific value that is greater than the predetermined value, the identification processing unit 290 may input a prompt including an instruction to output a suggestion to add a low-calorie ingredient to the recipe to the data generation model 58.

[0543] In the twelfth embodiment, the identification processing unit 290 may adjust ingredients used in a recipe based on the daily biological data of the user 20 stored in the data storage unit 57. For example, if it is determined that the user 20 is suffering from mental fatigue based on the heart rate variability indicated by the daily biological data, the identification processing unit 290 may input a prompt including an instruction to the data generation model 58 to output a suggestion to add ingredients containing nutrients effective for recovering from fatigue to the recipe.

[0544] In the twelfth embodiment, the specific processing unit 290 may output the result of the suggestion process to a user terminal such as a smartphone or tablet carried by the user 20 in addition to the necklace-type terminal 14. This allows a description of the recipe selected by the user 20 to be displayed on the screen of the user terminal, allowing the user 20 to proceed with cooking while checking the status on the user terminal at hand.

[0545] In the twelfth embodiment, the specific processing unit 290 may store the biometric data of the user 20 collected during the cooking of the recipe in the data storage unit 57 and use it for personalization when proposing a recipe to the user 20 next time or later. Specifically, the specific processing unit 290 may fine-tune the data generation model 58 using the biometric data of the user 20 collected during the cooking of the recipe as learning data.

[0546] [Thirteenth Embodiment] Next, a thirteenth embodiment will be described, omitting or simplifying portions overlapping with the above-described embodiments. The thirteenth embodiment is characterized in that the data processing system 10 activates a cleaning function based on environmental data representing the living environment surrounding the wearer of the necklace-type terminal 14. In the thirteenth embodiment, a case will be described in which the air purifier 70 is activated based on environmental data detected indoors. The environmental data includes, for example, air quality data indicating the state of the indoor air, cleanliness data indicating the state of dirt (e.g., dust, dirt, etc.) accumulated on floors, and soiling data indicating the state of dirt adhesion on windows. The cleaning function may be any cleaning device or function that can be automatically executed, such as a robot vacuum cleaner, an automatic window cleaning robot, a ventilation system, or an automatic toilet flushing function.

[0547] (Overall Configuration) The data processing system 10 of the thirteenth embodiment includes an air purifier 70 as a cleaning function. As shown in FIG. 37 , the air purifier 70 is a so-called IoT (Internet of Things) device equipped with a communication function, and the necklace-type terminal 14 is connected to the air purifier 70 via a communication I / F 44. The necklace-type terminal 14 may be connected to the air purifier 70 via direct communication using short-range wireless communication or infrared communication, or may be connected via a network 54. The air purifier 70 is activated by receiving instructions transmitted from the necklace-type terminal 14. Although only one of each of the data processing device 12, necklace-type terminal 14, and air purifier 70 is illustrated in the present embodiment, multiple of each may be present.

[0548] (Hardware) The sensor 39 (see FIG. 1 ) included in the necklace-type terminal 14 of the thirteenth embodiment is one or more sensors for detecting environmental data representing the living environment. The sensor 39 of this embodiment is, for example, an air quality sensor, an optical sensor, or a fine dust detection sensor. The necklace-type terminal 14 of this embodiment uses the sensor 39 to measure the concentrations of PM2.5, PM10, ozone, nitrogen dioxide, sulfur dioxide, carbon monoxide, and the like in the atmosphere. The sensor 39 of this embodiment is, for example, disposed on the outside of the necklace-type terminal 14 so that air quality data around the wearer can be detected when the necklace-type terminal 14 is worn. The position of the sensor 39 may be changed as appropriate depending on the type of data to be detected.

[0549] 38 shows a schematic functional configuration of a control unit 46A of a necklace-type terminal 14 according to the 13th embodiment. The control unit 46A according to the 13th embodiment includes a data collection unit 100, a communication unit 102, and an operation unit 104.

[0550] The data collection unit 100 of the thirteenth embodiment has a function of collecting environmental data around the wearer of the necklace-type terminal 14. Specifically, the data collection unit 100 collects environmental data measured by the sensor 39 of the necklace-type terminal 14.

[0551] The communication unit 102 of the thirteenth embodiment has a function of communicating with a cleaning function around the wearer. The communication unit 102 communicates with, for example, an air purifier 70 present near the wearer and transmits instructions to the air purifier 70. Note that the vicinity of the wearer may be, for example, a predetermined range (for example, within 10 m horizontally) from the location of the wearer, or may be the range within the room where the wearer is present.

[0552] The communication unit 102 also has a function of communicating with the data processing device 12. Specifically, the communication unit 102 transmits the environmental data collected by the data collection unit 100 to the data processing device 12. Then, the communication unit 102 receives an instruction for a cleaning function generated by the data processing device 12 and corresponding to the transmitted environmental data.

[0553] The actuation unit 104 has a function of managing the actuation of the cleaning function around the wearer. Specifically, the actuation unit 104 transmits instructions for the cleaning function received from the data processing device 12 by the communication unit 102 to the cleaning function around the wearer. The actuation unit 104 transmits an actuation instruction to an air purifier 70 installed near the wearer, for example, to actuate the air purifier 70.

[0554] Next, the processing of the specific processing unit 290 according to the thirteenth embodiment will be described with reference to Fig. 6. The specific processing unit 290 according to the thirteenth embodiment includes an input unit 292, a processing unit 294, and an output unit 296.

[0555] The input unit 292 of the thirteenth embodiment stores the environmental data received from the necklace-type terminal 14 in the data storage unit 54. The environmental data stored in the data storage unit 54 is used, for example, to diagnose the air quality indicating the state of the atmosphere.

[0556] The processing unit 294 of the thirteenth embodiment performs identification processing using the data generation model 58. Specifically, a prompt including environmental data is input to the data generation model 58 to obtain a generation result. At this time, the prompt may include a behavioral pattern of the wearer.

[0557] The output unit 296 in the thirteenth embodiment transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A transmits the result of the specific processing to the air purifier 70. In this embodiment, an instruction corresponding to the environmental data detected by the sensor 39 is transmitted to the air purifier 70. Note that the output unit 296 may also transmit an instruction corresponding to the environmental data detected by the sensor 39 to the air purifier 70.

[0558] 39 is a sequence diagram showing an example of the flow of processing of the data processing system 10 according to the thirteenth embodiment. The data processing system 10 repeatedly executes the following processing, for example, after the wearer of the necklace-type terminal 14 returns home.

[0559] 39, the necklace-type terminal 14 detects air quality data. Specifically, the necklace-type terminal 14 detects the concentration of PM2.5 around the wearer using the sensor 39.

[0560] In step S11-13, the necklace-type terminal 14 collects air quality data. Specifically, the necklace-type terminal 14 collects data on the concentration of PM2.5 detected in step S10-13.

[0561] In step S12-13, the necklace-type terminal 14 transmits the air quality data, specifically, the PM2.5 concentration data collected in step S11-13, to the data processing device 12.

[0562] In step S13-13, the data processing device 12 executes a specific process, which will be described later. The data processing device 12 generates an operation instruction for the air purifier 70 through the specific process.

[0563] In step S14-13, the data processing device 12 transmits an operation instruction to the air purifier 70. Specifically, the data processing device 12 transmits the operation instruction to the air purifier 70 generated in step S13-13 to the necklace-type terminal 14.

[0564] In step S15-13, the necklace-type terminal 14 transmits an operation instruction to the air purifier 70. Specifically, the necklace-type terminal 14 transmits the operation instruction for the air purifier 70 transmitted to the necklace-type terminal 14 in step S14-13 to the air purifier 70. Then, the air purifier 70 activates the functions of the air purifier 70 based on the operation instruction transmitted from the necklace-type terminal 14.

[0565] Next, a description will be given of a specification process according to the thirteenth embodiment. Fig. 40 is a flowchart showing an example of the flow of the specification process according to the thirteenth embodiment. The specification process is, for example, the process executed in step S13 of Fig. 39.

[0566] In step S400-13 of FIG. 40, the specific processing unit 290 determines whether a predetermined trigger condition is met. Specifically, the specific processing unit 290 determines whether the air quality data received from the necklace-type terminal 14 exceeds a predetermined threshold as the trigger condition. For example, the specific processing unit 290 determines whether the PM2.5 concentration exceeds 15 μg / m3. The specific processing unit 290 may also determine whether a predetermined standard based on an index such as the AQI (Air Quality Index) is exceeded.

[0567] In step S400-13, if the specific processing unit 290 determines that the predetermined trigger condition is satisfied (step S400-13: YES), the process proceeds to step S401-13. On the other hand, if the specific processing unit 290 determines that the predetermined trigger condition is not satisfied (step S400-13: NO), the specific processing ends. Note that if the specific processing ends because it determines that the predetermined trigger condition is not satisfied, no instruction is generated for the air purifier 70.

[0568] In step S401-13, the identification processing unit 290 generates a prompt by adding an instruction for obtaining the result of the identification processing to the text indicating the received air quality data.

[0569] The specific processing unit 290 generates a prompt such as, "The current indoor PM2.5 concentration is 25 μg / m 3 . Please generate a signal to be sent to the air purifier based on this concentration."

[0570] In step S403-13, the identification processing unit 290 acquires the result of the identification process using the data generation model 58. Specifically, the identification processing unit 290 inputs the prompt generated in step S402-13 to the data generation model 58, and acquires the result of the identification process based on the output of the data generation model 58.

[0571] The specific processing unit 290 acquires, for example, a signal that states, "1. Set the air purifier mode to high power mode. 2. Set the fan speed to maximum. 3. Set the operating time to at least two hours."

[0572] In step S404-13, the identification processing unit 290 outputs the result of the identification processing. Specifically, the identification processing unit 290 transmits the result of the identification processing acquired in step S403-13 to the necklace-type terminal 14, and ends the identification processing.

[0573] (Summary of the thirteenth embodiment) The necklace-type terminal 14 of the thirteenth embodiment includes a sensor 39 that detects air quality data, collects air quality data around the wearer of the necklace-type terminal 14, and operates an air purifier 70 near the wearer based on the collected air quality data. Therefore, the necklace-type terminal 14 of this embodiment can improve the living environment around the user based on environmental data collected from the user's surrounding environment. Furthermore, the necklace-type terminal 14 of this embodiment can eliminate the need for the wearer to operate the air purifier 70 themselves, thereby improving convenience.

[0574] The necklace-type terminal 14 of the thirteenth embodiment transmits the collected air quality data to the data processing device 12, and operates the air purifier 70 according to instructions received from the data processing device 12 that correspond to the transmitted air quality data. Therefore, according to the necklace-type terminal 14 of this embodiment, by using the data processing device 12, the processing load on the necklace-type terminal 14 can be reduced, and the power consumption of the necklace-type terminal 14 can be reduced.

[0575] The data processing system 10 of the thirteenth embodiment includes a necklace-type terminal 14 and a data processing device 12. The data processing device 12 acquires instructions for the air purifier 70 by inputting a prompt including air quality data received from the necklace-type terminal 14 into the data generation model 58, and transmits the instructions to the necklace-type terminal 14. Therefore, the data processing system 10 of this embodiment can utilize the storage capacity and processing power of the data processing device 12 to quickly and accurately perform evaluations based on air quality data using the data generation model 58. Furthermore, the data processing system 10 of this embodiment can accurately issue instructions to the air purifier 70 using the data generation model 58.

[0576] The data processing system 10 of the thirteenth embodiment acquires instructions for the air purifier 70 on the condition that the air quality data exceeds a predetermined standard. Therefore, the data processing system 10 of this embodiment can suppress the issuance of instructions for the air purifier 70 and operate the air purifier 70 effectively.

[0577] The thirteenth embodiment can be modified in various ways as described below.

[0578] In the thirteenth embodiment, the necklace-type terminal 14 measured environmental data using the sensor 39. However, this is not limiting, and the necklace-type terminal 14 may measure environmental data using the camera 42. For example, the necklace-type terminal 14 may detect floor cleanliness data, window stain data, etc. from image information captured by the camera 42. Therefore, according to the necklace-type terminal 14 of this embodiment, dirt that can be detected from image information can be automatically cleaned.

[0579] The necklace-type terminal 14 in the thirteenth embodiment transmits instructions received from the data processing device 12 to the air purifier 70. However, this is not limiting, and instructions generated by the necklace-type terminal 14 may be transmitted to the air purifier 70. For example, when the PM2.5 concentration measured by the sensor 39 exceeds a predetermined standard, the necklace-type terminal 14 transmits a signal to the air purifier 70 near the wearer to instruct it to operate, thereby activating the air purifier 70. Therefore, the necklace-type terminal 14 of this embodiment can reduce costs such as equipment costs and communication costs compared to when communicating with the data processing device 12.

[0580] In the thirteenth embodiment, the identification processing unit 290 of the data processing device 12 inputs a prompt including environmental data to the data generation model 58. However, this is not limited to this, and the identification processing unit 290 may input a prompt including behavioral data of the wearer to the data generation model 58. Here, the behavioral data is data including the current location and past behavioral patterns of the wearer. For example, the identification processing unit 290 generates a prompt such as, "The current indoor PM2.5 concentration is 25 μg / m3. Furthermore, the user returns home around 7:00 PM on weekdays and often spends the weekday hours from 7:00 PM to 10:00 PM in the living room. Based on these concentrations and the schedule, please generate a signal to be sent to the air purifier." The specific processing unit 290 then acquires a signal such as, for example, "1. Set the air purifier to high power mode. 2. Operate in high power mode from 7 PM to 10 PM. 3. Set the fan speed to medium after 10 PM. 4. Reduce the fan speed when the PM2.5 concentration drops to 15 μg / m3 or less, and switch to automatic mode when it drops below 10 μg / m3." Therefore, according to the data processing device 12 of this embodiment, it is possible to operate the air purifier 70 in accordance with the lifestyle habits of the wearer, thereby further improving the convenience and comfort of the wearer.

[0581] [Fourteenth Embodiment] Next, a fourteenth embodiment will be described, omitting or simplifying portions overlapping with the above-described embodiments. The fourteenth embodiment is characterized in that the data processing system 10 activates a cleaning mechanism that cleans a predetermined position on the necklace-type terminal 14 based on behavioral data representing the behavior of the wearer of the necklace-type terminal 14. Here, the behavioral data includes data on the wearer's movements and posture, and may be categorized into daily activities such as walking, sitting, and standing, exercise such as running and strength training, and recreational activities such as camping and pet care. The behavioral data may also include a schedule including the wearer's commute to work, school, and home, and data indicating the wearer's location information measured using a GPS (Global Positioning System). The cleaning mechanism may be, for example, an air duster device, a vibration device, a brushing device, a wiper device, or a dust collection device, and cleans the surface of the necklace-type terminal 14, the sensor 39, the camera 42, and the like.

[0582] (Overall Configuration) As shown in Fig. 41 , the necklace-type terminal 14 of the fourteenth embodiment includes a cleaning mechanism 70. The cleaning mechanism 70 may be configured to be built into the necklace-type terminal 14, or may be configured to be attached externally to the necklace-type terminal 14. Note that, although only one of each of the data processing device 12, necklace-type terminal 14, and cleaning mechanism 70 is shown in the figure in this embodiment, there may be multiple of each.

[0583] The cleaning mechanism 70 of this embodiment includes an air duster device 71 and a vibration device 72. The air duster device 71 includes an electric motor, a fan, and a nozzle (not shown). Specifically, the air duster device 71 rotates the fan using the electric motor, and the generated airflow is sprayed from a nozzle provided at a predetermined position on the main body of the necklace-type terminal 14, thereby removing dust, dirt, and the like adhering to the surface of the necklace-type terminal 14 and the sensor 39, etc. The nozzle is provided, for example, around the sensor 39 and on the outer edge of the main body of the necklace-type terminal 14, and is installed in a direction that allows air to be sprayed onto the surface of the main body of the necklace-type terminal 14, the surface of the sensor 39, etc. Note that the air duster device 71 is not limited to an electric type, and may also be a gas type.

[0584] The vibration device 72 is a vibration device that generates minute vibrations and is incorporated into the necklace-type terminal 14, and the vibration removes fine dust that has adhered to the surface of the main body of the necklace-type terminal 14, the sensor 39, and the camera 42. The vibration device 72 may be either a device that uses an electric motor or a device that uses a piezoelectric element.

[0585] (Hardware) The sensor 39 (see FIG. 1) provided in the necklace type terminal 14 of the fourteenth embodiment is one or more sensors for detecting behavioral data representing the behavior of the wearer. The sensor 39 in this embodiment is, for example, an acceleration sensor, a gyro sensor, or the like. The necklace type terminal 14 of this embodiment measures the movement and posture of the wearer using the sensor 39. The sensor 39 in this embodiment is disposed, for example, inside the necklace type terminal 14. The position of the sensor 39 may be changed as appropriate depending on the type of data to be detected.

[0586] 42 shows a schematic functional configuration of a control unit 46A of a necklace-type terminal 14 according to the fourteenth embodiment. The control unit 46A of the fourteenth embodiment includes a data collection unit 100, a communication unit 102, and an operation unit 104.

[0587] The data collection unit 100 of the fourteenth embodiment has a function of collecting behavioral data of the wearer of the necklace-type terminal 14. Specifically, the data collection unit 100 collects behavioral data measured by the sensor 39 of the necklace-type terminal 14. The data collection unit 100 also collects data on the wearer's schedule and location information.

[0588] The communication unit 102 according to the fourteenth embodiment has a function of communicating with the data processing device 12. Specifically, the communication unit 102 transmits the behavioral data collected by the data collection unit 100 to the data processing device 12. Then, the communication unit 102 receives an instruction to the cleaning mechanism 70 generated by the data processing device 12 and corresponding to the transmitted behavioral data.

[0589] The operation unit 104 has a function of managing the operation of the cleaning mechanism 70. Specifically, the operation unit 104 operates the cleaning mechanism 70 based on an instruction to the cleaning mechanism 70 from the data processing device 12 received by the communication unit 102. The operation unit 104 operates, for example, the air duster device 71.

[0590] Next, the processing of the specific processing unit 290 according to the fourteenth embodiment will be described with reference to Fig. 6. The specific processing unit 290 according to the fourteenth embodiment includes an input unit 292, a processing unit 294, and an output unit 296.

[0591] The input unit 292 of the fourteenth embodiment stores the behavioral data received from the necklace-type terminal 14 in the data storage unit 54. The behavioral data stored in the data storage unit 54 is used, for example, to diagnose the movement of the wearer.

[0592] The processing unit 294 of the fourteenth embodiment performs a specification process using the data generation model 58. Specifically, a prompt including the movement of the wearer represented by the behavioral data is input to the data generation model 58, and a generation result is obtained.

[0593] The output unit 296 in the fourteenth embodiment transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A transmits the result of the specific processing to the cleaning mechanism 70. In this way, instructions corresponding to the behavioral data detected by the sensor 39 are transmitted to the cleaning mechanism 70.

[0594] 43 is a sequence diagram showing an example of the processing flow of the data processing system 10 according to the fourteenth embodiment. For example, the data processing system 10 repeatedly executes the following processing when the wearer is wearing the necklace-type terminal 14.

[0595] 43, the necklace-type terminal 14 detects the movement of the wearer. Specifically, the necklace-type terminal 14 detects the speed, direction, and posture changes of the wearer using the sensor 39.

[0596] In step S11-14, the necklace-type terminal 14 collects data on the movement of the wearer. Specifically, the necklace-type terminal 14 collects data on the speed, direction, and posture changes of the wearer detected in step S10-14.

[0597] In step S12-14, the necklace-type terminal 14 transmits the movement data. Specifically, the necklace-type terminal 14 transmits the data collected in step S11-14, such as the movement speed, direction of movement, and changes in posture of the wearer, to the data processing device 12.

[0598] In step S13-14, the data processing device 12 executes a specific process, which will be described later, to generate an operation instruction for the cleaning mechanism 70 through the specific process.

[0599] In step S14-14, the data processing device 12 transmits an operation instruction to the cleaning mechanism 70. Specifically, the data processing device 12 transmits the operation instruction to the cleaning mechanism 70 generated in step S13-14 to the necklace-type terminal 14.

[0600] In step S15-14, the necklace-type terminal 14 transmits an operation instruction to the cleaning mechanism 70. Specifically, the necklace-type terminal 14 activates the air duster device 71 of the cleaning mechanism 70 based on the operation instruction to the cleaning mechanism 70 transmitted to the necklace-type terminal 14 in step S14-14.

[0601] Next, the identification process according to the fourteenth embodiment will be described. Fig. 44 is a flowchart showing an example of the flow of the identification process according to the fourteenth embodiment. The identification process is, for example, the process executed in step S13-14 of Fig. 43.

[0602] 44, the specific processing unit 290 determines whether a predetermined trigger condition is satisfied. Specifically, the specific processing unit 290 determines whether the behavior data received from the necklace-type terminal 14 includes data indicating a predetermined movement of the wearer as the trigger condition. For example, the specific processing unit 290 determines whether the wearer has just finished running.

[0603] In step S400, if the specific processing unit 290 determines that the predetermined trigger condition is satisfied (step S400-14: YES), the process proceeds to step S401. On the other hand, if the specific processing unit 290 determines that the predetermined trigger condition is not satisfied (step S400: NO), the specific processing ends. Note that if the specific processing ends because it determines that the predetermined trigger condition is not satisfied, no instruction is generated for the cleaning mechanism 70.

[0604] In step S401, the identification processing unit 290 generates a prompt. Specifically, the identification processing unit 290 generates the prompt by adding an instruction sentence for obtaining the result of the identification processing to text indicating the movement represented by the received behavioral data.

[0605] The specific processing unit 290 generates a prompt such as, "After the wearer finishes running, please generate a specific signal to activate the air duster device."

[0606] In step S403-14, the identification processing unit 290 acquires the result of the identification process using the data generation model 58. Specifically, the identification processing unit 290 inputs the prompt generated in step S402-14 to the data generation model 58, and acquires the result of the identification process based on the output of the data generation model 58.

[0607] The specific processing unit 290 receives a signal such as, for example, "1. Start the air duster device and blow air for 5 seconds. 2. Set the air blow strength to medium. 3. Stop the air duster device after cleaning is complete."

[0608] In step S404-14, the identification processing unit 290 outputs the result of the identification processing. Specifically, the identification processing unit 290 transmits the result of the identification processing acquired in step S403-14 to the necklace-type terminal 14, and ends the identification processing.

[0609] (Summary of the Fourteenth Embodiment) The necklace-type terminal 14 of the fourteenth embodiment is equipped with a cleaning mechanism 70 that cleans a predetermined position on the terminal itself, collects behavioral data of the wearer of the necklace-type terminal 14, and activates the cleaning mechanism 70 based on the collected behavioral data. Therefore, according to the necklace-type terminal 14 of this embodiment, the functionality of the necklace-type terminal 14 can be maintained by automatically cleaning the necklace-type terminal 14.

[0610] The necklace-type terminal 14 of the fourteenth embodiment transmits collected behavioral data to the data processing device 12, and operates the cleaning mechanism 70 in response to instructions received from the data processing device 12 that correspond to the transmitted behavioral data. Therefore, according to the necklace-type terminal 14 of this embodiment, by using the data processing device 12, the processing load on the necklace-type terminal 14 can be reduced, and power consumption of the necklace-type terminal 14 can be reduced.

[0611] The data processing system 10 of the fourteenth embodiment includes a necklace-type terminal 14 and a data processing device 12, and the data processing device 12 acquires instructions for the cleaning mechanism 70 by inputting a prompt including behavioral data received from the necklace-type terminal 14 into the data generation model 58, and transmits the instructions to the necklace-type terminal 14. Therefore, the necklace-type terminal 14 of this embodiment can achieve more effective cleaning than cleaning based on instructions determined on a rule basis.

[0612] The data processing system 10 of the fourteenth embodiment acquires instructions for the cleaning mechanism 70 on the condition that the behavioral data is data indicating that a predetermined behavior has been completed. Therefore, the data processing system 10 of this embodiment can suppress the generation of instructions for the cleaning mechanism 70 and operate the cleaning mechanism 70 effectively.

[0613] The fourteenth embodiment can be modified in various ways as follows, each of which will be described below.

[0614] In the fourteenth embodiment, the specific processing unit 290 of the data processing device 12 inputs a prompt indicating that the user has finished running to the data generation model 58. However, this is not limited to this. The specific processing unit 290 may input a prompt indicating the wearer's schedule to the data generation model 58. For example, the specific processing unit 290 generates a prompt such as, "The user usually returns home around 7:00 PM on weekdays and takes a bath around 8:00 PM on weekdays. Based on this schedule, please generate a signal to be sent to the vibration device of the cleaning mechanism." The specific processing unit...

Claims

a necklace-type terminal including a microphone that picks up user utterances of a wearer, a collection unit that collects an output of the microphone, and a communication unit that transmits the output of the microphone collected by the collection unit to an external device; a data processing system comprising: The data processing device includes: an input unit that receives a user utterance picked up by the microphone; a processing unit that, when it is determined that there is a sign of dementia based on the user utterance, notifies a predetermined notification destination that the wearer may have dementia, Data processing system. 　 When there is no consistency between a plurality of utterances made by the wearer during the user's speech, the processing unit determines that there is a sign of dementia, and notifies a preset notification destination that the wearer may have dementia.

10. The data processing system of claim 1. 　 the processing unit determines whether or not there is a sign of dementia based on the frequency of appearance of a specific word in the user's utterance; 10. The data processing system of claim 1.

4. The data processing system of claim 3, wherein the particular word is a pronoun. 　 The processing unit counts the number of times that it is determined that there are signs of dementia based on the user utterances, and when the count value within a predetermined period of time becomes equal to or greater than a threshold value, notifies a predetermined notification destination that the wearer may have dementia.

10. The data processing system of claim 1. 　 the necklace-type terminal further includes a speaker; The processing unit outputs a prepared question as sound from the speaker, determines whether the wearer has a sign of dementia based on the answer to the question picked up by the microphone, and notifies a predetermined notification destination that the wearer may have dementia if the determined possibility is equal to or greater than a predetermined value.

10. The data processing system of claim 1. 　 The processing unit determines whether or not there are signs of dementia based on a pattern of changes in the wearer's emotions estimated from the user's utterance, and if it is determined that there are signs of dementia, notifies a preset notification destination that the wearer may have dementia.

10. The data processing system of claim 1. 　 the necklace-type terminal further includes a heart rate sensor for detecting the heart rate of the wearer, the processing unit determines the type of dementia by performing frequency analysis of heart rate variability in the heart rate data of the wearer detected by the heart rate sensor, and when notifying the wearer that the wearer may have dementia, notifies the wearer of information about the determined type of dementia.

8. A data processing system according to any one of claims 1 to 7. 　 a necklace-type terminal including a microphone that picks up user utterances of a wearer, a sensor that detects biometric data of the wearer, a collection unit that collects outputs from the microphone and the sensor, and a communication unit that transmits the outputs from the microphone and the sensor collected by the collection unit to an external device; a data processing system comprising: The data processing device includes: an input unit that receives a user utterance picked up by the microphone; a processing unit that determines whether the wearer has a sign of heart disease based on a change pattern of emotion of the wearer estimated from the user's utterance and the biological data, and when it is determined that the wearer has a sign of heart disease, notifies a preset notification destination that the wearer has a sign of heart disease, Data processing system. 　 a sensor for detecting biometric data of the wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into voice data; a collection unit that collects outputs from the sensors and the microphones; a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device; and a data processing system comprising: The data processing device includes: an input unit that receives the biological data collected by the sensor and the audio data collected by the microphone; a processing unit that acquires medically important information from the voice data based on the amplitude of the wearer's emotion estimated from the biometric data; an output unit that outputs the important information to a preset output destination, Data processing system. 　 a sensor for detecting biometric data of a wearer who is a medical professional; a microphone that picks up the conversation between the wearer and the patient and converts it into voice data; a collection unit that collects outputs from the sensors and the microphones; a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device; and a data processing system comprising: The data processing device includes: an input unit that receives the biological data collected by the sensor and the audio data collected by the microphone; a processing unit that acquires medically important information from the voice data based on changes in the wearer's emotions estimated from the biometric data, creates a summary that summarizes the content of the dialogue so as to preferentially include the important information, and records the summary in a recording unit; an output unit that outputs the summary to a predetermined output destination, Data processing system. 　 a sensor for detecting biometric data of the wearer; a microphone that picks up a conversation between the wearer and a medical professional and converts it into voice data; a collection unit that collects outputs from the sensors and the microphones; a communication unit that transmits the outputs of the sensor and the microphone collected by the collection unit to an external device; and a data processing device; and a data processing system comprising: The data processing device includes: an input unit that receives the biological data collected by the sensor and the audio data collected by the microphone; a processing unit that acquires and infers medically important information from the voice data based on a change in the wearer's emotion estimated from at least one of the biometric data and the voice data, and acquires the inference result as diagnostic support information; an output unit that outputs the diagnostic assistance information acquired by the processing unit to a preset output destination, Data processing system. 　 an emotion recognition unit that recognizes an emotional state of the customer based on voice data including a speech voice of the customer and image data including a facial expression of the customer; a stress level deriving unit that derives a stress level of the sales representative based on biometric data of the sales representative; a feedback generation unit that generates effective feedback for successfully conducting sales activities with the customer using a data generation model based on the recognized emotional state of the customer and the derived stress level of the salesperson; and 2. A data processing device comprising: 　 an input unit that acquires biometric data transmitted from a necklace-type terminal worn by a user; a processing unit that inputs the biometric data and a prompt that instructs generating an action plan for health management of the user based on the biometric data into a data generation model, and acquires the action plan output from the data generation model; an output unit that outputs the action plan to the necklace type terminal; 2. A data processing device comprising: 　 A data processing system comprising: a wearable device having a sensor for detecting biometric data of a wearer and surrounding environmental data; and a data processing device for receiving output data from the sensor and a prompt requesting a response regarding the wearer's health condition based on the output data, The data processing device includes: a proactive intervention processing unit that analyzes output data from the sensor, predicts a risk score related to the wearer's health condition, and sets alert data according to the degree of risk; a providing unit that provides the alert data set by the proactive intervention processing unit to the wearer via the wearable device in response to the prompt; A data processing system having: 　 a teacher terminal worn by the teacher; a student terminal attached to each of a plurality of students; a data processing device communicably connected to the teacher terminal and the student terminal, The data processing device comprises a processor; The processor: collecting data output from at least one of a microphone, a sensor, and a camera provided on the student terminal; analyzing the data to estimate a state of the plurality of students; providing feedback to the teacher terminal based on the estimated result; Data processing system. 　 A sensor that detects biometric data of the student who wears it; A camera that captures the facial expressions and movements of the students; A microphone and a data collection unit that collects outputs from the sensors, the camera, and the microphone; A provision unit that provides a learning plan generated by a generative AI based on the output collected by the data collection unit; A necklace-type terminal including: 　 A camera that captures the wearer's surroundings, a sensor for detecting biological data of the wearer; microphone, a collection unit that collects the outputs of the camera, the sensor, and the microphone; a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the collection unit to a data processing device; and a speaker that outputs a response corresponding to the user's utterance picked up by the microphone; a necklace-type terminal including the an input unit for acquiring the user utterance; an acquisition unit that acquires, from a recipe database, recipe data corresponding to a recipe selected by the wearer indicated by the user utterance; a first output unit that outputs the recipe data acquired by the acquisition unit to the necklace-type terminal; a processing unit that, when the state of the wearer identified based on the biometric data transmitted from the necklace-type device satisfies a predetermined condition, inputs a first prompt including the biometric data into a data generation model, and obtains a cooking suggestion for the recipe using an output of the data generation model; and a second output unit that outputs suggestions regarding cooking the recipe acquired by the processing unit to the necklace-type terminal; the data processing device including: A data processing system comprising: 　 A necklace-type terminal equipped with a sensor that detects environmental data representing a living environment, a data collection unit that collects the environmental data around the wearer of the necklace type terminal detected by the sensor; an activation unit that activates a cleaning function around the wearer based on the collected environmental data; A necklace-type terminal equipped with the above. 　 A necklace-type terminal equipped with a cleaning mechanism that cleans a predetermined position of the terminal itself, a data collection unit that collects behavioral data of the wearer of the necklace-type terminal; an operating unit that operates the cleaning mechanism based on the collected behavioral data; A necklace-type terminal equipped with the above. 　 A necklace-type terminal formed with a self-repairing material, a data collection unit that collects damage data representing damage to the necklace-type terminal; an operating unit that activates a self-repair mechanism that acts on the self-repair material based on the collected damage data; A necklace-type terminal equipped with the above. 　 a collection unit that collects outputs from at least one of a camera that captures an image of the wearer's surroundings, a sensor that detects biometric data of the wearer, and a microphone; an execution unit that uses the collected output to execute a convenience process to improve convenience when the wearer is traveling; A necklace-type terminal including: an acceleration sensor that detects the wearer's movements; a skin electrodermal response sensor for detecting biological data of the wearer; an environmental sensor for acquiring external environmental data of the wearer; a current location acquisition module that acquires the current location of the wearer; a collection unit that collects data from the acceleration sensor, the electrodermal response sensor, the environmental sensor, and the current position acquisition module while the wearer is training; a control unit that acquires an analysis result of the data collected by the collection unit and provides feedback regarding the training to the wearer during the training; A necklace-type terminal equipped with the above. 　 a collection unit that collects outputs from an environmental sensor that detects environmental data around the wearer and a biometric sensor that detects biometric data of the wearer; an execution unit that executes an environment improvement process including a process of presenting proposal information for dynamically improving the environment around the wearer using the collected output; A necklace-type terminal including: 　 a collection unit that collects an output of a microphone that picks up the wearer's speech; an execution unit that executes a health problem prevention process that uses the collected output to present proposal information for preventing the occurrence of health problems in at least one of mental health and physical health of the wearer; and A necklace-type terminal including: 　 a data collection unit that collects biometric data of the wearer and environmental data including the temperature and humidity of the wearer's surroundings; a processing unit that performs specific processing using the biological data, the environmental data, and a specific algorithm that evaluates the wearer's comfort level in an air-conditioned environment; a communication unit that transmits a result of the identification process to an air conditioning device that conditions air in an indoor space where the wearer is present; Equipped with the processing unit performs, as the identification process, a process of calculating an air conditioning setting taking into consideration the physical condition of the wearer, the temperature, and the humidity; The communication unit transmits a control signal corresponding to the air conditioning setting to the air conditioning apparatus as a result of the identification process. 　 a data collection unit that collects biometric data of the wearer and environmental data including the temperature, amount of light, and color tone of the surroundings of the wearer; a processing unit that performs specific processing using the biological data, the environmental data, and a specific algorithm that evaluates the wearer's comfort level with respect to the surrounding environment, including the air conditioning and brightness around the wearer; a communication unit that transmits a result of the identification process to an air conditioning device that conditions air in an indoor space where the wearer is present and to an illumination device that illuminates the indoor space; Equipped with the processing unit performs, as the identification process, a process of calculating air conditioning settings and lighting settings taking into consideration the physical condition of the wearer, the temperature, the light amount, and the color tone; The communication unit transmits control signals corresponding to the air conditioning setting and the lighting setting to the air conditioning device and the lighting device as a result of the identification process. 　 a terminal worn by a user; a data processing device communicably connected to the terminal, The data processing device comprises a processor; The processor: Acquire an image of the user's meal taken by a camera provided on the terminal; analyzing the image to generate meal data indicative of details of the meal; Acquire biometric data of the user detected by a sensor provided in the terminal; Integrating and analyzing the dietary data and the biological data; outputting information according to the results of the analysis; Data processing system. 　 A camera that captures the work environment of the wearer, an engineer, and a sensor for detecting biometric data of the engineer; A microphone and A virtual display and a processor; The processor: collecting the output of each of the camera, the sensor, and the microphone; transmitting the collected outputs of each of the camera, the sensor, and the microphone to a data processing device; displaying the information received from the data processing device on the virtual display; Necklace-type device. 　 a necklace-type terminal to be worn by a wearer and having a jewelry part, the necklace-type terminal including a sensor and an output part; a data processor that determines feedback to the jewelry according to the data obtained by the sensor; Equipped with The necklace-type terminal outputs information about the feedback to the wearer through the output unit in accordance with the feedback, an interactive jewelry feedback system. 　 A camera that captures the wearer's surroundings, a sensor for detecting biological data of the wearer; microphone, a collection unit that collects the outputs of the camera, the sensor, and the microphone; a communication unit that transmits the outputs of the camera, the sensor, and the microphone collected by the collection unit to a data processing device; a necklace-type terminal including a speaker that outputs a response corresponding to a user utterance picked up by the microphone; an input unit for acquiring the biometric data and user learning data; an analysis unit that analyzes the user's condition indicated by the biometric data; a processing unit that inputs a prompt including the user state and the user learning data into a data generation model, and obtains a learning plan corresponding to the user state and the user learning data using an output of the data generation model; the data processing device including an output unit that outputs suggestions based on the study plan to the necklace-type terminal; A data processing system comprising: 　 a collection unit that collects outputs of sensors that detect biological data of animals present around the wearer; and a communication unit that transmits the biological data of the animal collected by the collection unit to a data processing device. a necklace-type terminal including the a processing unit that inputs a prompt to a data generation model, the prompt instructing the estimation of information regarding the health condition of the animal based on the biological data of the animal received from the necklace-type terminal, and obtains information regarding the health condition of the animal using an output of the data generation model; and an output unit that outputs information regarding the health condition of the animal. a data processing device including: A data processing system comprising: 　 A necklace-type terminal configured to be worn by a mobile living body and to collect data in an area where a plant is grown, a biosensor device configured to sense biometric data of the living body; an environmental sensor device configured to detect environmental data outside the necklace-type terminal; an acoustic collector configured to detect acoustic vibrations including sound waves from outside the necklace-type terminal and sound waves emitted by the living body; a data collector configured to collect data signals from the biosensor device, the environmental sensor device, and the acoustic collector to generate respective output data; a first communication unit configured to receive the output data from the data collection unit and capable of communicating with a data processing device disposed on the outside of the necklace-type terminal; an electroacoustic conversion device that converts the electrical signal from the first communication unit into acoustic vibrations; Equipped with the environmental sensor device includes a light sensor device that detects light at the position of the necklace-type terminal; Necklace-type device.

Images