Data Processing System

Abstract

This disclosure aims to provide suggestions regarding cooking methods that are suitable for the wearer's condition, as identified from the wearer's biometric data while wearing a necklace-type device. [Solution] The data processing system includes a processing unit that, when the wearer's condition identified based on biometric data transmitted from a necklace-type terminal satisfies predetermined conditions, inputs a first prompt including biometric data into a data generation model and uses the output of the data generation model to obtain suggestions regarding recipe preparation.

Description

Technical Field

[0001] The technology of the present disclosure relates to a data processing system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, with the demand for shortening cooking time and improving efficiency in cooking, wearable devices are being used in various aspects of daily life. However, conventional wearable devices for cooking assistance only provide simple timing instructions and recipe suggestions, and there is room for improvement in providing cooking assistance that meets various needs of users.

[0005] Therefore, an object of the present disclosure is to provide a data processing system that can make a proposal regarding cooking suitable for the state of a wearer specified from biometric data of the wearer wearing a necklace-type terminal.

Means for Solving the Problems

[0006] A data processing system according to a first embodiment of the technology of this disclosure includes a necklace-type terminal including a camera for photographing the area around the wearer, a sensor for detecting the wearer's biometric data, a microphone, a collection unit for collecting the outputs of the camera, the sensor, and the microphone, a communication unit for transmitting the outputs of the camera, the sensor, and the microphone collected by the collection unit to a data processing device, and a speaker for outputting a response corresponding to user utterances picked up by the microphone; and a data processing device including an input unit for acquiring user utterances, an acquisition unit for acquiring recipe data from a recipe database corresponding to a recipe selected by the wearer as indicated by the user utterances, a first output unit for outputting the recipe data acquired by the acquisition unit to the necklace-type terminal, a processing unit that, when the wearer's state identified based on the biometric data transmitted from the necklace-type terminal satisfies predetermined conditions, inputs a first prompt including the biometric data to a data generation model and uses the output of the data generation model to acquire a cooking suggestion for the recipe, and a second output unit for outputting the cooking suggestion for the recipe acquired by the processing unit to the necklace-type terminal.

[0007] In the first embodiment of the data processing system, the input unit acquires user utterances picked up by a microphone. The acquisition unit acquires recipe data from a recipe database corresponding to the recipe selected by the wearer as indicated by the user utterance. The first output unit outputs the recipe data acquired by the acquisition unit to the necklace-type terminal. If the wearer's state, identified based on biometric data transmitted from the necklace-type terminal, satisfies predetermined conditions, the processing unit inputs a first prompt including biometric data to the data generation model and uses the output of the data generation model to acquire a cooking suggestion for the recipe. The second output unit outputs the cooking suggestion for the recipe acquired by the processing unit to the necklace-type terminal. With this configuration, the necklace-type terminal outputs a cooking suggestion for a recipe suitable for the wearer's state, identified from the wearer's biometric data, through its speaker. Thus, this data processing system can provide a cooking suggestion suitable for the wearer's state, identified from the wearer's biometric data.

[0008] A second embodiment of the data processing system relating to the technology of the present disclosure is a data processing system of the first embodiment, wherein the processing unit adds an instruction to the first prompt to output a suggestion to simplify the cooking procedure of the recipe if the stress level of the wearer identified based on the biometric data is above a threshold.

[0009] In the second embodiment of the data processing system, the processing unit adds an instruction to the first prompt to output a suggestion to simplify the recipe's cooking procedure if the wearer's stress level, identified based on biometric data, is above a threshold. When this first prompt is input to the data generation model, the necklace-type terminal outputs a suggestion to simplify the recipe's cooking procedure from its speaker. Therefore, according to this data processing system, it is possible to suggest a cooking procedure that is appropriate to the wearer's stress level.

[0010] A third embodiment of the data processing system relating to the technology of this disclosure is a data processing system of the first or second embodiment, wherein the processing unit adds an instruction statement to the first prompt for outputting a suggestion of cooking content that would lead to a decrease in the value of the biological data if the value of the biological data is above a threshold.

[0011] In the third embodiment of the data processing system, the processing unit adds an instruction to the first prompt to output a suggestion for cooking methods that would lead to a decrease in the value of the biometric data if the biometric data value is above a threshold. When this first prompt is input to the data generation model, the necklace-type terminal outputs a suggestion for cooking methods that would lead to a decrease in the value of the biometric data from its speaker. Therefore, according to this data processing system, it is possible to suggest cooking methods that would improve the wearer's health condition.

[0012] A fourth embodiment of the data processing system relating to the technology of the present disclosure is a data processing system of the first, second, or third embodiment, wherein the processing unit, when the ingredients used in the recipe identified based on an image of the wearer's surroundings transmitted from the necklace-type terminal include a specific ingredient for which the wearer has at least one of a food allergy and / or food intolerance registered in a characteristics database indicating the wearer's characteristics regarding food, inputs a second prompt including the wearer's characteristics regarding food to the data generation model, obtains a suggestion to remove the specific ingredient from the recipe using the output of the data generation model, and outputs the suggestion to remove the specific ingredient from the recipe obtained by the processing unit to the necklace-type terminal.

[0013] In the fourth embodiment of the data processing system, if a specific ingredient is included in the ingredients used in a recipe, the processing unit inputs a second prompt, including the wearer's characteristics regarding ingredients, into the data generation model, and uses the output of the data generation model to obtain a suggestion to remove the specific ingredient from the recipe. The second output unit outputs the suggestion to remove the specific ingredient from the recipe obtained by the processing unit to the necklace-type terminal. As a result of the input of the second prompt into the data generation model, the necklace-type terminal outputs a suggestion to remove the specific ingredient from the recipe. Therefore, according to this data processing system, the risk of the wearer ingesting ingredients that may be harmful to their health can be reduced. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system. [Figure 2] This is a conceptual diagram showing an example of the main functions of a data processing device and a necklace-type terminal. [Figure 3] This is a side view showing the configuration of a necklace-type terminal. [Figure 4] This is a top view showing the configuration of a necklace-type terminal. [Figure 5] The functional configuration of the control unit of the necklace-type terminal is shown in general terms. [Figure 6] The functional configuration of the specific processing unit of the data processing device according to the first embodiment is schematically shown. [Figure 7] This diagram outlines an example of the operation flow of a specific process performed by a data processing device. [Figure 8] The functional configuration of the specific processing unit of the data processing device according to the second embodiment is schematically shown. [Figure 9] The storage configuration of the data processing unit is shown in general terms. [Figure 10] A schematic example of the first operational flow of the proposed processing by the data processing device is shown below. [Figure 11] A schematic example of the second operational flow of the proposed processing by the data processing device is shown below. [Modes for carrying out the invention]

[0015] Hereinafter, an example of an embodiment of a data processing apparatus, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a signed storage is one or more non-volatile storage devices for storing various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] (First embodiment) First, a first embodiment of the data processing system 10 according to the embodiment will be described.

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the embodiment. As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of this disclosure, and the necklace-type terminal 14 is an example of a "necklace-type terminal" according to the technology of this disclosure.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The necklace-type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 38, speaker 40, and camera 42 are also connected to the bus 52.

[0026] The user 20 wearing the necklace-type terminal 14 may be, for example, a patient whose health condition is being diagnosed, or a regular user.

[0027] The microphone 38 picks up the voice emitted by the user 20, who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions from the user 20 by receiving the voice emitted by the user 20. The microphone 38 captures the voice emitted by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio according to the instructions from the processor 46. The speaker 40 is, for example, a directional speaker and outputs audio towards the user 20's ears.

[0028] Sensor 39 is a sensor that detects biometric data of the user 20, who is wearing the necklace-type terminal. For example, sensor 39 may be a heart rate sensor, a blood oxygen sensor, or a blood pressure sensor.

[0029] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0032] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. The storage 32 stores a specific processing program 56. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58. The data generation model 58 is used by the specific processing unit 290. The storage 32 also includes a data storage unit 57.

[0034] In the necklace-type terminal 14, data acquisition processing is performed by the processor 46. The storage 50 stores the data acquisition program 60. The processor 46 reads the data acquisition program 60 from the storage 50 and executes the read data acquisition program 60 on the RAM 48. The data acquisition processing is realized by the processor 46 operating as a control unit 46A according to the data acquisition program 60 executed on the RAM 48.

[0035] As shown in Figures 3 and 4, the necklace-type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. Figures 3 and 4 show an example where two microphones 38 are positioned in front of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two sensors 39 are positioned to the right and left of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two speakers 40 are positioned to the right rear and left rear of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two cameras 42 are positioned to the right front and left front of the user 20 when the user 20 wears the necklace-type terminal 14. Finally, they show an example where two sensors 39 are positioned inside the necklace-type terminal 14 so as to contact the user 20's neck when the user 20 wears the necklace-type terminal 14.

[0036] Next, we will explain the processing of the control unit 46A when the necklace-type terminal 14 performs data collection processing to collect data.

[0037] In this embodiment, the data collection process collects the user's biometric data in real time. Furthermore, it collects not only biometric data but also all surrounding environment data. This makes it possible to detect early signs of conditions such as Alzheimer's disease and dementia. It also allows for monitoring of the user's health status (e.g., heart disease).

[0038] As shown in Figure 5, the control unit 46A includes a data acquisition unit 100 and a communication unit 102.

[0039] The data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42, respectively.

[0040] The communication unit 102 transmits the outputs of the microphone 38, sensor 39, and camera 42, which are collected by the data acquisition unit 100, to the data processing unit 12.

[0041] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs specific processing to obtain a response corresponding to a user utterance.

[0042] In the specific processing in this embodiment, a response corresponding to the user utterance picked up by the microphone 38 of the necklace-type terminal 14 is obtained using the data generation model 58.

[0043] As shown in Figure 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0044] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, respectively, received from the necklace-type terminal 14, in the data storage unit 57.

[0045] The input unit 292 acquires user utterances received by the necklace-type terminal 14. Specifically, it acquires user utterances picked up by the microphone 38 of the necklace-type terminal 14.

[0046] The processing unit 294 performs specific processing using the data generation model 58. Specifically, it inputs a prompt including user utterances to the data generation model 58 and obtains a generation result. At this time, the outputs of the sensor 39 and camera 42 collected by the data acquisition unit 100 may also be included in the prompt.

[0047] The output unit 296 transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the specific processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the specific processing. The control unit 46A transmits the audio data indicating the user utterance acquired by the microphone 38 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the user utterance.

[0048] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0049] The outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 57 are used, for example, to diagnose the health status of user 20. In this case, the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 57 may be transmitted to a terminal on the medical institution side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 57 to diagnose the health status of user 20.

[0050] Next, the operation of the data processing system 10 will be explained.

[0051] First, let's explain an example of the data collection process flow.

[0052] When user 20 is wearing the necklace-type terminal 14, the data acquisition unit 100 sequentially collects the outputs of the microphone 38, sensor 39, and camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, sensor 39, and camera 42 collected by the data acquisition unit 100 to the data processing unit 12.

[0053] Next, an example of the flow of a specific process will be explained with reference to Figure 7. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 57.

[0054] In step S300, the processing unit 294 determines whether a predetermined trigger condition is met. Specifically, the trigger condition may be that the user utterance picked up by the microphone 38 contains a specific word (for example, the name of the agent installed in the necklace-type terminal 14) or a phrase (for example, "Hi! XX" (where XX is the name of the agent)).

[0055] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 terminates the specific processing.

[0056] In step S301, the processing unit 294 generates a prompt by adding an instruction to obtain the result of a specific process to the text representing the user utterance picked up by the microphone 38.

[0057] For example, a prompt such as "The user is speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech) can be generated. Alternatively, the outputs of sensor 39 and camera 42 can be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and video data representing the user's surroundings. The user is also speaking as follows: XXX. Please respond as the agent." (XXX is the user's speech)

[0058] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58 and obtains the result of a specific process based on the output of the data generation model 58.

[0059] In step S304, the output unit 296 outputs the result of the specific processing to the necklace-type terminal 14 and terminates the specific processing.

[0060] (Second embodiment) Next, a second embodiment of the data processing system 10 according to the embodiment will be described, omitting or simplifying parts that overlap with the above embodiment.

[0061] In the data processing system 10 according to the second embodiment, the data processing device 12 performs a suggestion process to generate suggestions for cooking a selected recipe for a user 20 who is cooking the recipe, based on biometric data transmitted from the necklace-type terminal 14. The necklace-type terminal 14 outputs the results of the suggestion process transmitted from the data processing device 12 through the speaker 40.

[0062] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs the proposed processing.

[0063] As shown in Figure 8, the specific processing unit 290 according to the second embodiment includes an input unit 292, an acquisition unit 298, an output unit 296, and a processing unit 294. The output unit 296 is an example of the "first output unit and second output unit" according to the technology of this disclosure.

[0064] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, respectively, received from the necklace-type terminal 14, in the data storage unit 57.

[0065] The input unit 292 acquires user utterances received by the necklace-type terminal 14. Specifically, it acquires user utterances picked up by the microphone 38 of the necklace-type terminal 14.

[0066] The acquisition unit 298 acquires recipe data from the recipe database 70 (see Figure 9) that corresponds to the recipe selected by the user 20, as indicated by the user utterance received by the necklace-type terminal 14.

[0067] The output unit 296 transmits the recipe data acquired by the acquisition unit 298 to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A outputs the acquired recipe data from the speaker 40. As a result, the speaker 40 outputs audio explaining the recipe selected by the user 20 to the user 20.

[0068] The processing unit 294 performs suggestion processing using the data generation model 58. Specifically, it inputs a first prompt containing the user 20's biometric data transmitted from the necklace-type terminal 14, or a second prompt containing the user 20's characteristics regarding food ingredients, to the data generation model 58 and obtains the generation result. Specific examples of the first and second prompts will be described later.

[0069] The output unit 296 transmits the result of the suggestion processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the suggestion processing. In this way, the speaker 40 outputs the cooking suggestions for the recipe selected by the user 20 to the user 20.

[0070] Next, the configuration of the storage 32 of the data processing device 12 will be described. Figure 9 is a block diagram showing the configuration of the storage 32 of the data processing device 12 according to the second embodiment.

[0071] As shown in Figure 9, the storage 32 stores a specific processing program 56, a data generation model 58, a data storage unit 57, a recipe database 70, and a characteristics database 72.

[0072] The proposed processing is realized by the processor 28 operating as a specific processing unit 290 shown in Figure 8, according to a specific processing program 56 executed on the RAM 30. The data generation model 58 and the data storage unit 57 are used by the specific processing unit 290 when the proposed processing is executed.

[0073] The recipe database 70 stores recipe data, for example, showing multiple recipes posted by various users on a designated website. Here, "recipe" refers to an instruction sheet that describes how to make a dish (procedure) and the necessary ingredients, used to check the steps and ingredients when making a dish. The recipe data consists of the recipe name corresponding to each recipe, the instructions for making the dish shown in the recipe, the ingredients used in the recipe, and the nutritional value and allergy information of those ingredients.

[0074] The characteristics database 72 stores, for example, the characteristics of various ingredients registered by different users. Here, "characteristics of ingredients" include the presence or absence of food allergies and food intolerances. For example, the characteristics database 72 registers ingredients that user 20 is allergic to and ingredients that user 20 is intolerant to. Hereinafter, ingredients for which user 20 is allergic to or intolerant to at least one of the following will be referred to as "specific ingredients."

[0075] Next, an example of the proposed processing flow will be explained with reference to Figures 10 and 11. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 57.

[0076] First, we will explain the first operational flow of the proposal process using Figure 10. In step S310, the input unit 292 acquires the user utterance received by the necklace-type terminal 14. Here, it is assumed that the user utterance includes a specific phrase for selecting the recipe to be made (for example, "I want to eat XX" (where XX is the name of the dish)).

[0077] In step S311, the acquisition unit 298 acquires recipe data from the recipe database 70 that corresponds to the recipe selected by user 20, as indicated by the user utterance acquired in step S310.

[0078] In step S312, the output unit 296 transmits the recipe data acquired in step S311 to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A outputs the acquired recipe data from the speaker 40. As a result, the speaker 40 outputs audio explaining the recipe selected by the user 20 to the user 20. Therefore, the user 20 can proceed with cooking the recipe based on the audio output from the speaker 40. The necklace-type terminal 14 then automatically sets a timer according to the cooking procedure while the recipe is being cooked and outputs audio explaining the next step from the speaker 40 at the appropriate timing.

[0079] In step S313, the processing unit 294 determines whether the state of user 20 identified based on biometric data satisfies predetermined conditions. Specifically, the processing unit 294 determines that predetermined conditions are met if the stress level of user 20 identified based on biometric data is above a threshold, or if the value of the biometric data is above a threshold. If the predetermined conditions are met, the process proceeds to step S314; otherwise, the process proceeds to step S317.

[0080] As an example, the processing unit 294 identifies the stress level based on the user 20's heart rate variability included in the biometric data. A smaller variability between heartbeats indicates higher stress, while a larger variability indicates lower stress. The processing unit 294 identifies the user 20's stress level between 0 and 100, setting 76 as the stress level threshold.

[0081] Furthermore, the processing unit 294 uses blood pressure as a value for biological data and sets "maximum blood pressure of 140 mmHg" as the blood pressure threshold.

[0082] In step S314, the processing unit 294 generates a first prompt that includes the user 20's biometric data and an instruction statement for outputting a suggestion to simplify the cooking procedure of the recipe selected by the user 20 or a suggestion for cooking content that will lead to a decrease in the user 20's blood pressure.

[0083] For example, the processing unit 294 generates a first prompt that reads, "This is biometric data representing the user's heart rate. The user's stress level is high, so as an agent, please suggest simplifying the cooking procedure for the recipe." The processing unit 294 also generates a first prompt that reads, "This is biometric data representing the user's blood pressure. The user's blood pressure is high, so as an agent, please suggest cooking methods that will help lower the user's blood pressure."

[0084] In step S315, the processing unit 294 inputs the generated first prompt to the data generation model 58 and obtains the result of the proposed process based on the output of the data generation model 58.

[0085] In step S316, the output unit 296 outputs the result of the suggestion process to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the suggestion process. For example, the speaker 40 outputs a voice message such as "Would you like to simplify the procedure and change to a time-saving recipe?" as a suggestion to simplify the cooking procedure of the recipe selected by the user 20. The speaker 40 also outputs a voice message such as "Let's reduce the amount of salt to lower the user 20's blood pressure" as a suggestion for cooking methods that may lower the user 20's blood pressure.

[0086] In step S317, the processing unit 294 determines whether the termination conditions for the proposal process are met. Specifically, the processing unit 294 determines that the termination conditions are met if the user utterance picked up by the microphone 38 contains a specific word (for example, "finished") or phrase (for example, "XX, the food is ready" (XX is the agent's name)). If the termination conditions are met, the proposal process ends; otherwise, the process returns to step S313.

[0087] Next, we will explain the second operation flow of the proposal process using Figure 11. In step S320, the input unit 292 acquires the user utterance received by the necklace-type terminal 14. Here, it is assumed that the user utterance includes a specific phrase for selecting the recipe to be made (for example, "I want to eat XX" (where XX is the name of the dish)).

[0088] In step S321, the acquisition unit 298 acquires recipe data from the recipe database 70 that corresponds to the recipe selected by user 20, as indicated by the user utterance acquired in step S320.

[0089] In step S322, the output unit 296 transmits the recipe data acquired in step S321 to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A outputs the acquired recipe data from the speaker 40. As a result, the user 20 can proceed with cooking the recipe based on the voice explanation of the recipe output from the speaker 40.

[0090] In step S323, the processing unit 294 determines whether a specific ingredient is included in the ingredients used in the recipe, based on the image of the user 20's surroundings received from the necklace-type terminal 14. Specifically, the processing unit 294 performs known image processing on the image of the user 20's surroundings to identify the ingredients shown in the image. The processing unit 294 then determines that a specific ingredient is included if the identified ingredient is one for which the user 20, registered in the characteristics database 72, has at least one food allergy and / or food intolerance. If a specific ingredient is included, the process proceeds to step S324; otherwise, the process proceeds to step S327.

[0091] In step S324, the processing unit 294 generates a second prompt that includes the characteristics of the user 20's ingredients and instructions for outputting a suggestion to remove a specific ingredient from the recipe selected by the user 20.

[0092] For example, the processing unit 294 generates a second prompt that says, "This is the user's characteristic regarding ingredients. The user has a food allergy to XX (XX is the name of a specific ingredient). As an agent, please suggest removing XX from the recipe."

[0093] In step S325, the processing unit 294 inputs the generated second prompt to the data generation model 58 and obtains the result of the proposed process based on the output of the data generation model 58.

[0094] In step S326, the output unit 296 outputs the result of the suggestion process to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the suggestion process. For example, the speaker 40 outputs a voice message such as, "The recipe contains nuts, would you like to change them to a substitute?" as a suggestion to remove a specific ingredient from the recipe selected by the user 20.

[0095] In step S327, the processing unit 294 determines whether the termination conditions for the proposal process are met. Specifically, the processing unit 294 determines that the termination conditions are met if the user utterance picked up by the microphone 38 contains a specific word (for example, "finished") or phrase (for example, "XX, the food is ready" (XX is the agent's name)). If the termination conditions are met, the proposal process ends; otherwise, the process returns to step S323.

[0096] As described above, in the data processing system 10, the input unit 292 acquires user utterances picked up by the microphone 38. The acquisition unit 298 acquires recipe data from the recipe database 70 corresponding to the recipe selected by user 20 as indicated by the user utterance. The output unit 296 outputs the recipe data acquired by the acquisition unit 298 to the necklace-type terminal 14. If the state of user 20 identified based on the biometric data transmitted from the necklace-type terminal 14 satisfies predetermined conditions, the processing unit 294 inputs a first prompt including biometric data to the data generation model 58 and uses the output of the data generation model 58 to acquire a cooking suggestion for the recipe. The output unit 296 outputs the cooking suggestion for the recipe acquired by the processing unit 294 to the necklace-type terminal 14. With this configuration, the necklace-type terminal 14 outputs a cooking suggestion for a recipe suitable for user 20's state identified from user 20's biometric data from the speaker 40. Thus, according to the data processing system 10, it is possible to make cooking suggestions that are suitable for the user 20's condition, which is identified from the user 20's biometric data.

[0097] Furthermore, in the data processing system 10, if the stress level of user 20, identified based on biometric data, is above a threshold, the processing unit 294 adds an instruction to the first prompt to output a suggestion to simplify the recipe's cooking procedure. When this first prompt is input to the data generation model 58, the necklace-type terminal 14 outputs a suggestion to simplify the recipe's cooking procedure from the speaker 40. Therefore, according to the data processing system 10, a cooking procedure suitable for user 20's stress level can be suggested.

[0098] Furthermore, in the data processing system 10, the processing unit 294 adds an instruction to the first prompt to output a suggestion for cooking methods that would lead to a decrease in blood pressure if the blood pressure is above a threshold. When this first prompt is input to the data generation model 58, the necklace-type terminal 14 outputs a suggestion for cooking methods that would lead to a decrease in blood pressure from the speaker 40. Therefore, according to the data processing system 10, it is possible to suggest cooking methods that would improve the health condition of the user 20.

[0099] Furthermore, in the data processing system 10, if a specific ingredient is included in the ingredients used in the recipe, the processing unit 294 inputs a second prompt containing the user 20's characteristics regarding ingredients to the data generation model 58, and uses the output of the data generation model 58 to obtain a suggestion to remove the specific ingredient from the recipe. The output unit 296 outputs the suggestion to remove the specific ingredient from the recipe obtained by the processing unit 294 to the necklace-type terminal 14. As a result of the input of the second prompt to the data generation model 58, the necklace-type terminal 14 outputs a suggestion to remove the specific ingredient from the recipe. Therefore, according to the data processing system 10, the risk of the user 20 consuming ingredients that may be harmful to their health can be reduced.

[0100] In the second embodiment, blood pressure was used as the biometric data used to determine whether the value was above or below a threshold, but this biometric data is not limited to blood pressure. For example, the biometric data may be heart rate, blood glucose level, etc., instead of or in addition to blood pressure.

[0101] In the second embodiment, the specific processing unit 290 generates a first prompt containing instructions to output a suggestion to simplify the cooking procedure of the recipe selected by the user 20 if the user 20's stress level is above a threshold, but the processing is not limited to this. In this case, in addition to generating the first prompt, the specific processing unit 290 may also send an instruction to the necklace-type terminal 14 to play music that helps the user 20 relax. This makes it possible to create an environment in which the user 20 can cook while supporting their mental health.

[0102] In the second embodiment, the identification processing unit 290 may adjust the ingredients used in the recipe based on a comparison between the calories consumed by the user 20, identified based on biometric data (e.g., heart rate), and the user 20's target calorie intake stored in the storage 32. For example, if the value obtained by subtracting the calories consumed from the user 20's target calorie intake falls below a predetermined value, the identification processing unit 290 may input a prompt to the data generation model 58 that includes an instruction to output a suggestion to add high-calorie ingredients to the recipe. Alternatively, if the value obtained by subtracting the calories consumed from the user 20's target calorie intake falls above a predetermined value, the identification processing unit 290 may input a prompt to the data generation model 58 that includes an instruction to output a suggestion to add low-calorie ingredients to the recipe.

[0103] In the second embodiment, the identification processing unit 290 may adjust the ingredients used in the recipe based on the user 20's daily biometric data stored in the data storage unit 57. For example, if the identification processing unit 290 identifies that the user 20 is experiencing mental fatigue based on the heart rate variability shown in the daily biometric data, it may input a prompt to the data generation model 58 that includes an instruction to output a suggestion to add ingredients containing nutrients effective for fatigue recovery to the recipe.

[0104] In the second embodiment, the specific processing unit 290 may output the results of the suggestion processing to a user terminal such as a smartphone or tablet held by the user 20, in addition to the necklace-type terminal 14. This allows the explanation of the recipe selected by the user 20 to be displayed on the screen of the user terminal, enabling the user 20 to proceed with cooking while checking the status on their user terminal.

[0105] In the second embodiment, the specific processing unit 290 may store the user 20's biometric data collected during the cooking of the recipe in the data storage unit 57 and use it for personalization when suggesting recipes to the user 20 in the future. Specifically, the specific processing unit 290 may use the user 20's biometric data collected during the cooking of the recipe as training data to fine-tune the data generation model 58.

[0106] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0107] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0108] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0109] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0110] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0111] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0112] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0113] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0114] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0115] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0116] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference. [Explanation of Symbols]

[0117] 10 Data Processing Systems 12 Data Processing Devices 14 Necklace-type terminal 38 Microphones 39 Sensors 40 speakers 42 cameras 46A Control Unit 100 Data Acquisition Unit 102 Communications Department 290 Specific Processing Unit 292 Input section 294 Processing Unit 296 Output section< / url:>

Claims

1. A camera that takes pictures of the area around the wearer. A sensor that detects the wearer's biometric data, microphone, A collection unit that collects the outputs of the camera, the sensor, and the microphone, A communication unit transmits the outputs of the camera, the sensor, and the microphone, respectively, collected by the collection unit, to a data processing unit, and A speaker that outputs a response corresponding to the user utterance picked up by the aforementioned microphone. A necklace-type terminal including, The input unit that acquires the user utterance, A unit that obtains recipe data from a recipe database corresponding to the recipe selected by the wearer as indicated by the user utterance. A first output unit outputs the recipe data acquired by the acquisition unit to the necklace-type terminal. A processing unit, which, when the wearer's status identified based on the biometric data transmitted from the necklace-type terminal satisfies predetermined conditions, inputs a first prompt including the biometric data into a data generation model and uses the output of the data generation model to obtain a cooking suggestion for the recipe, and A second output unit outputs cooking suggestions for the recipe acquired by the processing unit to the necklace-type terminal. The data processing device includes, A data processing system equipped with the following features.

2. If the stress level of the wearer, identified based on the biological data, is above a threshold, the processing unit adds an instruction to the first prompt to output a suggestion to simplify the cooking procedure of the recipe. The data processing system according to claim 1.

3. The processing unit, when the value of the biological data is above a threshold, adds an instruction to the first prompt to output a suggestion for cooking methods that would lead to a decrease in the value of the biological data. The data processing system according to claim 1.

4. The processing unit, if the ingredients used in the recipe identified based on the image of the wearer's surroundings transmitted from the necklace-type terminal include a specific ingredient for which the wearer has at least one food allergy and / or food intolerance, as registered in a characteristics database showing the wearer's characteristics regarding food, inputs a second prompt including the wearer's characteristics regarding food to the data generation model, and uses the output of the data generation model to obtain a suggestion to remove the specific ingredient from the recipe. The second output unit outputs to the necklace-type terminal a suggestion to remove specific ingredients from the recipe acquired by the processing unit. The data processing system according to claim 1.

Images