System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting food allergen information from a store; means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; means for updating the allergen information in the database; means for providing the updated allergen information to a user; means for receiving a report from the user when an allergic reaction occurs; means for analyzing the received report information and identifying related menus and components; and means for linking the analysis result to a primary care physician.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, the number of people with food allergies is increasing, making ensuring a safe diet a major issue. However, insufficient information about allergens contained in food is provided at restaurants and convenience stores, making it difficult for consumers with food allergies to choose meals with confidence. Furthermore, while a rapid response is required when an allergic reaction occurs, the current system makes it difficult to share sufficient information. These issues make people with food allergies afraid to eat out or take out meals, which is a problem. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a system that includes a means for collecting food allergen information from restaurants, a means for analyzing the collected information and identifying allergen components, and a means for storing and updating the identified allergen components in a database. The system also includes a means for providing updated allergen information to users, a means for receiving reports from users when allergic reactions occur, a means for analyzing the received reports and identifying relevant menu items and ingredients, and a means for sharing the analysis results with a family doctor. This allows consumers with food allergies to select safe meals based on the latest allergen information, and allows for prompt response in the unlikely event of an allergic reaction. By utilizing this system, an environment can be provided in which people with food allergies can enjoy eating out or taking home meals with peace of mind.

[0006] "Store" refers to a place such as a restaurant or store that serves food.

[0007] "Food allergen information" refers to detailed data about how specific foods may cause allergic reactions.

[0008] "Collection means" refers to the methods and devices used to obtain and retain information.

[0009] "Means of analysis" refers to methods or devices for analyzing collected information and deriving specific purposes or meanings.

[0010] "Allergens" refer to substances contained in certain foods that can cause allergic reactions.

[0011] A "database" refers to a structured system that organizes and efficiently stores information and data.

[0012] "User" refers to any individual who uses the system, particularly consumers with allergies.

[0013] "Means for receiving reports" refers to a method or device for receiving information from a user.

[0014] "Means of collaboration" refers to methods and devices for sharing relevant information with other systems or agencies.

[0015] "Family doctor" refers to the doctor in charge of managing the user's health. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] MODE FOR CARRYING OUT THE INVENTION

[0038] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports safe meal choices for users with food allergies. It also enables information sharing and rapid response in the event of an allergic reaction. The system's program processing is explained below in natural language.

[0039] Basic system configuration

[0040] This system is mainly composed of three main components: a server, a user's device, and a database. The server plays a central role, while the user's device acts as an interface for obtaining information. The database stores and manages all allergen information.

[0041] Program processing flow

[0042] 1. Data Collection Phase

[0043] Obtaining store information

[0044] The server automatically collects menu information from the restaurant's website and dedicated API. This process covers all menu information that the restaurant has made public.

[0045] For example, restaurant A collects the menu information (menu name, ingredient list, allergen ingredients) provided on its website.

[0046] Analysis of allergen information

[0047] The server analyzes the collected menu information and uses an AI model to identify any allergen ingredients it contains.

[0048] For example, for a menu item called "chocolate cake," the AI ​​model will identify allergens such as "dairy products" and "nuts" from the ingredient list.

[0049] 2. Saving and updating to the database

[0050] Conservation

[0051] After the allergen information is identified, the server stores this information in a database.

[0052] Each menu item is linked to ingredient information and allergen information, allowing users to refer to it later.

[0053] Regular data updates

[0054] The server periodically recollects store information and updates the database, ensuring that allergen information is always up-to-date.

[0055] 3. Providing Information to End Users

[0056] Obtaining allergen information

[0057] The user's terminal accesses the server and requests allergen information for a specific store or menu item.

[0058] For example, if user A searches for "dairy-free menu at restaurant B," the server will return relevant information.

[0059] User interface display

[0060] The user's device displays the allergen information obtained from the server on the user interface, helping the user select a safe menu.

[0061] 4. Reporting and dealing with allergic reactions

[0062] Response reporting

[0063] If a user experiences an allergic reaction after eating, they can use a dedicated application to report the information to the server.

[0064] For example, user B reports that "I developed hives after eating at restaurant C."

[0065] Analysis and collaboration

[0066] The server analyzes reported allergic reaction information to identify whether a particular menu item or ingredient may be the cause.

[0067] If necessary, the analysis results will be shared with the patient's primary care physician to assist in prompt response.

[0068] The system of the present invention provides an environment in which users can choose meals with peace of mind when eating out or eating at home. It also enables prompt and appropriate response in the event of an allergic reaction, improving the quality of life for people with food allergies.

[0069] The processing flow will be explained below.

[0070] Step 1: Data collection

[0071] The server collects menu information from each store's website or dedicated API.

[0072] The server temporarily stores the collected data for analysis.

[0073] Step 2: Allergen information analysis

[0074] The server analyzes the collected menu information using an AI model to identify allergen ingredients.

[0075] Analysis uses techniques such as natural language processing and image analysis.

[0076] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[0077] Step 3: Saving to the database

[0078] The server stores the analysis results in a database.

[0079] Ingredient information and allergen information are stored linked to each menu item.

[0080] Step 4: Regularly update your data

[0081] The server recollects new menu information and change information from the stores according to a preset schedule.

[0082] The server analyzes the new information collected and updates the database.

[0083] This ensures that the latest allergen information is always maintained.

[0084] Step 5: Obtain allergen information

[0085] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[0086] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0087] Step 6: Viewing information

[0088] The user's terminal displays the allergen information received from the server on a user interface.

[0089] The user can select a safe menu based on the displayed information.

[0090] Step 7: Report an allergic reaction

[0091] If a user experiences an allergic reaction after eating, the user reports the information to the server using a dedicated application.

[0092] For example, enter a specific symptom such as "I experienced difficulty breathing 20 minutes after eating."

[0093] Step 8: Analyze the reported information

[0094] The server analyzes the allergic reaction information reported by the user and identifies relevant menu items and ingredients.

[0095] The server may issue an alert based on the analysis results.

[0096] Step 9: Linking with medical institutions

[0097] The server shares the analysis results with the user's family doctor.

[0098] The server uses the linked information to help doctors respond quickly.

[0099] Step 10: Provide feedback

[0100] The server may notify all users of the collected allergic reaction information as a warning or caution.

[0101] This will help other users avoid similar allergic reactions.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional systems lack sufficient food allergy information, and data collection and analysis are often done manually, resulting in problems with accuracy and speed. Another issue is the difficulty of sharing information and responding immediately when an allergic reaction occurs. This means that people with food allergies cannot enjoy meals safely when eating out or choosing foods, which reduces their quality of life.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for collecting food allergen information from stores via websites or APIs, means for analyzing the collected food allergen information using a generative AI model to identify allergen components, means for saving the identified allergen components in a relational database, means for periodically updating the allergen information in the database, means for providing the updated allergen information to the user's device, means for receiving reports from users when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with medical institutions. This automates the entire process from collection to analysis, storage, provision, and response, enabling users with food allergies to choose safe and reliable meals.

[0107] A "website" is a collection of document formats for publishing and viewing information on the Internet, created using HTML or other formats.

[0108] "API" stands for Application Programming Interface, an interface that enables communication between different software programs.

[0109] "Food allergen information" refers to information about ingredients contained in food that may cause allergic reactions in certain individuals.

[0110] A "generative AI model" is an artificial intelligence model that has been trained using machine learning or deep learning techniques for a specific task.

[0111] "Analysis" is the process of examining and processing collected data in detail to extract and identify specific information.

[0112] An "allergen" is a specific component in food that may cause an allergic reaction.

[0113] A "relational database" is a database system that represents data in tabular format and manages and manipulates data using the relationships between tables.

[0114] "Update" is the process of replacing existing data or information with new data or information.

[0115] A "terminal" is a device that a user uses to input and check information, including smartphones and personal computers.

[0116] "Reporting" is the act of providing information about a particular event or condition, in this case an allergic reaction.

[0117] "Analysis results" refer to the final information or conclusions obtained during the data analysis process.

[0118] A "medical institution" is a facility where doctors and medical professionals provide medical services, including hospitals and clinics.

[0119] MODE FOR CARRYING OUT THE INVENTION

[0120] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports users with food allergies in making safe meal choices. It also enables information sharing and rapid response in the event of an allergic reaction. The following describes in detail how the system's program processing is implemented.

[0121] Basic system configuration

[0122] This system consists of three main components: a server, a user's device, and a relational database. The server plays a central role, while the user's device acts as an interface for retrieving information. The database stores and manages all allergen information.

[0123] Hardware and Software

[0124] Server: A general server computer with a Linux OS is used.

[0125] Device: The smartphone or computer used by the user.

[0126] Database: Use a relational database such as MySQL or PostgreSQL.

[0127] Software and Libraries:

[0128] Web scraping: Python's Beautiful Soup and Selenium

[0129] Generative AI model: Generative AI using natural language processing technology (e.g., GPT-3)

[0130] Scheduling: Cron Jobs in Linux

[0131] Program processing flow

[0132] 1. Collecting store information

[0133] The server automatically collects menu information from each restaurant's website or through a dedicated API. Information collected includes the menu name, ingredient list, and allergen ingredients. Beautiful Soup and Selenium are used for web scraping.

[0134] Examples:

[0135] The server accesses the restaurant's website and extracts information from the HTML page, such as "Chocolate cake, ingredient list: sugar, flour, dairy, nuts."

[0136] 2. Analysis of allergen information

[0137] The collected data is analyzed by a server using a generative AI model, which uses natural language processing techniques to identify allergens from the ingredient list.

[0138] Examples:

[0139] The server inputs the list of ingredients for "chocolate cake" into the AI ​​model and identifies "dairy" and "nuts" as allergens.

[0140] Example prompt sentence:

[0141] "Collect restaurant menu information and use an AI model to identify ingredients that can identify allergens."

[0142] 3. Saving to the database

[0143] The server stores the identified allergen information in a relational database, linking ingredient information and allergen information for each menu item.

[0144] Examples:

[0145] The server links allergen information for "dairy products" and "nuts" to "chocolate cake" and stores it in a database.

[0146] 4. Regular data updates

[0147] The server periodically recollects store information and updates the database, using a cron job to update the data weekly.

[0148] Examples:

[0149] The server runs web scraping every Sunday to retrieve new menu information and update the database.

[0150] 5. Providing allergen information

[0151] The user's terminal requests allergen information for a specific restaurant or menu from the server based on the user's request. In response to the request, the server retrieves relevant information from the database and provides it to the user's terminal.

[0152] Examples:

[0153] When a user searches for "dairy-free menu items at a specific restaurant," the terminal sends a request to the server, which responds with relevant menu information.

[0154] Example prompt sentence:

[0155] "Please extract the relevant allergen information from the database and provide it to the user."

[0156] 6. Reporting and dealing with allergic reactions

[0157] Users use an application to report allergic reactions and send the information to a server, which analyzes the information, identifies related menu items and ingredients, and contacts a medical institution.

[0158] Examples:

[0159] When a user reports that they broke out in hives at a particular restaurant, the server compares the information with the ingredients, identifies the offending ingredient, and notifies a medical professional.

[0160] This system provides users with an environment where they can choose their meals with peace of mind when eating out, and enables quick and appropriate response in the event of an allergic reaction, improving quality of life.

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Step 1: Gather store information

[0163] server

[0164] Input: Website URL or API endpoint for each store.

[0165] How it works: The server uses Python's Beautiful Soup and Selenium to perform web scraping. Specifically, it retrieves the HTML of a web page from a specified URL and extracts information such as menu names, ingredient lists, and allergens.

[0166] Data processing: The acquired HTML is analyzed as a DOM tree and the necessary information is extracted in text format.

[0167] Output: A list of extracted menu information.

[0168] Step 2: Analysis of allergen information

[0169] server

[0170] Input: Menu information collected in step 1.

[0171] Operation: The collected menu information is input into the generative AI model and analyzed.

[0172] Data processing: The generative AI model uses natural language processing techniques to identify allergens from an ingredient list.

[0173] Output: A list of menu items that contain the identified allergen ingredients.

[0174] Step 3: Saving to the database

[0175] server

[0176] Input: Allergen information parsed in step 2.

[0177] How it works: The server connects to a relational database such as MySQL or PostgreSQL and stores allergen information.

[0178] Data processing: Ingredient information and allergen information are linked for each menu item and stored in a database.

[0179] Output: Success status of the save operation.

[0180] Step 4: Regularly update your data

[0181] server

[0182] Input: Scheduled to run periodically (Cron job).

[0183] What it does: Uses a cron job to re-run web scraping at scheduled times to retrieve new menu information and update existing data.

[0184] Data processing: Compare new data with existing data and update any changes.

[0185] Output: A list of updated allergen information.

[0186] Step 5: Request Allergen Information

[0187] Terminal

[0188] Input: What the user wants to search for (store name, whether or not it contains allergens, etc.).

[0189] Operation: The device sends an API request to the server based on this input data.

[0190] Data transformation: Constructing request data based on user input.

[0191] Output: Allergen information returned by the server.

[0192] Step 6: Display allergen information

[0193] Terminal

[0194] Input: Allergen information obtained from the server in step 5.

[0195] Operation: The terminal displays the acquired allergen information on the user interface.

[0196] Data processing: Converting data into a display format.

[0197] Output: A display of the menu and its allergen information on the user interface.

[0198] Step 7: Report an allergic reaction

[0199] User

[0200] Input: Detailed information in case of allergic reaction (store name, food name, symptoms, etc.).

[0201] How it works: Using a dedicated application, users enter details of their allergic reaction and send them to a server.

[0202] Data Processing: Converting entered information into a reporting format.

[0203] Output: Notification that report information has been sent to the server.

[0204] Step 8: Analysis and integration of reaction information

[0205] server

[0206] Input: Allergic reaction information reported by the user in step 7.

[0207] How it works: The server parses the reported information and identifies relevant menu items and allergen ingredients.

[0208] Data processing: The reported information is compared with information in the database to identify the causative allergen.

[0209] Output: Analysis results are shared with medical institutions. Details of the analysis results are notified to medical institutions to support a prompt response.

[0210] (Application example 1)

[0211] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0212] It is difficult to provide an environment where consumers with food allergies can safely choose foods when eating out or purchasing them. In particular, in brick-and-mortar stores, consumers face the challenge of having to choose foods while taking the risk of an allergic reaction, due to a lack of easy access to information about allergens contained in menus and foods. Furthermore, there is a lack of systems for quickly sharing information and taking appropriate measures in the event of an allergic reaction.

[0213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0214] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; means for updating the allergen information in the database; means for providing the user with the updated allergen information; means for receiving reports from users when an allergic reaction occurs; means for analyzing the received reports and identifying related menu items and ingredients; means for sharing the analysis results with a family doctor; means for scanning QR codes on food menus with a camera; means for transmitting the scanned information to the server to acquire allergen information; and means for displaying the acquired allergen information on a head-up display. This allows consumers to easily obtain information about allergens contained in foods, enabling safe food selection. It also enables prompt response when an allergic reaction occurs.

[0215] A "store" is a physical location that provides food and merchandise to consumers.

[0216] "Food allergen information" is information about allergen components contained in specific foods.

[0217] "Means of collection" refers to the method or technology used to obtain food allergen information from stores.

[0218] "Means for analyzing" refers to a method or technique for analyzing collected food allergen information and identifying specific allergen components.

[0219] A "database" is an electronic system for systematically storing and managing specific information.

[0220] An "updating means" is a method or technique for replacing existing data with new information.

[0221] "Means for providing to users" refers to the methods or technologies for communicating the collected and analyzed allergen information to users.

[0222] The "means for receiving a report from a user when an allergic reaction occurs" is a method or technology for receiving information about an allergic reaction from a user.

[0223] "Means for analyzing reported information" refers to a method or technique for analyzing the received information about an allergic reaction and identifying its cause.

[0224] "Means for linking to primary care physician" refers to the method or technology for communicating analysis results to healthcare providers.

[0225] A "food menu QR code" is a two-dimensional barcode that contains information about a specific food menu item.

[0226] "Camera scanning means" refers to a method or technology for capturing information in a QR code using a camera.

[0227] "Means for sending information to a server" refers to a method or technique for transmitting acquired data to a server.

[0228] "Means for obtaining allergen information" refers to a method or technology for obtaining the necessary allergen information from the server.

[0229] "Means for displaying on a head-up display" refers to a method or technology for visually displaying the acquired allergen information.

[0230] The system of the present invention provides support for users to safely obtain food allergen information in physical stores and avoid food allergies. The system is mainly composed of a server, a user device (such as smart glasses), a camera, and a database.

[0231] Basic configuration

[0232] 1. Data Collection and Analysis:

[0233] The server uses a camera to read the QR code on the food menu provided by the physical store. The data read by the camera is sent to the server, where an AI model runs to analyze allergen information. The collected food allergen information is analyzed, and the identified allergen ingredients are stored in a database.

[0234] 2. Data Management:

[0235] The database stores identified allergens and is updated regularly. This database management is important to ensure that users are always provided with the most up-to-date allergen information.

[0236] 3. User Interface:

[0237] Users use a user device such as smart glasses to scan QR codes on store menus or food shelves. By scanning the QR code, the menu ID is sent to the server, and the collected allergen information is retrieved. The retrieved allergen information is visually displayed on the head-up display (HUD) of the smart glasses.

[0238] 4. Allergic Reactions:

[0239] If a user experiences an allergic reaction after eating, they can report that information to the server through a dedicated application. The server analyzes the reported information and identifies the relevant menu items and ingredients. If necessary, the analysis results are shared with the user's doctor, allowing for prompt treatment.

[0240] Hardware and software used

[0241] Hardware:

[0242] Smart glasses (e.g. Google Glass, Vuzix Blade): Equipped with a HUD and camera.

[0243] Camera: Used to read QR codes.

[0244] software:

[0245] OpenCV: Processes camera images.

[0246] pyzbar: A library for decoding QR codes.

[0247] requests: Used to send HTTP requests to the server.

[0248] Database management software: stores and manages allergen information.

[0249] Specific examples

[0250] When a user eats at a restaurant, they scan a QR code placed on the table or menu with the smart glasses. For example, the user can check allergen information such as "This menu contains dairy products." If an allergic reaction occurs, the user can report through the application that they "developed hives after eating at a restaurant," and the server will analyze the information.

[0251] Prompt Sentence Examples

[0252] "Scan the QR code displayed in the restaurant with your smart glasses and you'll be able to see allergen information."

[0253] This invention allows consumers with food allergies to eat out safely and make safe food choices, and also allows for quick response if an allergic reaction occurs.

[0254] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0255] Step 1:

[0256] A user scans a QR code on a food menu at a physical store with the camera in their smart glasses. The camera in the smart glasses reads the QR code and obtains its data (menu ID). The input is the QR code, and the output is the menu ID.

[0257] Step 2:

[0258] The smart glasses send the acquired menu ID to the server using an HTTP request. The input is the menu ID, and the output is the request sent to the server.

[0259] Step 3:

[0260] The server searches the database based on the received menu ID and retrieves the corresponding allergen information. The input is the menu ID and the output is the allergen information. The server identifies the information in the database and extracts the necessary data.

[0261] Step 4:

[0262] The server returns the acquired allergen information to the smart glasses. The response is an HTTP response. The input is the allergen information acquired from the database, and the output is the response sent to the smart glasses.

[0263] Step 5:

[0264] The smart glasses receive allergen information from the server and display it on a HUD (head-up display). The user can visually check the allergen information. The input is the allergen information, and the output is what is displayed on the HUD. For example, it might say, "This menu contains dairy products."

[0265] Step 6:

[0266] If a user experiences an allergic reaction after eating, they report the information to the server using a dedicated application. The input is detailed information about the allergic reaction, and the output is a report to the server.

[0267] Step 7:

[0268] The server analyzes the reported information and identifies the relevant menu items and ingredients. The input is the allergic reaction report, and the output is the specific information for the relevant menu items and ingredients. The server then searches the database again based on the report and derives the analysis results.

[0269] Step 8:

[0270] The server will share the analysis results with the patient's primary care physician as needed. This sharing is done via email or API. The input is the analysis results, and the output is shared with the patient's primary care physician. For example, it will send information that "the user has had an allergic reaction to dairy products."

[0271] The above processing steps enable users to safely obtain food allergen information at physical stores, and also create a system that allows for quick response in the event of an allergic reaction.

[0272] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0273] MODE FOR CARRYING OUT THE INVENTION

[0274] This invention is a system that allows users to safely select meals when eating out or taking out a meal. In particular, it combines a function to analyze and provide food allergen information collected from restaurants with an emotion engine that recognizes the user's emotions and makes suggestions related to meal selection. The specific operation of this system is described by explaining the program processing in natural language and using concrete examples.

[0275] Basic system configuration

[0276] The system consists of the following main components:

[0277] 1. Server

[0278] 2. User's device (smartphone, PC, etc.)

[0279] 3. Database

[0280] 4. Emotion Engine

[0281] The server plays a central role, the user's device acts as an interface to retrieve information, the database stores and manages allergen information, and the emotion engine analyzes the user's emotions and makes suggestions based on them.

[0282] Program processing flow

[0283] 1. Data Collection Phase

[0284] The server collects menu information from each store's website or dedicated API.

[0285] The collected data includes menu names, ingredient lists, allergen information, and more.

[0286] Example: Collect information about "chocolate cake" from Restaurant A and temporarily store it.

[0287] 2. Allergen information analysis phase

[0288] The server analyzes the collected menu information and uses an AI model to identify allergen ingredients.

[0289] Identified allergen components are recorded in a database.

[0290] Example: Identifying that "chocolate cake" contains "dairy" and "nuts."

[0291] 3. Saving to the database and updating phase

[0292] The server stores the analysis results in a database.

[0293] Ingredient information and allergen information are linked and saved for each menu item.

[0294] Example: Save allergen information for "chocolate cake" in a database.

[0295] 4. Information provision phase for end users

[0296] The user's terminal connects to the server and requests allergen information for the store and menu.

[0297] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0298] Example: When User A searches for "nut-free menu at Restaurant B," relevant menu information is provided.

[0299] 5. Emotion engine proposal phase

[0300] The server analyzes the emotion data acquired from the user's terminal.

[0301] The emotion engine makes suggestions related to meal selection based on the analyzed emotions.

[0302] Example: If user A is feeling "frustrated," suggest a menu containing ingredients that have a relaxing effect.

[0303] 6. Allergic Reaction Reporting Phase

[0304] If a user experiences an allergic reaction after eating, they enter that information into a dedicated app and report it to the server.

[0305] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[0306] Example: If user B reports that "hives appeared 20 minutes after eating at restaurant C," analyze that information.

[0307] 7. Collaboration with medical institutions phase

[0308] The server then shares the analysis results with the patient's primary care physician.

[0309] Assist doctors in responding quickly when necessary.

[0310] Example: User B's doctor is notified that "nut ingredients are suspected to be the cause of the hives."

[0311] The system of this invention provides a comprehensive solution for users with food allergies to safely enjoy eating out or taking out meals. In addition, the introduction of an emotion engine enables optimal meal selection based on the user's emotional state, contributing to improved user satisfaction.

[0312] The processing flow will be explained below.

[0313] MODE FOR CARRYING OUT THE INVENTION (SPECIFIC PROCESSING STEPS)

[0314] Basic system configuration

[0315] This system consists of a server, a user's terminal, a database, and an emotion engine.

[0316] Specific processing flow of the program

[0317] Step 1: Get store information

[0318] The server periodically collects menu information from each store's website or dedicated API.

[0319] The data collected includes menu names, ingredient lists, and allergen information.

[0320] Step 2: Analysis of allergen information

[0321] The server uses AI to analyze the collected data and identify the allergen ingredients contained in each menu item.

[0322] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[0323] Step 3: Saving to the database

[0324] The server stores the identified allergen component information in a database.

[0325] By storing ingredient and allergen information for each menu item, users can refer to it later.

[0326] Step 4: Regularly update your data

[0327] The server recollects the latest store information at a set time (for example, late at night every day), compares it with the previous data, and updates it.

[0328] If changes are detected, the database is updated to keep the information up to date.

[0329] Step 5: Obtain allergen information

[0330] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[0331] For example, User A searches for "dairy-free menu items at Restaurant B."

[0332] Step 6: Viewing information

[0333] In response to a user request, the server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0334] The user's terminal displays the received information on a user interface, for example, displaying a list of dairy-free menu items.

[0335] Step 7: Recognize emotions

[0336] The user's device uses built-in cameras and sensors to collect emotional data from the user's face and voice.

[0337] For example, when user A is tired, that emotion is recognized.

[0338] Step 8: Send and analyze emotion data

[0339] The user's device transmits the collected emotion data to the server.

[0340] The server's emotion engine analyzes the transmitted emotion data to determine the user's current emotional state.

[0341] Step 9: Emotion-Based Suggestions

[0342] The server proposes the most suitable menu to the user based on the analysis results.

[0343] For example, if the user is feeling stressed, menus that have a relaxing effect are preferentially suggested.

[0344] Step 10: Report an Allergic Reaction

[0345] If a user experiences an allergic reaction after eating, the user uses a dedicated application to report the information to the server.

[0346] For example, user B reports that "dyspnea occurs 20 minutes after eating."

[0347] Step 11: Analyze the reported information

[0348] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[0349] If necessary, the identified information is reflected in the database.

[0350] Step 12: Linking with medical institutions

[0351] The server shares the analysis results with the user's family doctor to help them respond quickly.

[0352] For example, user B's doctor may be notified that "nuts are suspected to be the cause of the breathing difficulties."

[0353] The system of this invention not only allows users to make safe food choices based on the latest allergen information, but also provides optimal suggestions based on the emotion engine.It also enables rapid response in the unlikely event of an allergic reaction, thereby improving the quality of life for people with food allergies.

[0354] Example 2

[0355] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Previously, when eating out or taking home meals, the problem was that food allergen information was not sufficiently obtained, and meal suggestions based on the user's feelings were not adequately provided.In addition, there was a lack of prompt identification of information and coordination with medical institutions when an allergic reaction occurred, making it an issue to increase user safety and satisfaction.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0358] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and using a natural language processing model to identify allergen components; means for saving the identified allergen components in a database; means for updating the allergen information in the database; means for providing the updated allergen information to the user; means for analyzing the user's emotional data; means for making suggestions based on the analyzed emotional data; means for receiving reports from the user when an allergic reaction occurs; means for analyzing the received report information and identifying related menu items and ingredients; and means for sharing the analysis results with a medical institution. This allows the user to safely select meals, reducing the risk of food allergies and receiving meal suggestions tailored to their emotional state. Furthermore, this allows for a rapid response when an allergic reaction occurs, increasing the user's sense of security.

[0359] "Means of collecting food allergen information from stores" refers to the technology and methods for obtaining menu information from store websites or dedicated APIs.

[0360] "Means of using a natural language processing model to analyze collected food allergen information and identify allergen ingredients" refers to a method of analyzing acquired menu information using natural language processing technology and identifying specific allergen ingredients.

[0361] "Means for storing identified allergen components in a database" refers to the technology or method for recording data on allergen components identified through analysis in a database.

[0362] "Means for updating allergen information in the database" refers to techniques or methods for rewriting allergen information already stored in the database with new information.

[0363] "Means for providing updated allergen information to a user" refers to techniques or methods for providing revised allergen information to a user upon request.

[0364] "Means for analyzing user emotion data" refers to techniques and methods for analyzing emotion data provided by a user and understanding its meaning and state.

[0365] "Means for making suggestions based on analyzed emotional data" refers to techniques or methods for generating meal selection or other suggestions based on a user's emotional state.

[0366] "Means for receiving a report from a user when an allergic reaction occurs" refers to an interface for a user to report a food allergic reaction and the technology or method for receiving that report.

[0367] "Means for analyzing received report information and identifying related menu items or ingredients" refers to technologies and methods for analyzing a user's report of an allergic reaction and identifying the menu item or ingredient that caused it.

[0368] "Means of sharing analysis results with medical institutions" refers to technologies and methods for notifying or sharing the analysis results of allergic reactions with medical institutions.

[0369] This invention is a system that enables users to make safe and effective food choices when eating out or taking home meals. This system combines a function to analyze and provide food allergen information collected from restaurants, and an emotion engine that recognizes the user's emotions and makes suggestions related to food selection.

[0370] Hardware and Software

[0371] Hardware: Servers (e.g., cloud computing services), user devices (e.g., smartphones, PCs, etc.)

[0372] software:

[0373] Database management system (e.g. MySQL)

[0374] Sentiment analysis engine (e.g. Google Cloud Natural Language API)

[0375] Website scraping tools (e.g. Beautiful Soup)

[0376] Basic system configuration

[0377] 1. Server: Serves as the central processing unit, collecting, analyzing, storing, and providing various data.

[0378] 2. User's terminal: Acts as an interface for users to obtain and input information.

[0379] 3. Database: Stores and manages allergen information and analysis data.

[0380] 4. Emotion Engine: Analyzes user emotion data and makes suggestions based on it.

[0381] Operation overview

[0382] The operation of this system is as follows.

[0383] Data collection and analysis

[0384] The server collects menu information from restaurant websites and dedicated APIs. This collection is done using the web scraping tool Beautiful Soup. The collected menu information includes the menu name, ingredient list, and allergen information. For example, the server collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[0385] The server then analyzes the collected information using the Google Cloud Natural Language API to identify allergens in each menu item, and the analysis results are stored in a database.

[0386] Providing information

[0387] When a user requests specific allergen information through their device, the server retrieves the relevant information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[0388] Emotion data analysis and proposals

[0389] The server analyzes the emotional data obtained from the user's device using the Google Cloud Natural Language API. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[0390] Reporting allergic reactions and contacting medical institutions

[0391] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. The analysis results are provided to medical institutions as needed. For example, if User B reports that "hives developed 20 minutes after eating at Restaurant D," the server analyzes the information and notifies the medical institution that "nut ingredients may be the cause."

[0392] Prompt Sentence Examples

[0393] 1. Example 1: User A requests "I have a nut allergy, so I would like to find nut-free menu items at Restaurant C."

[0394] Example prompt: "What are the nut-free menu items at Restaurant C?"

[0395] 2. Example 2: User B requests, "I'm feeling stressed today, so please suggest a meal that will help me relax."

[0396] Example prompt: "I'm feeling stressed today, so please suggest something relaxing."

[0397] 3. Example 3: User C requests, "I had an allergic reaction after eating, so I want to report it and identify the cause."

[0398] Example prompt: "I broke out in hives after eating at Restaurant D. Please identify the cause of my allergy."

[0399] As described above, this system is designed to reduce the risk of food allergies and provide meal suggestions based on the user's emotional state.

[0400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0401] Step 1:

[0402] The server collects menu information from restaurant websites and dedicated APIs. Using the web scraping tool Beautiful Soup, it obtains restaurant menu items, ingredient lists, and allergen information. For example, it collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[0403] Input: Store website URL or API endpoint.

[0404] Output: Data including menu name, ingredient list, and allergen information.

[0405] Step 2:

[0406] The server uses the Google Cloud Natural Language API to analyze the collected menu information. Here, natural language processing technology is used to analyze the ingredient information and identify allergens. For example, it can identify that "chocolate cake" contains "dairy products" and "nuts."

[0407] Input: Menu name, ingredient list, and allergen information collected in Step 1.

[0408] Output: Identified allergen ingredient information.

[0409] Step 3:

[0410] The server stores the identified allergen information in a database. In this case, a MySQL database is used to link ingredient information and allergen information for each menu item. For example, information for "Chocolate Cake" is recorded as "ID: 1234, Ingredients: Sugar, Cocoa, Dairy; Allergens: Dairy, Nuts."

[0411] Input: Allergen ingredient information identified in Step 2.

[0412] Output: Allergen information stored in a database.

[0413] Step 4:

[0414] A user's device sends a request for specific allergen information to the server. The server extracts the relevant allergen information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[0415] Input: A request from the user requesting allergen information.

[0416] Output: A list of relevant allergen information.

[0417] Step 5:

[0418] The server uses the Google Cloud Natural Language API to analyze the emotion data obtained from the user's device. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[0419] Input: User emotion data.

[0420] Output: Sentiment analysis results and meal suggestions based on them.

[0421] Step 6:

[0422] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. For example, if User C reports that "hives occurred 20 minutes after eating at Restaurant D," the server analyzes the information and identifies the specific ingredients (e.g., "nuts").

[0423] Input: User's report of an allergic reaction.

[0424] Output: Parsed allergen ingredients and associated menu information.

[0425] Step 7:

[0426] The server transfers the analysis results to a system linked to a medical institution. Based on the information notified to the medical institution, a prompt response can be made. For example, a notification could be sent to User C's family doctor stating that "nut ingredients are suspected to be the cause of the hives."

[0427] Input: Allergy analysis results.

[0428] Output: Report information to medical institutions.

[0429] (Application example 2)

[0430] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0431] Conventional systems cannot completely avoid the risk of food allergies when eating out or taking home meals, and do not suggest food choices based on the user's emotions. Therefore, a system that combines the analysis and provision of allergy information with emotion-based meal suggestions is needed.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0433] In this invention, the server includes means for collecting food allergen information from stores, means for analyzing the collected food allergen information and identifying allergen components, means for saving the identified allergen components in a database, means for updating the allergen information in the database, means for providing the updated allergen information to the user, means for recognizing the user's emotions and making suggestions related to meal selection based on the emotions, means for receiving reports from the user when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with a medical institution. This makes it possible to reduce the risk of food allergies and support meal selection based on the user's emotions.

[0434] The "means for collecting food allergen information from stores" refers to a combination of software and hardware for collecting menu and ingredient information from each store.

[0435] The "means for analyzing collected food allergen information and identifying allergen components" refers to algorithms and programs for analyzing collected data and identifying specific allergen components.

[0436] "Means for storing identified allergen components in a database" refers to a mechanism for storing allergen information and subsequently registering it in a database in a searchable format.

[0437] "Means for updating allergen information in the database" refers to a method for keeping existing allergen information up to date with new data.

[0438] The "means for providing updated allergen information to the user" is a system for notifying or displaying the latest allergen information to the user.

[0439] "Means for recognizing the user's emotions and making suggestions related to meal selection based on those emotions" refers to technology that analyzes the user's emotions from their facial expressions and voice and makes meal suggestions accordingly.

[0440] The "means for receiving a report from the user when an allergic reaction occurs" is a function for the user to report to the system any allergic reaction they have experienced.

[0441] The "means for analyzing the received report information and identifying the related menu or ingredient" is a process for analyzing the allergic reaction report from the user and identifying the menu or ingredient that caused it.

[0442] "Means for sharing analysis results with medical institutions" is a function for sending the analysis results of allergic reactions to the user's regular medical institution.

[0443] The system of the present invention is designed to suggest safe and comfortable meal options by combining food allergy and emotion recognition. Specifically, it is implemented using the following means.

[0444] 1. How to collect allergen information

[0445] The server collects menu information from each restaurant's website and dedicated API. The collected data includes the menu name, ingredient list, allergen information, etc. This makes it possible to always maintain the latest allergen information.

[0446] 2. Methods for analyzing allergen components

[0447] The server analyzes the collected menu information, specifically identifying allergens using an AI model and storing them in a database, where they can be later accessed by users.

[0448] 3. How to store and update allergen information

[0449] The identified allergens are stored in a database that is updated regularly, allowing users to access the most up-to-date allergen information.

[0450] 4. Means of Providing Allergen Information to Users

[0451] The user's device (such as a smartphone or PC) connects to the server and requests allergen information for the restaurant and menu. The server extracts the relevant allergen information from the database and returns it to the user's device.

[0452] 5. Emotion Recognition Suggestion Method

[0453] The emotion engine analyzes emotion data acquired from the user's device. It recognizes the user's emotions using a camera or voice input device, and makes suggestions related to meal selection based on the analysis results. These suggestions are intended to provide the optimal meal based on the user's emotional state.

[0454] 6. Allergic Reaction Reporting Method

[0455] If a user experiences an allergic reaction after eating, they can report the information to the server through a dedicated application. The server then analyzes the reported information and identifies the relevant menu items and ingredients.

[0456] 7. Linking to medical institutions

[0457] The server then sends the analysis results to the user's primary care physician, helping the physician to respond promptly if necessary.

[0458] Specific examples

[0459] For example, when a user visits "Restaurant X," they can use this system to search for "nut-free" menu items, and the camera can recognize their emotions to suggest menu items that include relaxing ingredients. If an allergic reaction occurs after eating, they can report this information to the app, which will promptly notify a medical institution.

[0460] Prompt Sentence Examples

[0461] An example of a prompt to be input to a generative AI model could be the following text:

[0462] "Design a smartphone app that uses an emotion engine to recognize the user's emotions and suggest menu items that match those emotions. This app also has the functionality to collect restaurant menu information and analyze allergens."

[0463] The above is an embodiment of the present invention.

[0464] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0465] Step 1:

[0466] The server collects food allergen information from stores.

[0467] Input: Menu information obtained from each store's website or dedicated API.

[0468] Data processing: Convert menu names, ingredient lists, and allergen information into an analyzable format.

[0469] Output: Menu information formatted in a format suitable for parsing.

[0470] Step 2:

[0471] The server analyzes the food allergen information collected and identifies the allergen components.

[0472] Input: Menu information collected and formatted in step 1.

[0473] Data processing: Using an AI model, allergen ingredients in each menu item are identified.

[0474] Output: Menu information linked to identified allergen ingredients.

[0475] Step 3:

[0476] The server stores the identified allergen components in a database.

[0477] Input: Menu information containing the allergen ingredients identified in step 2.

[0478] Data processing: Converting data into a format suitable for storing in a database.

[0479] Output: Menu and allergen information stored in a database.

[0480] Step 4:

[0481] The device connects to the server and requests allergen information for the store and menu.

[0482] Input: Store and menu information requested by the user.

[0483] Data processing: Extract the necessary menu information from the database.

[0484] Output: Providing relevant allergen information to the user's device.

[0485] Step 5:

[0486] The terminal acquires the user's emotional data, which is then analyzed by the server using an emotion engine.

[0487] Input: User's facial expressions and voice data.

[0488] Data processing: Analyze data using an emotion engine to identify emotional states.

[0489] Output: The analyzed emotional state of the user.

[0490] Step 6:

[0491] The server makes suggestions related to meal selection based on the analyzed emotional state.

[0492] Input: User's emotional state and menu information identified in step 5.

[0493] Data processing: Filter the optimal menu according to emotions and generate a list of suggestions.

[0494] Output: The optimal menu list proposed to the user's device.

[0495] Step 7:

[0496] The user reports an allergic reaction and the server receives the reported information.

[0497] Input: Allergic reaction report data from users.

[0498] Data processing: Analyze reported data and identify relevant menu items and ingredients.

[0499] Output: Parsed allergic reaction data and associated menu information.

[0500] Step 8:

[0501] The server shares the analysis results with the medical institution.

[0502] Input: Allergic reaction data analyzed in step 7.

[0503] Data processing: Converting data into a format that is easy for medical institutions to read.

[0504] Output: Allergic reaction data sent to healthcare provider.

[0505] The above are specific processing steps based on the application example.

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] MODE FOR CARRYING OUT THE INVENTION

[0523] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports safe meal choices for users with food allergies. It also enables information sharing and rapid response in the event of an allergic reaction. The system's program processing is explained below in natural language.

[0524] Basic system configuration

[0525] This system is mainly composed of three main components: a server, a user's device, and a database. The server plays a central role, while the user's device acts as an interface for obtaining information. The database stores and manages all allergen information.

[0526] Program processing flow

[0527] 1. Data Collection Phase

[0528] Obtaining store information

[0529] The server automatically collects menu information from the restaurant's website and dedicated API. This process covers all menu information that the restaurant has made public.

[0530] For example, restaurant A collects the menu information (menu name, ingredient list, allergen ingredients) provided on its website.

[0531] Analysis of allergen information

[0532] The server analyzes the collected menu information and uses an AI model to identify any allergen ingredients it contains.

[0533] For example, for a menu item called "chocolate cake," the AI ​​model will identify allergens such as "dairy products" and "nuts" from the ingredient list.

[0534] 2. Saving and updating to the database

[0535] Conservation

[0536] After the allergen information is identified, the server stores this information in a database.

[0537] Each menu item is linked to ingredient information and allergen information, allowing users to refer to it later.

[0538] Regular data updates

[0539] The server periodically recollects store information and updates the database, ensuring that allergen information is always up-to-date.

[0540] 3. Providing Information to End Users

[0541] Obtaining allergen information

[0542] The user's terminal accesses the server and requests allergen information for a specific store or menu item.

[0543] For example, if user A searches for "dairy-free menu at restaurant B," the server will return relevant information.

[0544] User interface display

[0545] The user's device displays the allergen information obtained from the server on the user interface, helping the user select a safe menu.

[0546] 4. Reporting and dealing with allergic reactions

[0547] Response reporting

[0548] If a user experiences an allergic reaction after eating, they can use a dedicated application to report the information to the server.

[0549] For example, user B reports that "I developed hives after eating at restaurant C."

[0550] Analysis and collaboration

[0551] The server analyzes reported allergic reaction information to identify whether a particular menu item or ingredient may be the cause.

[0552] If necessary, the analysis results will be shared with the patient's primary care physician to assist in prompt response.

[0553] The system of the present invention provides an environment in which users can choose meals with peace of mind when eating out or eating at home. It also enables prompt and appropriate response in the event of an allergic reaction, improving the quality of life for people with food allergies.

[0554] The processing flow will be explained below.

[0555] Step 1: Data collection

[0556] The server collects menu information from each store's website or dedicated API.

[0557] The server temporarily stores the collected data for analysis.

[0558] Step 2: Allergen information analysis

[0559] The server analyzes the collected menu information using an AI model to identify allergen ingredients.

[0560] Analysis uses techniques such as natural language processing and image analysis.

[0561] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[0562] Step 3: Saving to the database

[0563] The server stores the analysis results in a database.

[0564] Ingredient information and allergen information are stored linked to each menu item.

[0565] Step 4: Regularly update your data

[0566] The server recollects new menu information and change information from the stores according to a preset schedule.

[0567] The server analyzes the new information collected and updates the database.

[0568] This ensures that the latest allergen information is always maintained.

[0569] Step 5: Obtain allergen information

[0570] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[0571] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0572] Step 6: Viewing information

[0573] The user's terminal displays the allergen information received from the server on a user interface.

[0574] The user can select a safe menu based on the displayed information.

[0575] Step 7: Report an allergic reaction

[0576] If a user experiences an allergic reaction after eating, the user reports the information to the server using a dedicated application.

[0577] For example, enter a specific symptom such as "I experienced difficulty breathing 20 minutes after eating."

[0578] Step 8: Analyze the reported information

[0579] The server analyzes the allergic reaction information reported by the user and identifies relevant menu items and ingredients.

[0580] The server may issue an alert based on the analysis results.

[0581] Step 9: Linking with medical institutions

[0582] The server shares the analysis results with the user's family doctor.

[0583] The server uses the linked information to help doctors respond quickly.

[0584] Step 10: Provide feedback

[0585] The server may notify all users of the collected allergic reaction information as a warning or caution.

[0586] This will help other users avoid similar allergic reactions.

[0587] Example 1

[0588] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] Conventional systems lack sufficient food allergy information, and data collection and analysis are often done manually, resulting in problems with accuracy and speed. Another issue is the difficulty of sharing information and responding immediately when an allergic reaction occurs. This means that people with food allergies cannot enjoy meals safely when eating out or choosing foods, which reduces their quality of life.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0591] In this invention, the server includes means for collecting food allergen information from stores via websites or APIs, means for analyzing the collected food allergen information using a generative AI model to identify allergen components, means for saving the identified allergen components in a relational database, means for periodically updating the allergen information in the database, means for providing the updated allergen information to the user's device, means for receiving reports from users when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with medical institutions. This automates the entire process from collection to analysis, storage, provision, and response, enabling users with food allergies to choose safe and reliable meals.

[0592] A "website" is a collection of document formats for publishing and viewing information on the Internet, created using HTML or other formats.

[0593] "API" stands for Application Programming Interface, an interface that enables communication between different software programs.

[0594] "Food allergen information" refers to information about ingredients contained in food that may cause allergic reactions in certain individuals.

[0595] A "generative AI model" is an artificial intelligence model that has been trained using machine learning or deep learning techniques for a specific task.

[0596] "Analysis" is the process of examining and processing collected data in detail to extract and identify specific information.

[0597] An "allergen" is a specific component in food that may cause an allergic reaction.

[0598] A "relational database" is a database system that represents data in tabular format and manages and manipulates data using the relationships between tables.

[0599] "Update" is the process of replacing existing data or information with new data or information.

[0600] A "terminal" is a device that a user uses to input and check information, including smartphones and personal computers.

[0601] "Reporting" is the act of providing information about a particular event or condition, in this case an allergic reaction.

[0602] "Analysis results" refer to the final information or conclusions obtained during the data analysis process.

[0603] A "medical institution" is a facility where doctors and medical professionals provide medical services, including hospitals and clinics.

[0604] MODE FOR CARRYING OUT THE INVENTION

[0605] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports users with food allergies in making safe meal choices. It also enables information sharing and rapid response in the event of an allergic reaction. The following describes in detail how the system's program processing is implemented.

[0606] Basic system configuration

[0607] This system consists of three main components: a server, a user's device, and a relational database. The server plays a central role, while the user's device acts as an interface for retrieving information. The database stores and manages all allergen information.

[0608] Hardware and Software

[0609] Server: A general server computer with a Linux OS is used.

[0610] Device: The smartphone or computer used by the user.

[0611] Database: Use a relational database such as MySQL or PostgreSQL.

[0612] Software and Libraries:

[0613] Web scraping: Python's Beautiful Soup and Selenium

[0614] Generative AI model: Generative AI using natural language processing technology (e.g., GPT-3)

[0615] Scheduling: Cron Jobs in Linux

[0616] Program processing flow

[0617] 1. Collecting store information

[0618] The server automatically collects menu information from each restaurant's website or through a dedicated API. Information collected includes the menu name, ingredient list, and allergen ingredients. Beautiful Soup and Selenium are used for web scraping.

[0619] Examples:

[0620] The server accesses the restaurant's website and extracts information from the HTML page, such as "Chocolate cake, ingredient list: sugar, flour, dairy, nuts."

[0621] 2. Analysis of allergen information

[0622] The collected data is analyzed by a server using a generative AI model, which uses natural language processing techniques to identify allergens from the ingredient list.

[0623] Examples:

[0624] The server inputs the list of ingredients for "chocolate cake" into the AI ​​model and identifies "dairy" and "nuts" as allergens.

[0625] Example prompt sentence:

[0626] "Collect restaurant menu information and use an AI model to identify ingredients that can identify allergens."

[0627] 3. Saving to the database

[0628] The server stores the identified allergen information in a relational database, linking ingredient information and allergen information for each menu item.

[0629] Examples:

[0630] The server links allergen information for "dairy products" and "nuts" to "chocolate cake" and stores it in a database.

[0631] 4. Regular data updates

[0632] The server periodically recollects store information and updates the database, using a cron job to update the data weekly.

[0633] Examples:

[0634] The server runs web scraping every Sunday to retrieve new menu information and update the database.

[0635] 5. Providing allergen information

[0636] The user's terminal requests allergen information for a specific restaurant or menu from the server based on the user's request. In response to the request, the server retrieves relevant information from the database and provides it to the user's terminal.

[0637] Examples:

[0638] When a user searches for "dairy-free menu items at a specific restaurant," the terminal sends a request to the server, which responds with relevant menu information.

[0639] Example prompt sentence:

[0640] "Please extract the relevant allergen information from the database and provide it to the user."

[0641] 6. Reporting and dealing with allergic reactions

[0642] Users use an application to report allergic reactions and send the information to a server, which analyzes the information, identifies related menu items and ingredients, and contacts a medical institution.

[0643] Examples:

[0644] When a user reports that they broke out in hives at a particular restaurant, the server compares the information with the ingredients, identifies the offending ingredient, and notifies a medical professional.

[0645] This system provides users with an environment where they can choose their meals with peace of mind when eating out, and enables quick and appropriate response in the event of an allergic reaction, improving quality of life.

[0646] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0647] Step 1: Gather store information

[0648] server

[0649] Input: Website URL or API endpoint for each store.

[0650] How it works: The server uses Python's Beautiful Soup and Selenium to perform web scraping. Specifically, it retrieves the HTML of a web page from a specified URL and extracts information such as menu names, ingredient lists, and allergens.

[0651] Data processing: The acquired HTML is analyzed as a DOM tree and the necessary information is extracted in text format.

[0652] Output: A list of extracted menu information.

[0653] Step 2: Analysis of allergen information

[0654] server

[0655] Input: Menu information collected in step 1.

[0656] Operation: The collected menu information is input into the generative AI model and analyzed.

[0657] Data processing: The generative AI model uses natural language processing techniques to identify allergens from an ingredient list.

[0658] Output: A list of menu items that contain the identified allergen ingredients.

[0659] Step 3: Saving to the database

[0660] server

[0661] Input: Allergen information parsed in step 2.

[0662] How it works: The server connects to a relational database such as MySQL or PostgreSQL and stores allergen information.

[0663] Data processing: Ingredient information and allergen information are linked for each menu item and stored in a database.

[0664] Output: Success status of the save operation.

[0665] Step 4: Regularly update your data

[0666] server

[0667] Input: Scheduled to run periodically (Cron job).

[0668] What it does: Uses a cron job to re-run web scraping at scheduled times to retrieve new menu information and update existing data.

[0669] Data processing: Compare new data with existing data and update any changes.

[0670] Output: A list of updated allergen information.

[0671] Step 5: Request Allergen Information

[0672] Terminal

[0673] Input: What the user wants to search for (store name, whether or not it contains allergens, etc.).

[0674] Operation: The device sends an API request to the server based on this input data.

[0675] Data transformation: Constructing request data based on user input.

[0676] Output: Allergen information returned by the server.

[0677] Step 6: Display allergen information

[0678] Terminal

[0679] Input: Allergen information obtained from the server in step 5.

[0680] Operation: The terminal displays the acquired allergen information on the user interface.

[0681] Data processing: Converting data into a display format.

[0682] Output: A display of the menu and its allergen information on the user interface.

[0683] Step 7: Report an allergic reaction

[0684] User

[0685] Input: Detailed information in case of allergic reaction (store name, food name, symptoms, etc.).

[0686] How it works: Using a dedicated application, users enter details of their allergic reaction and send them to a server.

[0687] Data Processing: Converting entered information into a reporting format.

[0688] Output: Notification that report information has been sent to the server.

[0689] Step 8: Analysis and integration of reaction information

[0690] server

[0691] Input: Allergic reaction information reported by the user in step 7.

[0692] How it works: The server parses the reported information and identifies relevant menu items and allergen ingredients.

[0693] Data processing: The reported information is compared with information in the database to identify the causative allergen.

[0694] Output: Analysis results are shared with medical institutions. Details of the analysis results are notified to medical institutions to support a prompt response.

[0695] (Application example 1)

[0696] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] It is difficult to provide an environment where consumers with food allergies can safely choose foods when eating out or purchasing them. In particular, in brick-and-mortar stores, consumers face the challenge of having to choose foods while taking the risk of an allergic reaction, due to a lack of easy access to information about allergens contained in menus and foods. Furthermore, there is a lack of systems for quickly sharing information and taking appropriate measures in the event of an allergic reaction.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0699] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; means for updating the allergen information in the database; means for providing the user with the updated allergen information; means for receiving reports from users when an allergic reaction occurs; means for analyzing the received reports and identifying related menu items and ingredients; means for sharing the analysis results with a family doctor; means for scanning QR codes on food menus with a camera; means for transmitting the scanned information to the server to acquire allergen information; and means for displaying the acquired allergen information on a head-up display. This allows consumers to easily obtain information about allergens contained in foods, enabling safe food selection. It also enables prompt response when an allergic reaction occurs.

[0700] A "store" is a physical location that provides food and merchandise to consumers.

[0701] "Food allergen information" is information about allergen components contained in specific foods.

[0702] "Means of collection" refers to the method or technology used to obtain food allergen information from stores.

[0703] "Means for analyzing" refers to a method or technique for analyzing collected food allergen information and identifying specific allergen components.

[0704] A "database" is an electronic system for systematically storing and managing specific information.

[0705] An "updating means" is a method or technique for replacing existing data with new information.

[0706] "Means for providing to users" refers to the methods or technologies for communicating the collected and analyzed allergen information to users.

[0707] The "means for receiving a report from a user when an allergic reaction occurs" is a method or technology for receiving information about an allergic reaction from a user.

[0708] "Means for analyzing reported information" refers to a method or technique for analyzing the received information about an allergic reaction and identifying its cause.

[0709] "Means for linking to primary care physician" refers to the method or technology for communicating analysis results to healthcare providers.

[0710] A "food menu QR code" is a two-dimensional barcode that contains information about a specific food menu item.

[0711] "Camera scanning means" refers to a method or technology for capturing information in a QR code using a camera.

[0712] "Means for sending information to a server" refers to a method or technique for transmitting acquired data to a server.

[0713] "Means for obtaining allergen information" refers to a method or technology for obtaining the necessary allergen information from the server.

[0714] "Means for displaying on a head-up display" refers to a method or technology for visually displaying the acquired allergen information.

[0715] The system of the present invention provides support for users to safely obtain food allergen information in physical stores and avoid food allergies. The system is mainly composed of a server, a user device (such as smart glasses), a camera, and a database.

[0716] Basic configuration

[0717] 1. Data Collection and Analysis:

[0718] The server uses a camera to read the QR code on the food menu provided by the physical store. The data read by the camera is sent to the server, where an AI model runs to analyze allergen information. The collected food allergen information is analyzed, and the identified allergen ingredients are stored in a database.

[0719] 2. Data Management:

[0720] The database stores identified allergens and is updated regularly. This database management is important to ensure that users are always provided with the most up-to-date allergen information.

[0721] 3. User Interface:

[0722] Users use a user device such as smart glasses to scan QR codes on store menus or food shelves. By scanning the QR code, the menu ID is sent to the server, and the collected allergen information is retrieved. The retrieved allergen information is visually displayed on the head-up display (HUD) of the smart glasses.

[0723] 4. Allergic Reactions:

[0724] If a user experiences an allergic reaction after eating, they can report that information to the server through a dedicated application. The server analyzes the reported information and identifies the relevant menu items and ingredients. If necessary, the analysis results are shared with the user's doctor, allowing for prompt treatment.

[0725] Hardware and software used

[0726] Hardware:

[0727] Smart glasses (e.g. Google Glass, Vuzix Blade): Equipped with a HUD and camera.

[0728] Camera: Used to read QR codes.

[0729] software:

[0730] OpenCV: Processes camera images.

[0731] pyzbar: A library for decoding QR codes.

[0732] requests: Used to send HTTP requests to the server.

[0733] Database management software: stores and manages allergen information.

[0734] Specific examples

[0735] When a user eats at a restaurant, they scan a QR code placed on the table or menu with the smart glasses. For example, the user can check allergen information such as "This menu contains dairy products." If an allergic reaction occurs, the user can report through the application that they "developed hives after eating at a restaurant," and the server will analyze the information.

[0736] Prompt Sentence Examples

[0737] "Scan the QR code displayed in the restaurant with your smart glasses and you'll be able to see allergen information."

[0738] This invention allows consumers with food allergies to eat out safely and make safe food choices, and also allows for quick response if an allergic reaction occurs.

[0739] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0740] Step 1:

[0741] A user scans a QR code on a food menu at a physical store with the camera in their smart glasses. The camera in the smart glasses reads the QR code and obtains its data (menu ID). The input is the QR code, and the output is the menu ID.

[0742] Step 2:

[0743] The smart glasses send the acquired menu ID to the server using an HTTP request. The input is the menu ID, and the output is the request sent to the server.

[0744] Step 3:

[0745] The server searches the database based on the received menu ID and retrieves the corresponding allergen information. The input is the menu ID and the output is the allergen information. The server identifies the information in the database and extracts the necessary data.

[0746] Step 4:

[0747] The server returns the acquired allergen information to the smart glasses. The response is an HTTP response. The input is the allergen information acquired from the database, and the output is the response sent to the smart glasses.

[0748] Step 5:

[0749] The smart glasses receive allergen information from the server and display it on a HUD (head-up display). The user can visually check the allergen information. The input is the allergen information, and the output is what is displayed on the HUD. For example, it might say, "This menu contains dairy products."

[0750] Step 6:

[0751] If a user experiences an allergic reaction after eating, they report the information to the server using a dedicated application. The input is detailed information about the allergic reaction, and the output is a report to the server.

[0752] Step 7:

[0753] The server analyzes the reported information and identifies the relevant menu items and ingredients. The input is the allergic reaction report, and the output is the specific information for the relevant menu items and ingredients. The server then searches the database again based on the report and derives the analysis results.

[0754] Step 8:

[0755] The server will share the analysis results with the patient's primary care physician as needed. This sharing is done via email or API. The input is the analysis results, and the output is shared with the patient's primary care physician. For example, it will send information that "the user has had an allergic reaction to dairy products."

[0756] The above processing steps enable users to safely obtain food allergen information at physical stores, and also create a system that allows for quick response in the event of an allergic reaction.

[0757] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0758] MODE FOR CARRYING OUT THE INVENTION

[0759] This invention is a system that allows users to safely select meals when eating out or taking out a meal. In particular, it combines a function to analyze and provide food allergen information collected from restaurants with an emotion engine that recognizes the user's emotions and makes suggestions related to meal selection. The specific operation of this system is described by explaining the program processing in natural language and using concrete examples.

[0760] Basic system configuration

[0761] The system consists of the following main components:

[0762] 1. Server

[0763] 2. User's device (smartphone, PC, etc.)

[0764] 3. Database

[0765] 4. Emotion Engine

[0766] The server plays a central role, the user's device acts as an interface to retrieve information, the database stores and manages allergen information, and the emotion engine analyzes the user's emotions and makes suggestions based on them.

[0767] Program processing flow

[0768] 1. Data Collection Phase

[0769] The server collects menu information from each store's website or dedicated API.

[0770] The collected data includes menu names, ingredient lists, allergen information, and more.

[0771] Example: Collect information about "chocolate cake" from Restaurant A and temporarily store it.

[0772] 2. Allergen information analysis phase

[0773] The server analyzes the collected menu information and uses an AI model to identify allergen ingredients.

[0774] Identified allergen components are recorded in a database.

[0775] Example: Identifying that "chocolate cake" contains "dairy" and "nuts."

[0776] 3. Saving to the database and updating phase

[0777] The server stores the analysis results in a database.

[0778] Ingredient information and allergen information are linked and saved for each menu item.

[0779] Example: Save allergen information for "chocolate cake" in a database.

[0780] 4. Information provision phase for end users

[0781] The user's terminal connects to the server and requests allergen information for the store and menu.

[0782] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0783] Example: When User A searches for "nut-free menu at Restaurant B," relevant menu information is provided.

[0784] 5. Emotion engine proposal phase

[0785] The server analyzes the emotion data acquired from the user's terminal.

[0786] The emotion engine makes suggestions related to meal selection based on the analyzed emotions.

[0787] Example: If user A is feeling "frustrated," suggest a menu containing ingredients that have a relaxing effect.

[0788] 6. Allergic Reaction Reporting Phase

[0789] If a user experiences an allergic reaction after eating, they enter that information into a dedicated app and report it to the server.

[0790] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[0791] Example: If user B reports that "hives appeared 20 minutes after eating at restaurant C," analyze that information.

[0792] 7. Collaboration with medical institutions phase

[0793] The server then shares the analysis results with the patient's primary care physician.

[0794] Assist doctors in responding quickly when necessary.

[0795] Example: User B's doctor is notified that "nut ingredients are suspected to be the cause of the hives."

[0796] The system of this invention provides a comprehensive solution for users with food allergies to safely enjoy eating out or taking out meals. In addition, the introduction of an emotion engine enables optimal meal selection based on the user's emotional state, contributing to improved user satisfaction.

[0797] The processing flow will be explained below.

[0798] MODE FOR CARRYING OUT THE INVENTION (SPECIFIC PROCESSING STEPS)

[0799] Basic system configuration

[0800] This system consists of a server, a user's terminal, a database, and an emotion engine.

[0801] Specific processing flow of the program

[0802] Step 1: Get store information

[0803] The server periodically collects menu information from each store's website or dedicated API.

[0804] The data collected includes menu names, ingredient lists, and allergen information.

[0805] Step 2: Analysis of allergen information

[0806] The server uses AI to analyze the collected data and identify the allergen ingredients contained in each menu item.

[0807] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[0808] Step 3: Saving to the database

[0809] The server stores the identified allergen component information in a database.

[0810] By storing ingredient and allergen information for each menu item, users can refer to it later.

[0811] Step 4: Regularly update your data

[0812] The server recollects the latest store information at a set time (for example, late at night every day), compares it with the previous data, and updates it.

[0813] If changes are detected, the database is updated to keep the information up to date.

[0814] Step 5: Obtain allergen information

[0815] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[0816] For example, User A searches for "dairy-free menu items at Restaurant B."

[0817] Step 6: Viewing information

[0818] In response to a user request, the server extracts the relevant allergen information from the database and returns it to the user's terminal.

[0819] The user's terminal displays the received information on a user interface, for example, displaying a list of dairy-free menu items.

[0820] Step 7: Recognize emotions

[0821] The user's device uses built-in cameras and sensors to collect emotional data from the user's face and voice.

[0822] For example, when user A is tired, that emotion is recognized.

[0823] Step 8: Send and analyze emotion data

[0824] The user's device transmits the collected emotion data to the server.

[0825] The server's emotion engine analyzes the transmitted emotion data to determine the user's current emotional state.

[0826] Step 9: Emotion-Based Suggestions

[0827] The server proposes the most suitable menu to the user based on the analysis results.

[0828] For example, if the user is feeling stressed, menus that have a relaxing effect are preferentially suggested.

[0829] Step 10: Report an Allergic Reaction

[0830] If a user experiences an allergic reaction after eating, the user uses a dedicated application to report the information to the server.

[0831] For example, user B reports that "dyspnea occurs 20 minutes after eating."

[0832] Step 11: Analyze the reported information

[0833] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[0834] If necessary, the identified information is reflected in the database.

[0835] Step 12: Linking with medical institutions

[0836] The server shares the analysis results with the user's family doctor to help them respond quickly.

[0837] For example, user B's doctor may be notified that "nuts are suspected to be the cause of the breathing difficulties."

[0838] The system of this invention not only allows users to make safe food choices based on the latest allergen information, but also provides optimal suggestions based on the emotion engine.It also enables rapid response in the unlikely event of an allergic reaction, thereby improving the quality of life for people with food allergies.

[0839] Example 2

[0840] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] Previously, when eating out or taking home meals, the problem was that food allergen information was not sufficiently obtained, and meal suggestions based on the user's feelings were not adequately provided.In addition, there was a lack of prompt identification of information and coordination with medical institutions when an allergic reaction occurred, making it an issue to increase user safety and satisfaction.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0843] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and using a natural language processing model to identify allergen components; means for saving the identified allergen components in a database; means for updating the allergen information in the database; means for providing the updated allergen information to the user; means for analyzing the user's emotional data; means for making suggestions based on the analyzed emotional data; means for receiving reports from the user when an allergic reaction occurs; means for analyzing the received report information and identifying related menu items and ingredients; and means for sharing the analysis results with a medical institution. This allows the user to safely select meals, reducing the risk of food allergies and receiving meal suggestions tailored to their emotional state. Furthermore, this allows for a rapid response when an allergic reaction occurs, increasing the user's sense of security.

[0844] "Means of collecting food allergen information from stores" refers to the technology and methods for obtaining menu information from store websites or dedicated APIs.

[0845] "Means of using a natural language processing model to analyze collected food allergen information and identify allergen ingredients" refers to a method of analyzing acquired menu information using natural language processing technology and identifying specific allergen ingredients.

[0846] "Means for storing identified allergen components in a database" refers to the technology or method for recording data on allergen components identified through analysis in a database.

[0847] "Means for updating allergen information in the database" refers to techniques or methods for rewriting allergen information already stored in the database with new information.

[0848] "Means for providing updated allergen information to a user" refers to techniques or methods for providing revised allergen information to a user upon request.

[0849] "Means for analyzing user emotion data" refers to techniques and methods for analyzing emotion data provided by a user and understanding its meaning and state.

[0850] "Means for making suggestions based on analyzed emotional data" refers to techniques or methods for generating meal selection or other suggestions based on a user's emotional state.

[0851] "Means for receiving a report from a user when an allergic reaction occurs" refers to an interface for a user to report a food allergic reaction and the technology or method for receiving that report.

[0852] "Means for analyzing received report information and identifying related menu items or ingredients" refers to technologies and methods for analyzing a user's report of an allergic reaction and identifying the menu item or ingredient that caused it.

[0853] "Means of sharing analysis results with medical institutions" refers to technologies and methods for notifying or sharing the analysis results of allergic reactions with medical institutions.

[0854] This invention is a system that enables users to make safe and effective food choices when eating out or taking home meals. This system combines a function to analyze and provide food allergen information collected from restaurants, and an emotion engine that recognizes the user's emotions and makes suggestions related to food selection.

[0855] Hardware and Software

[0856] Hardware: Servers (e.g., cloud computing services), user devices (e.g., smartphones, PCs, etc.)

[0857] software:

[0858] Database management system (e.g. MySQL)

[0859] Sentiment analysis engine (e.g. Google Cloud Natural Language API)

[0860] Website scraping tools (e.g. Beautiful Soup)

[0861] Basic system configuration

[0862] 1. Server: Serves as the central processing unit, collecting, analyzing, storing, and providing various data.

[0863] 2. User's terminal: Acts as an interface for users to obtain and input information.

[0864] 3. Database: Stores and manages allergen information and analysis data.

[0865] 4. Emotion Engine: Analyzes user emotion data and makes suggestions based on it.

[0866] Operation overview

[0867] The operation of this system is as follows.

[0868] Data collection and analysis

[0869] The server collects menu information from restaurant websites and dedicated APIs. This collection is done using the web scraping tool Beautiful Soup. The collected menu information includes the menu name, ingredient list, and allergen information. For example, the server collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[0870] The server then analyzes the collected information using the Google Cloud Natural Language API to identify allergens in each menu item, and the analysis results are stored in a database.

[0871] Providing information

[0872] When a user requests specific allergen information through their device, the server retrieves the relevant information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[0873] Emotion data analysis and proposals

[0874] The server analyzes the emotional data obtained from the user's device using the Google Cloud Natural Language API. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[0875] Reporting allergic reactions and contacting medical institutions

[0876] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. The analysis results are provided to medical institutions as needed. For example, if User B reports that "hives developed 20 minutes after eating at Restaurant D," the server analyzes the information and notifies the medical institution that "nut ingredients may be the cause."

[0877] Prompt Sentence Examples

[0878] 1. Example 1: User A requests "I have a nut allergy, so I would like to find nut-free menu items at Restaurant C."

[0879] Example prompt: "What are the nut-free menu items at Restaurant C?"

[0880] 2. Example 2: User B requests, "I'm feeling stressed today, so please suggest a meal that will help me relax."

[0881] Example prompt: "I'm feeling stressed today, so please suggest something relaxing."

[0882] 3. Example 3: User C requests, "I had an allergic reaction after eating, so I want to report it and identify the cause."

[0883] Example prompt: "I broke out in hives after eating at Restaurant D. Please identify the cause of my allergy."

[0884] As described above, this system is designed to reduce the risk of food allergies and provide meal suggestions based on the user's emotional state.

[0885] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0886] Step 1:

[0887] The server collects menu information from restaurant websites and dedicated APIs. Using the web scraping tool Beautiful Soup, it obtains restaurant menu items, ingredient lists, and allergen information. For example, it collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[0888] Input: Store website URL or API endpoint.

[0889] Output: Data including menu name, ingredient list, and allergen information.

[0890] Step 2:

[0891] The server uses the Google Cloud Natural Language API to analyze the collected menu information. Here, natural language processing technology is used to analyze the ingredient information and identify allergens. For example, it can identify that "chocolate cake" contains "dairy products" and "nuts."

[0892] Input: Menu name, ingredient list, and allergen information collected in Step 1.

[0893] Output: Identified allergen ingredient information.

[0894] Step 3:

[0895] The server stores the identified allergen information in a database. In this case, a MySQL database is used to link ingredient information and allergen information for each menu item. For example, information for "Chocolate Cake" is recorded as "ID: 1234, Ingredients: Sugar, Cocoa, Dairy; Allergens: Dairy, Nuts."

[0896] Input: Allergen ingredient information identified in Step 2.

[0897] Output: Allergen information stored in a database.

[0898] Step 4:

[0899] A user's device sends a request for specific allergen information to the server. The server extracts the relevant allergen information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[0900] Input: A request from the user requesting allergen information.

[0901] Output: A list of relevant allergen information.

[0902] Step 5:

[0903] The server uses the Google Cloud Natural Language API to analyze the emotion data obtained from the user's device. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[0904] Input: User emotion data.

[0905] Output: Sentiment analysis results and meal suggestions based on them.

[0906] Step 6:

[0907] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. For example, if User C reports that "hives occurred 20 minutes after eating at Restaurant D," the server analyzes the information and identifies the specific ingredients (e.g., "nuts").

[0908] Input: User's report of an allergic reaction.

[0909] Output: Parsed allergen ingredients and associated menu information.

[0910] Step 7:

[0911] The server transfers the analysis results to a system linked to a medical institution. Based on the information notified to the medical institution, a prompt response can be made. For example, a notification could be sent to User C's family doctor stating that "nut ingredients are suspected to be the cause of the hives."

[0912] Input: Allergy analysis results.

[0913] Output: Report information to medical institutions.

[0914] (Application example 2)

[0915] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0916] Conventional systems cannot completely avoid the risk of food allergies when eating out or taking home meals, and do not suggest food choices based on the user's emotions. Therefore, a system that combines the analysis and provision of allergy information with emotion-based meal suggestions is needed.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0918] In this invention, the server includes means for collecting food allergen information from stores, means for analyzing the collected food allergen information and identifying allergen components, means for saving the identified allergen components in a database, means for updating the allergen information in the database, means for providing the updated allergen information to the user, means for recognizing the user's emotions and making suggestions related to meal selection based on the emotions, means for receiving reports from the user when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with a medical institution. This makes it possible to reduce the risk of food allergies and support meal selection based on the user's emotions.

[0919] The "means for collecting food allergen information from stores" refers to a combination of software and hardware for collecting menu and ingredient information from each store.

[0920] The "means for analyzing collected food allergen information and identifying allergen components" refers to algorithms and programs for analyzing collected data and identifying specific allergen components.

[0921] "Means for storing identified allergen components in a database" refers to a mechanism for storing allergen information and subsequently registering it in a database in a searchable format.

[0922] "Means for updating allergen information in the database" refers to a method for keeping existing allergen information up to date with new data.

[0923] The "means for providing updated allergen information to the user" is a system for notifying or displaying the latest allergen information to the user.

[0924] "Means for recognizing the user's emotions and making suggestions related to meal selection based on those emotions" refers to technology that analyzes the user's emotions from their facial expressions and voice and makes meal suggestions accordingly.

[0925] The "means for receiving a report from the user when an allergic reaction occurs" is a function for the user to report to the system any allergic reaction they have experienced.

[0926] The "means for analyzing the received report information and identifying the related menu or ingredient" is a process for analyzing the allergic reaction report from the user and identifying the menu or ingredient that caused it.

[0927] "Means for sharing analysis results with medical institutions" is a function for sending the analysis results of allergic reactions to the user's regular medical institution.

[0928] The system of the present invention is designed to suggest safe and comfortable meal options by combining food allergy and emotion recognition. Specifically, it is implemented using the following means.

[0929] 1. How to collect allergen information

[0930] The server collects menu information from each restaurant's website and dedicated API. The collected data includes the menu name, ingredient list, allergen information, etc. This makes it possible to always maintain the latest allergen information.

[0931] 2. Methods for analyzing allergen components

[0932] The server analyzes the collected menu information, specifically identifying allergens using an AI model and storing them in a database, where they can be later accessed by users.

[0933] 3. How to store and update allergen information

[0934] The identified allergens are stored in a database that is updated regularly, allowing users to access the most up-to-date allergen information.

[0935] 4. Means of Providing Allergen Information to Users

[0936] The user's device (such as a smartphone or PC) connects to the server and requests allergen information for the restaurant and menu. The server extracts the relevant allergen information from the database and returns it to the user's device.

[0937] 5. Emotion Recognition Suggestion Method

[0938] The emotion engine analyzes emotion data acquired from the user's device. It recognizes the user's emotions using a camera or voice input device, and makes suggestions related to meal selection based on the analysis results. These suggestions are intended to provide the optimal meal based on the user's emotional state.

[0939] 6. Allergic Reaction Reporting Method

[0940] If a user experiences an allergic reaction after eating, they can report the information to the server through a dedicated application. The server then analyzes the reported information and identifies the relevant menu items and ingredients.

[0941] 7. Linking to medical institutions

[0942] The server then sends the analysis results to the user's primary care physician, helping the physician to respond promptly if necessary.

[0943] Specific examples

[0944] For example, when a user visits "Restaurant X," they can use this system to search for "nut-free" menu items, and the camera can recognize their emotions to suggest menu items that include relaxing ingredients. If an allergic reaction occurs after eating, they can report this information to the app, which will promptly notify a medical institution.

[0945] Prompt Sentence Examples

[0946] An example of a prompt to be input to a generative AI model could be the following text:

[0947] "Design a smartphone app that uses an emotion engine to recognize the user's emotions and suggest menu items that match those emotions. This app also has the functionality to collect restaurant menu information and analyze allergens."

[0948] The above is an embodiment of the present invention.

[0949] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0950] Step 1:

[0951] The server collects food allergen information from stores.

[0952] Input: Menu information obtained from each store's website or dedicated API.

[0953] Data processing: Convert menu names, ingredient lists, and allergen information into an analyzable format.

[0954] Output: Menu information formatted in a format suitable for parsing.

[0955] Step 2:

[0956] The server analyzes the food allergen information collected and identifies the allergen components.

[0957] Input: Menu information collected and formatted in step 1.

[0958] Data processing: Using an AI model, allergen ingredients in each menu item are identified.

[0959] Output: Menu information linked to identified allergen ingredients.

[0960] Step 3:

[0961] The server stores the identified allergen components in a database.

[0962] Input: Menu information containing the allergen ingredients identified in step 2.

[0963] Data processing: Converting data into a format suitable for storing in a database.

[0964] Output: Menu and allergen information stored in a database.

[0965] Step 4:

[0966] The device connects to the server and requests allergen information for the store and menu.

[0967] Input: Store and menu information requested by the user.

[0968] Data processing: Extract the necessary menu information from the database.

[0969] Output: Providing relevant allergen information to the user's device.

[0970] Step 5:

[0971] The terminal acquires the user's emotional data, which is then analyzed by the server using an emotion engine.

[0972] Input: User's facial expressions and voice data.

[0973] Data processing: Analyze data using an emotion engine to identify emotional states.

[0974] Output: The analyzed emotional state of the user.

[0975] Step 6:

[0976] The server makes suggestions related to meal selection based on the analyzed emotional state.

[0977] Input: User's emotional state and menu information identified in step 5.

[0978] Data processing: Filter the optimal menu according to emotions and generate a list of suggestions.

[0979] Output: The optimal menu list proposed to the user's device.

[0980] Step 7:

[0981] The user reports an allergic reaction and the server receives the reported information.

[0982] Input: Allergic reaction report data from users.

[0983] Data processing: Analyze reported data and identify relevant menu items and ingredients.

[0984] Output: Parsed allergic reaction data and associated menu information.

[0985] Step 8:

[0986] The server shares the analysis results with the medical institution.

[0987] Input: Allergic reaction data analyzed in step 7.

[0988] Data processing: Converting data into a format that is easy for medical institutions to read.

[0989] Output: Allergic reaction data sent to healthcare provider.

[0990] The above are specific processing steps based on the application example.

[0991] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0992] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0993] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0994] [Third embodiment]

[0995] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0996] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0997] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0998] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0999] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1000] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1001] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1002] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1003] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1004] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1005] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1006] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1007] MODE FOR CARRYING OUT THE INVENTION

[1008] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports safe meal choices for users with food allergies. It also enables information sharing and rapid response in the event of an allergic reaction. The system's program processing is explained below in natural language.

[1009] Basic system configuration

[1010] This system is mainly composed of three main components: a server, a user's device, and a database. The server plays a central role, while the user's device acts as an interface for obtaining information. The database stores and manages all allergen information.

[1011] Program processing flow

[1012] 1. Data Collection Phase

[1013] Obtaining store information

[1014] The server automatically collects menu information from the restaurant's website and dedicated API. This process covers all menu information that the restaurant has made public.

[1015] For example, restaurant A collects the menu information (menu name, ingredient list, allergen ingredients) provided on its website.

[1016] Analysis of allergen information

[1017] The server analyzes the collected menu information and uses an AI model to identify any allergen ingredients it contains.

[1018] For example, for a menu item called "chocolate cake," the AI ​​model will identify allergens such as "dairy products" and "nuts" from the ingredient list.

[1019] 2. Saving and updating to the database

[1020] Conservation

[1021] After the allergen information is identified, the server stores this information in a database.

[1022] Each menu item is linked to ingredient information and allergen information, allowing users to refer to it later.

[1023] Regular data updates

[1024] The server periodically recollects store information and updates the database, ensuring that allergen information is always up-to-date.

[1025] 3. Providing Information to End Users

[1026] Obtaining allergen information

[1027] The user's terminal accesses the server and requests allergen information for a specific store or menu item.

[1028] For example, if user A searches for "dairy-free menu at restaurant B," the server will return relevant information.

[1029] User interface display

[1030] The user's device displays the allergen information obtained from the server on the user interface, helping the user select a safe menu.

[1031] 4. Reporting and dealing with allergic reactions

[1032] Response reporting

[1033] If a user experiences an allergic reaction after eating, they can use a dedicated application to report the information to the server.

[1034] For example, user B reports that "I developed hives after eating at restaurant C."

[1035] Analysis and collaboration

[1036] The server analyzes reported allergic reaction information to identify whether a particular menu item or ingredient may be the cause.

[1037] If necessary, the analysis results will be shared with the patient's primary care physician to assist in prompt response.

[1038] The system of the present invention provides an environment in which users can choose meals with peace of mind when eating out or eating at home. It also enables prompt and appropriate response in the event of an allergic reaction, improving the quality of life for people with food allergies.

[1039] The processing flow will be explained below.

[1040] Step 1: Data collection

[1041] The server collects menu information from each store's website or dedicated API.

[1042] The server temporarily stores the collected data for analysis.

[1043] Step 2: Allergen information analysis

[1044] The server analyzes the collected menu information using an AI model to identify allergen ingredients.

[1045] Analysis uses techniques such as natural language processing and image analysis.

[1046] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[1047] Step 3: Saving to the database

[1048] The server stores the analysis results in a database.

[1049] Ingredient information and allergen information are stored linked to each menu item.

[1050] Step 4: Regularly update your data

[1051] The server recollects new menu information and change information from the stores according to a preset schedule.

[1052] The server analyzes the new information collected and updates the database.

[1053] This ensures that the latest allergen information is always maintained.

[1054] Step 5: Obtain allergen information

[1055] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[1056] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1057] Step 6: Viewing information

[1058] The user's terminal displays the allergen information received from the server on a user interface.

[1059] The user can select a safe menu based on the displayed information.

[1060] Step 7: Report an allergic reaction

[1061] If a user experiences an allergic reaction after eating, the user reports the information to the server using a dedicated application.

[1062] For example, enter a specific symptom such as "I experienced difficulty breathing 20 minutes after eating."

[1063] Step 8: Analyze the reported information

[1064] The server analyzes the allergic reaction information reported by the user and identifies relevant menu items and ingredients.

[1065] The server may issue an alert based on the analysis results.

[1066] Step 9: Linking with medical institutions

[1067] The server shares the analysis results with the user's family doctor.

[1068] The server uses the linked information to help doctors respond quickly.

[1069] Step 10: Provide feedback

[1070] The server may notify all users of the collected allergic reaction information as a warning or caution.

[1071] This will help other users avoid similar allergic reactions.

[1072] Example 1

[1073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1074] Conventional systems lack sufficient food allergy information, and data collection and analysis are often done manually, resulting in problems with accuracy and speed. Another issue is the difficulty of sharing information and responding immediately when an allergic reaction occurs. This means that people with food allergies cannot enjoy meals safely when eating out or choosing foods, which reduces their quality of life.

[1075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1076] In this invention, the server includes means for collecting food allergen information from stores via websites or APIs, means for analyzing the collected food allergen information using a generative AI model to identify allergen components, means for saving the identified allergen components in a relational database, means for periodically updating the allergen information in the database, means for providing the updated allergen information to the user's device, means for receiving reports from users when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with medical institutions. This automates the entire process from collection to analysis, storage, provision, and response, enabling users with food allergies to choose safe and reliable meals.

[1077] A "website" is a collection of document formats for publishing and viewing information on the Internet, created using HTML or other formats.

[1078] "API" stands for Application Programming Interface, an interface that enables communication between different software programs.

[1079] "Food allergen information" refers to information about ingredients contained in food that may cause allergic reactions in certain individuals.

[1080] A "generative AI model" is an artificial intelligence model that has been trained using machine learning or deep learning techniques for a specific task.

[1081] "Analysis" is the process of examining and processing collected data in detail to extract and identify specific information.

[1082] An "allergen" is a specific component in food that may cause an allergic reaction.

[1083] A "relational database" is a database system that represents data in tabular format and manages and manipulates data using the relationships between tables.

[1084] "Update" is the process of replacing existing data or information with new data or information.

[1085] A "terminal" is a device that a user uses to input and check information, including smartphones and personal computers.

[1086] "Reporting" is the act of providing information about a particular event or condition, in this case an allergic reaction.

[1087] "Analysis results" refer to the final information or conclusions obtained during the data analysis process.

[1088] A "medical institution" is a facility where doctors and medical professionals provide medical services, including hospitals and clinics.

[1089] MODE FOR CARRYING OUT THE INVENTION

[1090] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports users with food allergies in making safe meal choices. It also enables information sharing and rapid response in the event of an allergic reaction. The following describes in detail how the system's program processing is implemented.

[1091] Basic system configuration

[1092] This system consists of three main components: a server, a user's device, and a relational database. The server plays a central role, while the user's device acts as an interface for retrieving information. The database stores and manages all allergen information.

[1093] Hardware and Software

[1094] Server: A general server computer with a Linux OS is used.

[1095] Device: The smartphone or computer used by the user.

[1096] Database: Use a relational database such as MySQL or PostgreSQL.

[1097] Software and Libraries:

[1098] Web scraping: Python's Beautiful Soup and Selenium

[1099] Generative AI model: Generative AI using natural language processing technology (e.g., GPT-3)

[1100] Scheduling: Cron Jobs in Linux

[1101] Program processing flow

[1102] 1. Collecting store information

[1103] The server automatically collects menu information from each restaurant's website or through a dedicated API. Information collected includes the menu name, ingredient list, and allergen ingredients. Beautiful Soup and Selenium are used for web scraping.

[1104] Examples:

[1105] The server accesses the restaurant's website and extracts information from the HTML page, such as "Chocolate cake, ingredient list: sugar, flour, dairy, nuts."

[1106] 2. Analysis of allergen information

[1107] The collected data is analyzed by a server using a generative AI model, which uses natural language processing techniques to identify allergens from the ingredient list.

[1108] Examples:

[1109] The server inputs the list of ingredients for "chocolate cake" into the AI ​​model and identifies "dairy" and "nuts" as allergens.

[1110] Example prompt sentence:

[1111] "Collect restaurant menu information and use an AI model to identify ingredients that can identify allergens."

[1112] 3. Saving to the database

[1113] The server stores the identified allergen information in a relational database, linking ingredient information and allergen information for each menu item.

[1114] Examples:

[1115] The server links allergen information for "dairy products" and "nuts" to "chocolate cake" and stores it in a database.

[1116] 4. Regular data updates

[1117] The server periodically recollects store information and updates the database, using a cron job to update the data weekly.

[1118] Examples:

[1119] The server runs web scraping every Sunday to retrieve new menu information and update the database.

[1120] 5. Providing allergen information

[1121] The user's terminal requests allergen information for a specific restaurant or menu from the server based on the user's request. In response to the request, the server retrieves relevant information from the database and provides it to the user's terminal.

[1122] Examples:

[1123] When a user searches for "dairy-free menu items at a specific restaurant," the terminal sends a request to the server, which responds with relevant menu information.

[1124] Example prompt sentence:

[1125] "Please extract the relevant allergen information from the database and provide it to the user."

[1126] 6. Reporting and dealing with allergic reactions

[1127] Users use an application to report allergic reactions and send the information to a server, which analyzes the information, identifies related menu items and ingredients, and contacts a medical institution.

[1128] Examples:

[1129] When a user reports that they broke out in hives at a particular restaurant, the server compares the information with the ingredients, identifies the offending ingredient, and notifies a medical professional.

[1130] This system provides users with an environment where they can choose their meals with peace of mind when eating out, and enables quick and appropriate response in the event of an allergic reaction, improving quality of life.

[1131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1132] Step 1: Gather store information

[1133] server

[1134] Input: Website URL or API endpoint for each store.

[1135] How it works: The server uses Python's Beautiful Soup and Selenium to perform web scraping. Specifically, it retrieves the HTML of a web page from a specified URL and extracts information such as menu names, ingredient lists, and allergens.

[1136] Data processing: The acquired HTML is analyzed as a DOM tree and the necessary information is extracted in text format.

[1137] Output: A list of extracted menu information.

[1138] Step 2: Analysis of allergen information

[1139] server

[1140] Input: Menu information collected in step 1.

[1141] Operation: The collected menu information is input into the generative AI model and analyzed.

[1142] Data processing: The generative AI model uses natural language processing techniques to identify allergens from an ingredient list.

[1143] Output: A list of menu items that contain the identified allergen ingredients.

[1144] Step 3: Saving to the database

[1145] server

[1146] Input: Allergen information parsed in step 2.

[1147] How it works: The server connects to a relational database such as MySQL or PostgreSQL and stores allergen information.

[1148] Data processing: Ingredient information and allergen information are linked for each menu item and stored in a database.

[1149] Output: Success status of the save operation.

[1150] Step 4: Regularly update your data

[1151] server

[1152] Input: Scheduled to run periodically (Cron job).

[1153] What it does: Uses a cron job to re-run web scraping at scheduled times to retrieve new menu information and update existing data.

[1154] Data processing: Compare new data with existing data and update any changes.

[1155] Output: A list of updated allergen information.

[1156] Step 5: Request Allergen Information

[1157] Terminal

[1158] Input: What the user wants to search for (store name, whether or not it contains allergens, etc.).

[1159] Operation: The device sends an API request to the server based on this input data.

[1160] Data transformation: Constructing request data based on user input.

[1161] Output: Allergen information returned by the server.

[1162] Step 6: Display allergen information

[1163] Terminal

[1164] Input: Allergen information obtained from the server in step 5.

[1165] Operation: The terminal displays the acquired allergen information on the user interface.

[1166] Data processing: Converting data into a display format.

[1167] Output: A display of the menu and its allergen information on the user interface.

[1168] Step 7: Report an allergic reaction

[1169] User

[1170] Input: Detailed information in case of allergic reaction (store name, food name, symptoms, etc.).

[1171] How it works: Using a dedicated application, users enter details of their allergic reaction and send them to a server.

[1172] Data Processing: Converting entered information into a reporting format.

[1173] Output: Notification that report information has been sent to the server.

[1174] Step 8: Analysis and integration of reaction information

[1175] server

[1176] Input: Allergic reaction information reported by the user in step 7.

[1177] How it works: The server parses the reported information and identifies relevant menu items and allergen ingredients.

[1178] Data processing: The reported information is compared with information in the database to identify the causative allergen.

[1179] Output: Analysis results are shared with medical institutions. Details of the analysis results are notified to medical institutions to support a prompt response.

[1180] (Application example 1)

[1181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1182] It is difficult to provide an environment where consumers with food allergies can safely choose foods when eating out or purchasing them. In particular, in brick-and-mortar stores, consumers face the challenge of having to choose foods while taking the risk of an allergic reaction, due to a lack of easy access to information about allergens contained in menus and foods. Furthermore, there is a lack of systems for quickly sharing information and taking appropriate measures in the event of an allergic reaction.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1184] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; means for updating the allergen information in the database; means for providing the user with the updated allergen information; means for receiving reports from users when an allergic reaction occurs; means for analyzing the received reports and identifying related menu items and ingredients; means for sharing the analysis results with a family doctor; means for scanning QR codes on food menus with a camera; means for transmitting the scanned information to the server to acquire allergen information; and means for displaying the acquired allergen information on a head-up display. This allows consumers to easily obtain information about allergens contained in foods, enabling safe food selection. It also enables prompt response when an allergic reaction occurs.

[1185] A "store" is a physical location that provides food and merchandise to consumers.

[1186] "Food allergen information" is information about allergen components contained in specific foods.

[1187] "Means of collection" refers to the method or technology used to obtain food allergen information from stores.

[1188] "Means for analyzing" refers to a method or technique for analyzing collected food allergen information and identifying specific allergen components.

[1189] A "database" is an electronic system for systematically storing and managing specific information.

[1190] An "updating means" is a method or technique for replacing existing data with new information.

[1191] "Means for providing to users" refers to the methods or technologies for communicating the collected and analyzed allergen information to users.

[1192] The "means for receiving a report from a user when an allergic reaction occurs" is a method or technology for receiving information about an allergic reaction from a user.

[1193] "Means for analyzing reported information" refers to a method or technique for analyzing the received information about an allergic reaction and identifying its cause.

[1194] "Means for linking to primary care physician" refers to the method or technology for communicating analysis results to healthcare providers.

[1195] A "food menu QR code" is a two-dimensional barcode that contains information about a specific food menu item.

[1196] "Camera scanning means" refers to a method or technology for capturing information in a QR code using a camera.

[1197] "Means for sending information to a server" refers to a method or technique for transmitting acquired data to a server.

[1198] "Means for obtaining allergen information" refers to a method or technology for obtaining the necessary allergen information from the server.

[1199] "Means for displaying on a head-up display" refers to a method or technology for visually displaying the acquired allergen information.

[1200] The system of the present invention provides support for users to safely obtain food allergen information in physical stores and avoid food allergies. The system is mainly composed of a server, a user device (such as smart glasses), a camera, and a database.

[1201] Basic configuration

[1202] 1. Data Collection and Analysis:

[1203] The server uses a camera to read the QR code on the food menu provided by the physical store. The data read by the camera is sent to the server, where an AI model runs to analyze allergen information. The collected food allergen information is analyzed, and the identified allergen ingredients are stored in a database.

[1204] 2. Data Management:

[1205] The database stores identified allergens and is updated regularly. This database management is important to ensure that users are always provided with the most up-to-date allergen information.

[1206] 3. User Interface:

[1207] Users use a user device such as smart glasses to scan QR codes on store menus or food shelves. By scanning the QR code, the menu ID is sent to the server, and the collected allergen information is retrieved. The retrieved allergen information is visually displayed on the head-up display (HUD) of the smart glasses.

[1208] 4. Allergic Reactions:

[1209] If a user experiences an allergic reaction after eating, they can report that information to the server through a dedicated application. The server analyzes the reported information and identifies the relevant menu items and ingredients. If necessary, the analysis results are shared with the user's doctor, allowing for prompt treatment.

[1210] Hardware and software used

[1211] Hardware:

[1212] Smart glasses (e.g. Google Glass, Vuzix Blade): Equipped with a HUD and camera.

[1213] Camera: Used to read QR codes.

[1214] software:

[1215] OpenCV: Processes camera images.

[1216] pyzbar: A library for decoding QR codes.

[1217] requests: Used to send HTTP requests to the server.

[1218] Database management software: stores and manages allergen information.

[1219] Specific examples

[1220] When a user eats at a restaurant, they scan a QR code placed on the table or menu with the smart glasses. For example, the user can check allergen information such as "This menu contains dairy products." If an allergic reaction occurs, the user can report through the application that they "developed hives after eating at a restaurant," and the server will analyze the information.

[1221] Prompt Sentence Examples

[1222] "Scan the QR code displayed in the restaurant with your smart glasses and you'll be able to see allergen information."

[1223] This invention allows consumers with food allergies to eat out safely and make safe food choices, and also allows for quick response if an allergic reaction occurs.

[1224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1225] Step 1:

[1226] A user scans a QR code on a food menu at a physical store with the camera in their smart glasses. The camera in the smart glasses reads the QR code and obtains its data (menu ID). The input is the QR code, and the output is the menu ID.

[1227] Step 2:

[1228] The smart glasses send the acquired menu ID to the server using an HTTP request. The input is the menu ID, and the output is the request sent to the server.

[1229] Step 3:

[1230] The server searches the database based on the received menu ID and retrieves the corresponding allergen information. The input is the menu ID and the output is the allergen information. The server identifies the information in the database and extracts the necessary data.

[1231] Step 4:

[1232] The server returns the acquired allergen information to the smart glasses. The response is an HTTP response. The input is the allergen information acquired from the database, and the output is the response sent to the smart glasses.

[1233] Step 5:

[1234] The smart glasses receive allergen information from the server and display it on a HUD (head-up display). The user can visually check the allergen information. The input is the allergen information, and the output is what is displayed on the HUD. For example, it might say, "This menu contains dairy products."

[1235] Step 6:

[1236] If a user experiences an allergic reaction after eating, they report the information to the server using a dedicated application. The input is detailed information about the allergic reaction, and the output is a report to the server.

[1237] Step 7:

[1238] The server analyzes the reported information and identifies the relevant menu items and ingredients. The input is the allergic reaction report, and the output is the specific information for the relevant menu items and ingredients. The server then searches the database again based on the report and derives the analysis results.

[1239] Step 8:

[1240] The server will share the analysis results with the patient's primary care physician as needed. This sharing is done via email or API. The input is the analysis results, and the output is shared with the patient's primary care physician. For example, it will send information that "the user has had an allergic reaction to dairy products."

[1241] The above processing steps enable users to safely obtain food allergen information at physical stores, and also create a system that allows for quick response in the event of an allergic reaction.

[1242] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1243] MODE FOR CARRYING OUT THE INVENTION

[1244] This invention is a system that allows users to safely select meals when eating out or taking out a meal. In particular, it combines a function to analyze and provide food allergen information collected from restaurants with an emotion engine that recognizes the user's emotions and makes suggestions related to meal selection. The specific operation of this system is described by explaining the program processing in natural language and using concrete examples.

[1245] Basic system configuration

[1246] The system consists of the following main components:

[1247] 1. Server

[1248] 2. User's device (smartphone, PC, etc.)

[1249] 3. Database

[1250] 4. Emotion Engine

[1251] The server plays a central role, the user's device acts as an interface to retrieve information, the database stores and manages allergen information, and the emotion engine analyzes the user's emotions and makes suggestions based on them.

[1252] Program processing flow

[1253] 1. Data Collection Phase

[1254] The server collects menu information from each store's website or dedicated API.

[1255] The collected data includes menu names, ingredient lists, allergen information, and more.

[1256] Example: Collect information about "chocolate cake" from Restaurant A and temporarily store it.

[1257] 2. Allergen information analysis phase

[1258] The server analyzes the collected menu information and uses an AI model to identify allergen ingredients.

[1259] Identified allergen components are recorded in a database.

[1260] Example: Identifying that "chocolate cake" contains "dairy" and "nuts."

[1261] 3. Saving to the database and updating phase

[1262] The server stores the analysis results in a database.

[1263] Ingredient information and allergen information are linked and saved for each menu item.

[1264] Example: Save allergen information for "chocolate cake" in a database.

[1265] 4. Information provision phase for end users

[1266] The user's terminal connects to the server and requests allergen information for the store and menu.

[1267] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1268] Example: When User A searches for "nut-free menu at Restaurant B," relevant menu information is provided.

[1269] 5. Emotion engine proposal phase

[1270] The server analyzes the emotion data acquired from the user's terminal.

[1271] The emotion engine makes suggestions related to meal selection based on the analyzed emotions.

[1272] Example: If user A is feeling "frustrated," suggest a menu containing ingredients that have a relaxing effect.

[1273] 6. Allergic Reaction Reporting Phase

[1274] If a user experiences an allergic reaction after eating, they enter that information into a dedicated app and report it to the server.

[1275] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[1276] Example: If user B reports that "hives appeared 20 minutes after eating at restaurant C," analyze that information.

[1277] 7. Collaboration with medical institutions phase

[1278] The server then shares the analysis results with the patient's primary care physician.

[1279] Assist doctors in responding quickly when necessary.

[1280] Example: User B's doctor is notified that "nut ingredients are suspected to be the cause of the hives."

[1281] The system of this invention provides a comprehensive solution for users with food allergies to safely enjoy eating out or taking out meals. In addition, the introduction of an emotion engine enables optimal meal selection based on the user's emotional state, contributing to improved user satisfaction.

[1282] The processing flow will be explained below.

[1283] MODE FOR CARRYING OUT THE INVENTION (SPECIFIC PROCESSING STEPS)

[1284] Basic system configuration

[1285] This system consists of a server, a user's terminal, a database, and an emotion engine.

[1286] Specific processing flow of the program

[1287] Step 1: Get store information

[1288] The server periodically collects menu information from each store's website or dedicated API.

[1289] The data collected includes menu names, ingredient lists, and allergen information.

[1290] Step 2: Analysis of allergen information

[1291] The server uses AI to analyze the collected data and identify the allergen ingredients contained in each menu item.

[1292] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[1293] Step 3: Saving to the database

[1294] The server stores the identified allergen component information in a database.

[1295] By storing ingredient and allergen information for each menu item, users can refer to it later.

[1296] Step 4: Regularly update your data

[1297] The server recollects the latest store information at a set time (for example, late at night every day), compares it with the previous data, and updates it.

[1298] If changes are detected, the database is updated to keep the information up to date.

[1299] Step 5: Obtain allergen information

[1300] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[1301] For example, User A searches for "dairy-free menu items at Restaurant B."

[1302] Step 6: Viewing information

[1303] In response to a user request, the server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1304] The user's terminal displays the received information on a user interface, for example, displaying a list of dairy-free menu items.

[1305] Step 7: Recognize emotions

[1306] The user's device uses built-in cameras and sensors to collect emotional data from the user's face and voice.

[1307] For example, when user A is tired, that emotion is recognized.

[1308] Step 8: Send and analyze emotion data

[1309] The user's device transmits the collected emotion data to the server.

[1310] The server's emotion engine analyzes the transmitted emotion data to determine the user's current emotional state.

[1311] Step 9: Emotion-Based Suggestions

[1312] The server proposes the most suitable menu to the user based on the analysis results.

[1313] For example, if the user is feeling stressed, menus that have a relaxing effect are preferentially suggested.

[1314] Step 10: Report an Allergic Reaction

[1315] If a user experiences an allergic reaction after eating, the user uses a dedicated application to report the information to the server.

[1316] For example, user B reports that "dyspnea occurs 20 minutes after eating."

[1317] Step 11: Analyze the reported information

[1318] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[1319] If necessary, the identified information is reflected in the database.

[1320] Step 12: Linking with medical institutions

[1321] The server shares the analysis results with the user's family doctor to help them respond quickly.

[1322] For example, user B's doctor may be notified that "nuts are suspected to be the cause of the breathing difficulties."

[1323] The system of this invention not only allows users to make safe food choices based on the latest allergen information, but also provides optimal suggestions based on the emotion engine.It also enables rapid response in the unlikely event of an allergic reaction, thereby improving the quality of life for people with food allergies.

[1324] Example 2

[1325] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1326] Previously, when eating out or taking home meals, the problem was that food allergen information was not sufficiently obtained, and meal suggestions based on the user's feelings were not adequately provided.In addition, there was a lack of prompt identification of information and coordination with medical institutions when an allergic reaction occurred, making it an issue to increase user safety and satisfaction.

[1327] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1328] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and using a natural language processing model to identify allergen components; means for saving the identified allergen components in a database; means for updating the allergen information in the database; means for providing the updated allergen information to the user; means for analyzing the user's emotional data; means for making suggestions based on the analyzed emotional data; means for receiving reports from the user when an allergic reaction occurs; means for analyzing the received report information and identifying related menu items and ingredients; and means for sharing the analysis results with a medical institution. This allows the user to safely select meals, reducing the risk of food allergies and receiving meal suggestions tailored to their emotional state. Furthermore, this allows for a rapid response when an allergic reaction occurs, increasing the user's sense of security.

[1329] "Means of collecting food allergen information from stores" refers to the technology and methods for obtaining menu information from store websites or dedicated APIs.

[1330] "Means of using a natural language processing model to analyze collected food allergen information and identify allergen ingredients" refers to a method of analyzing acquired menu information using natural language processing technology and identifying specific allergen ingredients.

[1331] "Means for storing identified allergen components in a database" refers to the technology or method for recording data on allergen components identified through analysis in a database.

[1332] "Means for updating allergen information in the database" refers to techniques or methods for rewriting allergen information already stored in the database with new information.

[1333] "Means for providing updated allergen information to a user" refers to techniques or methods for providing revised allergen information to a user upon request.

[1334] "Means for analyzing user emotion data" refers to techniques and methods for analyzing emotion data provided by a user and understanding its meaning and state.

[1335] "Means for making suggestions based on analyzed emotional data" refers to techniques or methods for generating meal selection or other suggestions based on a user's emotional state.

[1336] "Means for receiving a report from a user when an allergic reaction occurs" refers to an interface for a user to report a food allergic reaction and the technology or method for receiving that report.

[1337] "Means for analyzing received report information and identifying related menu items or ingredients" refers to technologies and methods for analyzing a user's report of an allergic reaction and identifying the menu item or ingredient that caused it.

[1338] "Means of sharing analysis results with medical institutions" refers to technologies and methods for notifying or sharing the analysis results of allergic reactions with medical institutions.

[1339] This invention is a system that enables users to make safe and effective food choices when eating out or taking home meals. This system combines a function to analyze and provide food allergen information collected from restaurants, and an emotion engine that recognizes the user's emotions and makes suggestions related to food selection.

[1340] Hardware and Software

[1341] Hardware: Servers (e.g., cloud computing services), user devices (e.g., smartphones, PCs, etc.)

[1342] software:

[1343] Database management system (e.g. MySQL)

[1344] Sentiment analysis engine (e.g. Google Cloud Natural Language API)

[1345] Website scraping tools (e.g. Beautiful Soup)

[1346] Basic system configuration

[1347] 1. Server: Serves as the central processing unit, collecting, analyzing, storing, and providing various data.

[1348] 2. User's terminal: Acts as an interface for users to obtain and input information.

[1349] 3. Database: Stores and manages allergen information and analysis data.

[1350] 4. Emotion Engine: Analyzes user emotion data and makes suggestions based on it.

[1351] Operation overview

[1352] The operation of this system is as follows.

[1353] Data collection and analysis

[1354] The server collects menu information from restaurant websites and dedicated APIs. This collection is done using the web scraping tool Beautiful Soup. The collected menu information includes the menu name, ingredient list, and allergen information. For example, the server collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[1355] The server then analyzes the collected information using the Google Cloud Natural Language API to identify allergens in each menu item, and the analysis results are stored in a database.

[1356] Providing information

[1357] When a user requests specific allergen information through their device, the server retrieves the relevant information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[1358] Emotion data analysis and proposals

[1359] The server analyzes the emotional data obtained from the user's device using the Google Cloud Natural Language API. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[1360] Reporting allergic reactions and contacting medical institutions

[1361] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. The analysis results are provided to medical institutions as needed. For example, if User B reports that "hives developed 20 minutes after eating at Restaurant D," the server analyzes the information and notifies the medical institution that "nut ingredients may be the cause."

[1362] Prompt Sentence Examples

[1363] 1. Example 1: User A requests "I have a nut allergy, so I would like to find nut-free menu items at Restaurant C."

[1364] Example prompt: "What are the nut-free menu items at Restaurant C?"

[1365] 2. Example 2: User B requests, "I'm feeling stressed today, so please suggest a meal that will help me relax."

[1366] Example prompt: "I'm feeling stressed today, so please suggest something relaxing."

[1367] 3. Example 3: User C requests, "I had an allergic reaction after eating, so I want to report it and identify the cause."

[1368] Example prompt: "I broke out in hives after eating at Restaurant D. Please identify the cause of my allergy."

[1369] As described above, this system is designed to reduce the risk of food allergies and provide meal suggestions based on the user's emotional state.

[1370] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1371] Step 1:

[1372] The server collects menu information from restaurant websites and dedicated APIs. Using the web scraping tool Beautiful Soup, it obtains restaurant menu items, ingredient lists, and allergen information. For example, it collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[1373] Input: Store website URL or API endpoint.

[1374] Output: Data including menu name, ingredient list, and allergen information.

[1375] Step 2:

[1376] The server uses the Google Cloud Natural Language API to analyze the collected menu information. Here, natural language processing technology is used to analyze the ingredient information and identify allergens. For example, it can identify that "chocolate cake" contains "dairy products" and "nuts."

[1377] Input: Menu name, ingredient list, and allergen information collected in Step 1.

[1378] Output: Identified allergen ingredient information.

[1379] Step 3:

[1380] The server stores the identified allergen information in a database. In this case, a MySQL database is used to link ingredient information and allergen information for each menu item. For example, information for "Chocolate Cake" is recorded as "ID: 1234, Ingredients: Sugar, Cocoa, Dairy; Allergens: Dairy, Nuts."

[1381] Input: Allergen ingredient information identified in Step 2.

[1382] Output: Allergen information stored in a database.

[1383] Step 4:

[1384] A user's device sends a request for specific allergen information to the server. The server extracts the relevant allergen information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[1385] Input: A request from the user requesting allergen information.

[1386] Output: A list of relevant allergen information.

[1387] Step 5:

[1388] The server uses the Google Cloud Natural Language API to analyze the emotion data obtained from the user's device. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[1389] Input: User emotion data.

[1390] Output: Sentiment analysis results and meal suggestions based on them.

[1391] Step 6:

[1392] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. For example, if User C reports that "hives occurred 20 minutes after eating at Restaurant D," the server analyzes the information and identifies the specific ingredients (e.g., "nuts").

[1393] Input: User's report of an allergic reaction.

[1394] Output: Parsed allergen ingredients and associated menu information.

[1395] Step 7:

[1396] The server transfers the analysis results to a system linked to a medical institution. Based on the information notified to the medical institution, a prompt response can be made. For example, a notification could be sent to User C's family doctor stating that "nut ingredients are suspected to be the cause of the hives."

[1397] Input: Allergy analysis results.

[1398] Output: Report information to medical institutions.

[1399] (Application example 2)

[1400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1401] Conventional systems cannot completely avoid the risk of food allergies when eating out or taking home meals, and do not suggest food choices based on the user's emotions. Therefore, a system that combines the analysis and provision of allergy information with emotion-based meal suggestions is needed.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1403] In this invention, the server includes means for collecting food allergen information from stores, means for analyzing the collected food allergen information and identifying allergen components, means for saving the identified allergen components in a database, means for updating the allergen information in the database, means for providing the updated allergen information to the user, means for recognizing the user's emotions and making suggestions related to meal selection based on the emotions, means for receiving reports from the user when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with a medical institution. This makes it possible to reduce the risk of food allergies and support meal selection based on the user's emotions.

[1404] The "means for collecting food allergen information from stores" refers to a combination of software and hardware for collecting menu and ingredient information from each store.

[1405] The "means for analyzing collected food allergen information and identifying allergen components" refers to algorithms and programs for analyzing collected data and identifying specific allergen components.

[1406] "Means for storing identified allergen components in a database" refers to a mechanism for storing allergen information and subsequently registering it in a database in a searchable format.

[1407] "Means for updating allergen information in the database" refers to a method for keeping existing allergen information up to date with new data.

[1408] The "means for providing updated allergen information to the user" is a system for notifying or displaying the latest allergen information to the user.

[1409] "Means for recognizing the user's emotions and making suggestions related to meal selection based on those emotions" refers to technology that analyzes the user's emotions from their facial expressions and voice and makes meal suggestions accordingly.

[1410] The "means for receiving a report from the user when an allergic reaction occurs" is a function for the user to report to the system any allergic reaction they have experienced.

[1411] The "means for analyzing the received report information and identifying the related menu or ingredient" is a process for analyzing the allergic reaction report from the user and identifying the menu or ingredient that caused it.

[1412] "Means for sharing analysis results with medical institutions" is a function for sending the analysis results of allergic reactions to the user's regular medical institution.

[1413] The system of the present invention is designed to suggest safe and comfortable meal options by combining food allergy and emotion recognition. Specifically, it is implemented using the following means.

[1414] 1. How to collect allergen information

[1415] The server collects menu information from each restaurant's website and dedicated API. The collected data includes the menu name, ingredient list, allergen information, etc. This makes it possible to always maintain the latest allergen information.

[1416] 2. Methods for analyzing allergen components

[1417] The server analyzes the collected menu information, specifically identifying allergens using an AI model and storing them in a database, where they can be later accessed by users.

[1418] 3. How to store and update allergen information

[1419] The identified allergens are stored in a database that is updated regularly, allowing users to access the most up-to-date allergen information.

[1420] 4. Means of Providing Allergen Information to Users

[1421] The user's device (such as a smartphone or PC) connects to the server and requests allergen information for the restaurant and menu. The server extracts the relevant allergen information from the database and returns it to the user's device.

[1422] 5. Emotion Recognition Suggestion Method

[1423] The emotion engine analyzes emotion data acquired from the user's device. It recognizes the user's emotions using a camera or voice input device, and makes suggestions related to meal selection based on the analysis results. These suggestions are intended to provide the optimal meal based on the user's emotional state.

[1424] 6. Allergic Reaction Reporting Method

[1425] If a user experiences an allergic reaction after eating, they can report the information to the server through a dedicated application. The server then analyzes the reported information and identifies the relevant menu items and ingredients.

[1426] 7. Linking to medical institutions

[1427] The server then sends the analysis results to the user's primary care physician, helping the physician to respond promptly if necessary.

[1428] Specific examples

[1429] For example, when a user visits "Restaurant X," they can use this system to search for "nut-free" menu items, and the camera can recognize their emotions to suggest menu items that include relaxing ingredients. If an allergic reaction occurs after eating, they can report this information to the app, which will promptly notify a medical institution.

[1430] Prompt Sentence Examples

[1431] An example of a prompt to be input to a generative AI model could be the following text:

[1432] "Design a smartphone app that uses an emotion engine to recognize the user's emotions and suggest menu items that match those emotions. This app also has the functionality to collect restaurant menu information and analyze allergens."

[1433] The above is an embodiment of the present invention.

[1434] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1435] Step 1:

[1436] The server collects food allergen information from stores.

[1437] Input: Menu information obtained from each store's website or dedicated API.

[1438] Data processing: Convert menu names, ingredient lists, and allergen information into an analyzable format.

[1439] Output: Menu information formatted in a format suitable for parsing.

[1440] Step 2:

[1441] The server analyzes the food allergen information collected and identifies the allergen components.

[1442] Input: Menu information collected and formatted in step 1.

[1443] Data processing: Using an AI model, allergen ingredients in each menu item are identified.

[1444] Output: Menu information linked to identified allergen ingredients.

[1445] Step 3:

[1446] The server stores the identified allergen components in a database.

[1447] Input: Menu information containing the allergen ingredients identified in step 2.

[1448] Data processing: Converting data into a format suitable for storing in a database.

[1449] Output: Menu and allergen information stored in a database.

[1450] Step 4:

[1451] The device connects to the server and requests allergen information for the store and menu.

[1452] Input: Store and menu information requested by the user.

[1453] Data processing: Extract the necessary menu information from the database.

[1454] Output: Providing relevant allergen information to the user's device.

[1455] Step 5:

[1456] The terminal acquires the user's emotional data, which is then analyzed by the server using an emotion engine.

[1457] Input: User's facial expressions and voice data.

[1458] Data processing: Analyze data using an emotion engine to identify emotional states.

[1459] Output: The analyzed emotional state of the user.

[1460] Step 6:

[1461] The server makes suggestions related to meal selection based on the analyzed emotional state.

[1462] Input: User's emotional state and menu information identified in step 5.

[1463] Data processing: Filter the optimal menu according to emotions and generate a list of suggestions.

[1464] Output: The optimal menu list proposed to the user's device.

[1465] Step 7:

[1466] The user reports an allergic reaction and the server receives the reported information.

[1467] Input: Allergic reaction report data from users.

[1468] Data processing: Analyze reported data and identify relevant menu items and ingredients.

[1469] Output: Parsed allergic reaction data and associated menu information.

[1470] Step 8:

[1471] The server shares the analysis results with the medical institution.

[1472] Input: Allergic reaction data analyzed in step 7.

[1473] Data processing: Converting data into a format that is easy for medical institutions to read.

[1474] Output: Allergic reaction data sent to healthcare provider.

[1475] The above are specific processing steps based on the application example.

[1476] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1478] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1479] [Fourth embodiment]

[1480] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1481] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1483] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1487] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1488] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1491] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1492] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1493] MODE FOR CARRYING OUT THE INVENTION

[1494] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports safe meal choices for users with food allergies. It also enables information sharing and rapid response in the event of an allergic reaction. The system's program processing is explained below in natural language.

[1495] Basic system configuration

[1496] This system is mainly composed of three main components: a server, a user's device, and a database. The server plays a central role, while the user's device acts as an interface for obtaining information. The database stores and manages all allergen information.

[1497] Program processing flow

[1498] 1. Data Collection Phase

[1499] Obtaining store information

[1500] The server automatically collects menu information from the restaurant's website and dedicated API. This process covers all menu information that the restaurant has made public.

[1501] For example, restaurant A collects the menu information (menu name, ingredient list, allergen ingredients) provided on its website.

[1502] Analysis of allergen information

[1503] The server analyzes the collected menu information and uses an AI model to identify any allergen ingredients it contains.

[1504] For example, for a menu item called "chocolate cake," the AI ​​model will identify allergens such as "dairy products" and "nuts" from the ingredient list.

[1505] 2. Saving and updating to the database

[1506] Conservation

[1507] After the allergen information is identified, the server stores this information in a database.

[1508] Each menu item is linked to ingredient information and allergen information, allowing users to refer to it later.

[1509] Regular data updates

[1510] The server periodically recollects store information and updates the database, ensuring that allergen information is always up-to-date.

[1511] 3. Providing Information to End Users

[1512] Obtaining allergen information

[1513] The user's terminal accesses the server and requests allergen information for a specific store or menu item.

[1514] For example, if user A searches for "dairy-free menu at restaurant B," the server will return relevant information.

[1515] User interface display

[1516] The user's device displays the allergen information obtained from the server on the user interface, helping the user select a safe menu.

[1517] 4. Reporting and dealing with allergic reactions

[1518] Response reporting

[1519] If a user experiences an allergic reaction after eating, they can use a dedicated application to report the information to the server.

[1520] For example, user B reports that "I developed hives after eating at restaurant C."

[1521] Analysis and collaboration

[1522] The server analyzes reported allergic reaction information to identify whether a particular menu item or ingredient may be the cause.

[1523] If necessary, the analysis results will be shared with the patient's primary care physician to assist in prompt response.

[1524] The system of the present invention provides an environment in which users can choose meals with peace of mind when eating out or eating at home. It also enables prompt and appropriate response in the event of an allergic reaction, improving the quality of life for people with food allergies.

[1525] The processing flow will be explained below.

[1526] Step 1: Data collection

[1527] The server collects menu information from each store's website or dedicated API.

[1528] The server temporarily stores the collected data for analysis.

[1529] Step 2: Allergen information analysis

[1530] The server analyzes the collected menu information using an AI model to identify allergen ingredients.

[1531] Analysis uses techniques such as natural language processing and image analysis.

[1532] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[1533] Step 3: Saving to the database

[1534] The server stores the analysis results in a database.

[1535] Ingredient information and allergen information are stored linked to each menu item.

[1536] Step 4: Regularly update your data

[1537] The server recollects new menu information and change information from the stores according to a preset schedule.

[1538] The server analyzes the new information collected and updates the database.

[1539] This ensures that the latest allergen information is always maintained.

[1540] Step 5: Obtain allergen information

[1541] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[1542] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1543] Step 6: Viewing information

[1544] The user's terminal displays the allergen information received from the server on a user interface.

[1545] The user can select a safe menu based on the displayed information.

[1546] Step 7: Report an allergic reaction

[1547] If a user experiences an allergic reaction after eating, the user reports the information to the server using a dedicated application.

[1548] For example, enter a specific symptom such as "I experienced difficulty breathing 20 minutes after eating."

[1549] Step 8: Analyze the reported information

[1550] The server analyzes the allergic reaction information reported by the user and identifies relevant menu items and ingredients.

[1551] The server may issue an alert based on the analysis results.

[1552] Step 9: Linking with medical institutions

[1553] The server shares the analysis results with the user's family doctor.

[1554] The server uses the linked information to help doctors respond quickly.

[1555] Step 10: Provide feedback

[1556] The server may notify all users of the collected allergic reaction information as a warning or caution.

[1557] This will help other users avoid similar allergic reactions.

[1558] Example 1

[1559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] Conventional systems lack sufficient food allergy information, and data collection and analysis are often done manually, resulting in problems with accuracy and speed. Another issue is the difficulty of sharing information and responding immediately when an allergic reaction occurs. This means that people with food allergies cannot enjoy meals safely when eating out or choosing foods, which reduces their quality of life.

[1561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1562] In this invention, the server includes means for collecting food allergen information from stores via websites or APIs, means for analyzing the collected food allergen information using a generative AI model to identify allergen components, means for saving the identified allergen components in a relational database, means for periodically updating the allergen information in the database, means for providing the updated allergen information to the user's device, means for receiving reports from users when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with medical institutions. This automates the entire process from collection to analysis, storage, provision, and response, enabling users with food allergies to choose safe and reliable meals.

[1563] A "website" is a collection of document formats for publishing and viewing information on the Internet, created using HTML or other formats.

[1564] "API" stands for Application Programming Interface, an interface that enables communication between different software programs.

[1565] "Food allergen information" refers to information about ingredients contained in food that may cause allergic reactions in certain individuals.

[1566] A "generative AI model" is an artificial intelligence model that has been trained using machine learning or deep learning techniques for a specific task.

[1567] "Analysis" is the process of examining and processing collected data in detail to extract and identify specific information.

[1568] An "allergen" is a specific component in food that may cause an allergic reaction.

[1569] A "relational database" is a database system that represents data in tabular format and manages and manipulates data using the relationships between tables.

[1570] "Update" is the process of replacing existing data or information with new data or information.

[1571] A "terminal" is a device that a user uses to input and check information, including smartphones and personal computers.

[1572] "Reporting" is the act of providing information about a particular event or condition, in this case an allergic reaction.

[1573] "Analysis results" refer to the final information or conclusions obtained during the data analysis process.

[1574] A "medical institution" is a facility where doctors and medical professionals provide medical services, including hospitals and clinics.

[1575] MODE FOR CARRYING OUT THE INVENTION

[1576] The system of the present invention collects food allergen information from restaurants, analyzes and stores that information, and supports users with food allergies in making safe meal choices. It also enables information sharing and rapid response in the event of an allergic reaction. The following describes in detail how the system's program processing is implemented.

[1577] Basic system configuration

[1578] This system consists of three main components: a server, a user's device, and a relational database. The server plays a central role, while the user's device acts as an interface for retrieving information. The database stores and manages all allergen information.

[1579] Hardware and Software

[1580] Server: A general server computer with a Linux OS is used.

[1581] Device: The smartphone or computer used by the user.

[1582] Database: Use a relational database such as MySQL or PostgreSQL.

[1583] Software and Libraries:

[1584] Web scraping: Python's Beautiful Soup and Selenium

[1585] Generative AI model: Generative AI using natural language processing technology (e.g., GPT-3)

[1586] Scheduling: Cron Jobs in Linux

[1587] Program processing flow

[1588] 1. Collecting store information

[1589] The server automatically collects menu information from each restaurant's website or through a dedicated API. Information collected includes the menu name, ingredient list, and allergen ingredients. Beautiful Soup and Selenium are used for web scraping.

[1590] Examples:

[1591] The server accesses the restaurant's website and extracts information from the HTML page, such as "Chocolate cake, ingredient list: sugar, flour, dairy, nuts."

[1592] 2. Analysis of allergen information

[1593] The collected data is analyzed by a server using a generative AI model, which uses natural language processing techniques to identify allergens from the ingredient list.

[1594] Examples:

[1595] The server inputs the list of ingredients for "chocolate cake" into the AI ​​model and identifies "dairy" and "nuts" as allergens.

[1596] Example prompt sentence:

[1597] "Collect restaurant menu information and use an AI model to identify ingredients that can identify allergens."

[1598] 3. Saving to the database

[1599] The server stores the identified allergen information in a relational database, linking ingredient information and allergen information for each menu item.

[1600] Examples:

[1601] The server links allergen information for "dairy products" and "nuts" to "chocolate cake" and stores it in a database.

[1602] 4. Regular data updates

[1603] The server periodically recollects store information and updates the database, using a cron job to update the data weekly.

[1604] Examples:

[1605] The server runs web scraping every Sunday to retrieve new menu information and update the database.

[1606] 5. Providing allergen information

[1607] The user's terminal requests allergen information for a specific restaurant or menu from the server based on the user's request. In response to the request, the server retrieves relevant information from the database and provides it to the user's terminal.

[1608] Examples:

[1609] When a user searches for "dairy-free menu items at a specific restaurant," the terminal sends a request to the server, which responds with relevant menu information.

[1610] Example prompt sentence:

[1611] "Please extract the relevant allergen information from the database and provide it to the user."

[1612] 6. Reporting and dealing with allergic reactions

[1613] Users use an application to report allergic reactions and send the information to a server, which analyzes the information, identifies related menu items and ingredients, and contacts a medical institution.

[1614] Examples:

[1615] When a user reports that they broke out in hives at a particular restaurant, the server compares the information with the ingredients, identifies the offending ingredient, and notifies a medical professional.

[1616] This system provides users with an environment where they can choose their meals with peace of mind when eating out, and enables quick and appropriate response in the event of an allergic reaction, improving quality of life.

[1617] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1618] Step 1: Gather store information

[1619] server

[1620] Input: Website URL or API endpoint for each store.

[1621] How it works: The server uses Python's Beautiful Soup and Selenium to perform web scraping. Specifically, it retrieves the HTML of a web page from a specified URL and extracts information such as menu names, ingredient lists, and allergens.

[1622] Data processing: The acquired HTML is analyzed as a DOM tree and the necessary information is extracted in text format.

[1623] Output: A list of extracted menu information.

[1624] Step 2: Analysis of allergen information

[1625] server

[1626] Input: Menu information collected in step 1.

[1627] Operation: The collected menu information is input into the generative AI model and analyzed.

[1628] Data processing: The generative AI model uses natural language processing techniques to identify allergens from an ingredient list.

[1629] Output: A list of menu items that contain the identified allergen ingredients.

[1630] Step 3: Saving to the database

[1631] server

[1632] Input: Allergen information parsed in step 2.

[1633] How it works: The server connects to a relational database such as MySQL or PostgreSQL and stores allergen information.

[1634] Data processing: Ingredient information and allergen information are linked for each menu item and stored in a database.

[1635] Output: Success status of the save operation.

[1636] Step 4: Regularly update your data

[1637] server

[1638] Input: Scheduled to run periodically (Cron job).

[1639] What it does: Uses a cron job to re-run web scraping at scheduled times to retrieve new menu information and update existing data.

[1640] Data processing: Compare new data with existing data and update any changes.

[1641] Output: A list of updated allergen information.

[1642] Step 5: Request Allergen Information

[1643] Terminal

[1644] Input: What the user wants to search for (store name, whether or not it contains allergens, etc.).

[1645] Operation: The device sends an API request to the server based on this input data.

[1646] Data transformation: Constructing request data based on user input.

[1647] Output: Allergen information returned by the server.

[1648] Step 6: Display allergen information

[1649] Terminal

[1650] Input: Allergen information obtained from the server in step 5.

[1651] Operation: The terminal displays the acquired allergen information on the user interface.

[1652] Data processing: Converting data into a display format.

[1653] Output: A display of the menu and its allergen information on the user interface.

[1654] Step 7: Report an allergic reaction

[1655] User

[1656] Input: Detailed information in case of allergic reaction (store name, food name, symptoms, etc.).

[1657] How it works: Using a dedicated application, users enter details of their allergic reaction and send them to a server.

[1658] Data Processing: Converting entered information into a reporting format.

[1659] Output: Notification that report information has been sent to the server.

[1660] Step 8: Analysis and integration of reaction information

[1661] server

[1662] Input: Allergic reaction information reported by the user in step 7.

[1663] How it works: The server parses the reported information and identifies relevant menu items and allergen ingredients.

[1664] Data processing: The reported information is compared with information in the database to identify the causative allergen.

[1665] Output: Analysis results are shared with medical institutions. Details of the analysis results are notified to medical institutions to support a prompt response.

[1666] (Application example 1)

[1667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1668] It is difficult to provide an environment where consumers with food allergies can safely choose foods when eating out or purchasing them. In particular, in brick-and-mortar stores, consumers face the challenge of having to choose foods while taking the risk of an allergic reaction, due to a lack of easy access to information about allergens contained in menus and foods. Furthermore, there is a lack of systems for quickly sharing information and taking appropriate measures in the event of an allergic reaction.

[1669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1670] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; means for updating the allergen information in the database; means for providing the user with the updated allergen information; means for receiving reports from users when an allergic reaction occurs; means for analyzing the received reports and identifying related menu items and ingredients; means for sharing the analysis results with a family doctor; means for scanning QR codes on food menus with a camera; means for transmitting the scanned information to the server to acquire allergen information; and means for displaying the acquired allergen information on a head-up display. This allows consumers to easily obtain information about allergens contained in foods, enabling safe food selection. It also enables prompt response when an allergic reaction occurs.

[1671] A "store" is a physical location that provides food and merchandise to consumers.

[1672] "Food allergen information" is information about allergen components contained in specific foods.

[1673] "Means of collection" refers to the method or technology used to obtain food allergen information from stores.

[1674] "Means for analyzing" refers to a method or technique for analyzing collected food allergen information and identifying specific allergen components.

[1675] A "database" is an electronic system for systematically storing and managing specific information.

[1676] An "updating means" is a method or technique for replacing existing data with new information.

[1677] "Means for providing to users" refers to the methods or technologies for communicating the collected and analyzed allergen information to users.

[1678] The "means for receiving a report from a user when an allergic reaction occurs" is a method or technology for receiving information about an allergic reaction from a user.

[1679] "Means for analyzing reported information" refers to a method or technique for analyzing the received information about an allergic reaction and identifying its cause.

[1680] "Means for linking to primary care physician" refers to the method or technology for communicating analysis results to healthcare providers.

[1681] A "food menu QR code" is a two-dimensional barcode that contains information about a specific food menu item.

[1682] "Camera scanning means" refers to a method or technology for capturing information in a QR code using a camera.

[1683] "Means for sending information to a server" refers to a method or technique for transmitting acquired data to a server.

[1684] "Means for obtaining allergen information" refers to a method or technology for obtaining the necessary allergen information from the server.

[1685] "Means for displaying on a head-up display" refers to a method or technology for visually displaying the acquired allergen information.

[1686] The system of the present invention provides support for users to safely obtain food allergen information in physical stores and avoid food allergies. The system is mainly composed of a server, a user device (such as smart glasses), a camera, and a database.

[1687] Basic configuration

[1688] 1. Data Collection and Analysis:

[1689] The server uses a camera to read the QR code on the food menu provided by the physical store. The data read by the camera is sent to the server, where an AI model runs to analyze allergen information. The collected food allergen information is analyzed, and the identified allergen ingredients are stored in a database.

[1690] 2. Data Management:

[1691] The database stores identified allergens and is updated regularly. This database management is important to ensure that users are always provided with the most up-to-date allergen information.

[1692] 3. User Interface:

[1693] Users use a user device such as smart glasses to scan QR codes on store menus or food shelves. By scanning the QR code, the menu ID is sent to the server, and the collected allergen information is retrieved. The retrieved allergen information is visually displayed on the head-up display (HUD) of the smart glasses.

[1694] 4. Allergic Reactions:

[1695] If a user experiences an allergic reaction after eating, they can report that information to the server through a dedicated application. The server analyzes the reported information and identifies the relevant menu items and ingredients. If necessary, the analysis results are shared with the user's doctor, allowing for prompt treatment.

[1696] Hardware and software used

[1697] Hardware:

[1698] Smart glasses (e.g. Google Glass, Vuzix Blade): Equipped with a HUD and camera.

[1699] Camera: Used to read QR codes.

[1700] software:

[1701] OpenCV: Processes camera images.

[1702] pyzbar: A library for decoding QR codes.

[1703] requests: Used to send HTTP requests to the server.

[1704] Database management software: stores and manages allergen information.

[1705] Specific examples

[1706] When a user eats at a restaurant, they scan a QR code placed on the table or menu with the smart glasses. For example, the user can check allergen information such as "This menu contains dairy products." If an allergic reaction occurs, the user can report through the application that they "developed hives after eating at a restaurant," and the server will analyze the information.

[1707] Prompt Sentence Examples

[1708] "Scan the QR code displayed in the restaurant with your smart glasses and you'll be able to see allergen information."

[1709] This invention allows consumers with food allergies to eat out safely and make safe food choices, and also allows for quick response if an allergic reaction occurs.

[1710] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1711] Step 1:

[1712] A user scans a QR code on a food menu at a physical store with the camera in their smart glasses. The camera in the smart glasses reads the QR code and obtains its data (menu ID). The input is the QR code, and the output is the menu ID.

[1713] Step 2:

[1714] The smart glasses send the acquired menu ID to the server using an HTTP request. The input is the menu ID, and the output is the request sent to the server.

[1715] Step 3:

[1716] The server searches the database based on the received menu ID and retrieves the corresponding allergen information. The input is the menu ID and the output is the allergen information. The server identifies the information in the database and extracts the necessary data.

[1717] Step 4:

[1718] The server returns the acquired allergen information to the smart glasses. The response is an HTTP response. The input is the allergen information acquired from the database, and the output is the response sent to the smart glasses.

[1719] Step 5:

[1720] The smart glasses receive allergen information from the server and display it on a HUD (head-up display). The user can visually check the allergen information. The input is the allergen information, and the output is what is displayed on the HUD. For example, it might say, "This menu contains dairy products."

[1721] Step 6:

[1722] If a user experiences an allergic reaction after eating, they report the information to the server using a dedicated application. The input is detailed information about the allergic reaction, and the output is a report to the server.

[1723] Step 7:

[1724] The server analyzes the reported information and identifies the relevant menu items and ingredients. The input is the allergic reaction report, and the output is the specific information for the relevant menu items and ingredients. The server then searches the database again based on the report and derives the analysis results.

[1725] Step 8:

[1726] The server will share the analysis results with the patient's primary care physician as needed. This sharing is done via email or API. The input is the analysis results, and the output is shared with the patient's primary care physician. For example, it will send information that "the user has had an allergic reaction to dairy products."

[1727] The above processing steps enable users to safely obtain food allergen information at physical stores, and also create a system that allows for quick response in the event of an allergic reaction.

[1728] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1729] MODE FOR CARRYING OUT THE INVENTION

[1730] This invention is a system that allows users to safely select meals when eating out or taking out a meal. In particular, it combines a function to analyze and provide food allergen information collected from restaurants with an emotion engine that recognizes the user's emotions and makes suggestions related to meal selection. The specific operation of this system is described by explaining the program processing in natural language and using concrete examples.

[1731] Basic system configuration

[1732] The system consists of the following main components:

[1733] 1. Server

[1734] 2. User's device (smartphone, PC, etc.)

[1735] 3. Database

[1736] 4. Emotion Engine

[1737] The server plays a central role, the user's device acts as an interface to retrieve information, the database stores and manages allergen information, and the emotion engine analyzes the user's emotions and makes suggestions based on them.

[1738] Program processing flow

[1739] 1. Data Collection Phase

[1740] The server collects menu information from each store's website or dedicated API.

[1741] The collected data includes menu names, ingredient lists, allergen information, and more.

[1742] Example: Collect information about "chocolate cake" from Restaurant A and temporarily store it.

[1743] 2. Allergen information analysis phase

[1744] The server analyzes the collected menu information and uses an AI model to identify allergen ingredients.

[1745] Identified allergen components are recorded in a database.

[1746] Example: Identifying that "chocolate cake" contains "dairy" and "nuts."

[1747] 3. Saving to the database and updating phase

[1748] The server stores the analysis results in a database.

[1749] Ingredient information and allergen information are linked and saved for each menu item.

[1750] Example: Save allergen information for "chocolate cake" in a database.

[1751] 4. Information provision phase for end users

[1752] The user's terminal connects to the server and requests allergen information for the store and menu.

[1753] The server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1754] Example: When User A searches for "nut-free menu at Restaurant B," relevant menu information is provided.

[1755] 5. Emotion engine proposal phase

[1756] The server analyzes the emotion data acquired from the user's terminal.

[1757] The emotion engine makes suggestions related to meal selection based on the analyzed emotions.

[1758] Example: If user A is feeling "frustrated," suggest a menu containing ingredients that have a relaxing effect.

[1759] 6. Allergic Reaction Reporting Phase

[1760] If a user experiences an allergic reaction after eating, they enter that information into a dedicated app and report it to the server.

[1761] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[1762] Example: If user B reports that "hives appeared 20 minutes after eating at restaurant C," analyze that information.

[1763] 7. Collaboration with medical institutions phase

[1764] The server then shares the analysis results with the patient's primary care physician.

[1765] Assist doctors in responding quickly when necessary.

[1766] Example: User B's doctor is notified that "nut ingredients are suspected to be the cause of the hives."

[1767] The system of this invention provides a comprehensive solution for users with food allergies to safely enjoy eating out or taking out meals. In addition, the introduction of an emotion engine enables optimal meal selection based on the user's emotional state, contributing to improved user satisfaction.

[1768] The processing flow will be explained below.

[1769] MODE FOR CARRYING OUT THE INVENTION (SPECIFIC PROCESSING STEPS)

[1770] Basic system configuration

[1771] This system consists of a server, a user's terminal, a database, and an emotion engine.

[1772] Specific processing flow of the program

[1773] Step 1: Get store information

[1774] The server periodically collects menu information from each store's website or dedicated API.

[1775] The data collected includes menu names, ingredient lists, and allergen information.

[1776] Step 2: Analysis of allergen information

[1777] The server uses AI to analyze the collected data and identify the allergen ingredients contained in each menu item.

[1778] For example, for a menu item such as "chocolate cake," it identifies allergen ingredients such as "dairy products" and "nuts."

[1779] Step 3: Saving to the database

[1780] The server stores the identified allergen component information in a database.

[1781] By storing ingredient and allergen information for each menu item, users can refer to it later.

[1782] Step 4: Regularly update your data

[1783] The server recollects the latest store information at a set time (for example, late at night every day), compares it with the previous data, and updates it.

[1784] If changes are detected, the database is updated to keep the information up to date.

[1785] Step 5: Obtain allergen information

[1786] The user's terminal connects to the server and requests allergen information for a specific store or menu item.

[1787] For example, User A searches for "dairy-free menu items at Restaurant B."

[1788] Step 6: Viewing information

[1789] In response to a user request, the server extracts the relevant allergen information from the database and returns it to the user's terminal.

[1790] The user's terminal displays the received information on a user interface, for example, displaying a list of dairy-free menu items.

[1791] Step 7: Recognize emotions

[1792] The user's device uses built-in cameras and sensors to collect emotional data from the user's face and voice.

[1793] For example, when user A is tired, that emotion is recognized.

[1794] Step 8: Send and analyze emotion data

[1795] The user's device transmits the collected emotion data to the server.

[1796] The server's emotion engine analyzes the transmitted emotion data to determine the user's current emotional state.

[1797] Step 9: Emotion-Based Suggestions

[1798] The server proposes the most suitable menu to the user based on the analysis results.

[1799] For example, if the user is feeling stressed, menus that have a relaxing effect are preferentially suggested.

[1800] Step 10: Report an Allergic Reaction

[1801] If a user experiences an allergic reaction after eating, the user uses a dedicated application to report the information to the server.

[1802] For example, user B reports that "dyspnea occurs 20 minutes after eating."

[1803] Step 11: Analyze the reported information

[1804] The server analyzes the reported allergic reaction information and identifies relevant menu items and ingredients.

[1805] If necessary, the identified information is reflected in the database.

[1806] Step 12: Linking with medical institutions

[1807] The server shares the analysis results with the user's family doctor to help them respond quickly.

[1808] For example, user B's doctor may be notified that "nuts are suspected to be the cause of the breathing difficulties."

[1809] The system of this invention not only allows users to make safe food choices based on the latest allergen information, but also provides optimal suggestions based on the emotion engine.It also enables rapid response in the unlikely event of an allergic reaction, thereby improving the quality of life for people with food allergies.

[1810] Example 2

[1811] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1812] Previously, when eating out or taking home meals, the problem was that food allergen information was not sufficiently obtained, and meal suggestions based on the user's feelings were not adequately provided.In addition, there was a lack of prompt identification of information and coordination with medical institutions when an allergic reaction occurred, making it an issue to increase user safety and satisfaction.

[1813] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1814] In this invention, the server includes: means for collecting food allergen information from stores; means for analyzing the collected food allergen information and using a natural language processing model to identify allergen components; means for saving the identified allergen components in a database; means for updating the allergen information in the database; means for providing the updated allergen information to the user; means for analyzing the user's emotional data; means for making suggestions based on the analyzed emotional data; means for receiving reports from the user when an allergic reaction occurs; means for analyzing the received report information and identifying related menu items and ingredients; and means for sharing the analysis results with a medical institution. This allows the user to safely select meals, reducing the risk of food allergies and receiving meal suggestions tailored to their emotional state. Furthermore, this allows for a rapid response when an allergic reaction occurs, increasing the user's sense of security.

[1815] "Means of collecting food allergen information from stores" refers to the technology and methods for obtaining menu information from store websites or dedicated APIs.

[1816] "Means of using a natural language processing model to analyze collected food allergen information and identify allergen ingredients" refers to a method of analyzing acquired menu information using natural language processing technology and identifying specific allergen ingredients.

[1817] "Means for storing identified allergen components in a database" refers to the technology or method for recording data on allergen components identified through analysis in a database.

[1818] "Means for updating allergen information in the database" refers to techniques or methods for rewriting allergen information already stored in the database with new information.

[1819] "Means for providing updated allergen information to a user" refers to techniques or methods for providing revised allergen information to a user upon request.

[1820] "Means for analyzing user emotion data" refers to techniques and methods for analyzing emotion data provided by a user and understanding its meaning and state.

[1821] "Means for making suggestions based on analyzed emotional data" refers to techniques or methods for generating meal selection or other suggestions based on a user's emotional state.

[1822] "Means for receiving a report from a user when an allergic reaction occurs" refers to an interface for a user to report a food allergic reaction and the technology or method for receiving that report.

[1823] "Means for analyzing received report information and identifying related menu items or ingredients" refers to technologies and methods for analyzing a user's report of an allergic reaction and identifying the menu item or ingredient that caused it.

[1824] "Means of sharing analysis results with medical institutions" refers to technologies and methods for notifying or sharing the analysis results of allergic reactions with medical institutions.

[1825] This invention is a system that enables users to make safe and effective food choices when eating out or taking home meals. This system combines a function to analyze and provide food allergen information collected from restaurants, and an emotion engine that recognizes the user's emotions and makes suggestions related to food selection.

[1826] Hardware and Software

[1827] Hardware: Servers (e.g., cloud computing services), user devices (e.g., smartphones, PCs, etc.)

[1828] software:

[1829] Database management system (e.g. MySQL)

[1830] Sentiment analysis engine (e.g. Google Cloud Natural Language API)

[1831] Website scraping tools (e.g. Beautiful Soup)

[1832] Basic system configuration

[1833] 1. Server: Serves as the central processing unit, collecting, analyzing, storing, and providing various data.

[1834] 2. User's terminal: Acts as an interface for users to obtain and input information.

[1835] 3. Database: Stores and manages allergen information and analysis data.

[1836] 4. Emotion Engine: Analyzes user emotion data and makes suggestions based on it.

[1837] Operation overview

[1838] The operation of this system is as follows.

[1839] Data collection and analysis

[1840] The server collects menu information from restaurant websites and dedicated APIs. This collection is done using the web scraping tool Beautiful Soup. The collected menu information includes the menu name, ingredient list, and allergen information. For example, the server collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[1841] The server then analyzes the collected information using the Google Cloud Natural Language API to identify allergens in each menu item, and the analysis results are stored in a database.

[1842] Providing information

[1843] When a user requests specific allergen information through their device, the server retrieves the relevant information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[1844] Emotion data analysis and proposals

[1845] The server analyzes the emotional data obtained from the user's device using the Google Cloud Natural Language API. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[1846] Reporting allergic reactions and contacting medical institutions

[1847] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. The analysis results are provided to medical institutions as needed. For example, if User B reports that "hives developed 20 minutes after eating at Restaurant D," the server analyzes the information and notifies the medical institution that "nut ingredients may be the cause."

[1848] Prompt Sentence Examples

[1849] 1. Example 1: User A requests "I have a nut allergy, so I would like to find nut-free menu items at Restaurant C."

[1850] Example prompt: "What are the nut-free menu items at Restaurant C?"

[1851] 2. Example 2: User B requests, "I'm feeling stressed today, so please suggest a meal that will help me relax."

[1852] Example prompt: "I'm feeling stressed today, so please suggest something relaxing."

[1853] 3. Example 3: User C requests, "I had an allergic reaction after eating, so I want to report it and identify the cause."

[1854] Example prompt: "I broke out in hives after eating at Restaurant D. Please identify the cause of my allergy."

[1855] As described above, this system is designed to reduce the risk of food allergies and provide meal suggestions based on the user's emotional state.

[1856] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1857] Step 1:

[1858] The server collects menu information from restaurant websites and dedicated APIs. Using the web scraping tool Beautiful Soup, it obtains restaurant menu items, ingredient lists, and allergen information. For example, it collects information about "chocolate cake" from Restaurant A's website and temporarily stores it.

[1859] Input: Store website URL or API endpoint.

[1860] Output: Data including menu name, ingredient list, and allergen information.

[1861] Step 2:

[1862] The server uses the Google Cloud Natural Language API to analyze the collected menu information. Here, natural language processing technology is used to analyze the ingredient information and identify allergens. For example, it can identify that "chocolate cake" contains "dairy products" and "nuts."

[1863] Input: Menu name, ingredient list, and allergen information collected in Step 1.

[1864] Output: Identified allergen ingredient information.

[1865] Step 3:

[1866] The server stores the identified allergen information in a database. In this case, a MySQL database is used to link ingredient information and allergen information for each menu item. For example, information for "Chocolate Cake" is recorded as "ID: 1234, Ingredients: Sugar, Cocoa, Dairy; Allergens: Dairy, Nuts."

[1867] Input: Allergen ingredient information identified in Step 2.

[1868] Output: Allergen information stored in a database.

[1869] Step 4:

[1870] A user's device sends a request for specific allergen information to the server. The server extracts the relevant allergen information from the database and returns it to the user's device. For example, if User A searches for "nut-free menu at Restaurant C," the server provides the relevant menu information.

[1871] Input: A request from the user requesting allergen information.

[1872] Output: A list of relevant allergen information.

[1873] Step 5:

[1874] The server uses the Google Cloud Natural Language API to analyze the emotion data obtained from the user's device. Based on the analysis results, the emotion engine suggests optimal meal choices for the user. For example, if User A is feeling "frustrated," the server will suggest a menu with a relaxing effect.

[1875] Input: User emotion data.

[1876] Output: Sentiment analysis results and meal suggestions based on them.

[1877] Step 6:

[1878] Users use a dedicated app to report any allergic reactions they experience after eating to the server. The server analyzes the reported information and identifies the relevant menu items and ingredients. For example, if User C reports that "hives occurred 20 minutes after eating at Restaurant D," the server analyzes the information and identifies the specific ingredients (e.g., "nuts").

[1879] Input: User's report of an allergic reaction.

[1880] Output: Parsed allergen ingredients and associated menu information.

[1881] Step 7:

[1882] The server transfers the analysis results to a system linked to a medical institution. Based on the information notified to the medical institution, a prompt response can be made. For example, a notification could be sent to User C's family doctor stating that "nut ingredients are suspected to be the cause of the hives."

[1883] Input: Allergy analysis results.

[1884] Output: Report information to medical institutions.

[1885] (Application example 2)

[1886] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1887] Conventional systems cannot completely avoid the risk of food allergies when eating out or taking home meals, and do not suggest food choices based on the user's emotions. Therefore, a system that combines the analysis and provision of allergy information with emotion-based meal suggestions is needed.

[1888] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1889] In this invention, the server includes means for collecting food allergen information from stores, means for analyzing the collected food allergen information and identifying allergen components, means for saving the identified allergen components in a database, means for updating the allergen information in the database, means for providing the updated allergen information to the user, means for recognizing the user's emotions and making suggestions related to meal selection based on the emotions, means for receiving reports from the user when an allergic reaction occurs, means for analyzing the received report information and identifying related menu items and ingredients, and means for sharing the analysis results with a medical institution. This makes it possible to reduce the risk of food allergies and support meal selection based on the user's emotions.

[1890] The "means for collecting food allergen information from stores" refers to a combination of software and hardware for collecting menu and ingredient information from each store.

[1891] The "means for analyzing collected food allergen information and identifying allergen components" refers to algorithms and programs for analyzing collected data and identifying specific allergen components.

[1892] "Means for storing identified allergen components in a database" refers to a mechanism for storing allergen information and subsequently registering it in a database in a searchable format.

[1893] "Means for updating allergen information in the database" refers to a method for keeping existing allergen information up to date with new data.

[1894] The "means for providing updated allergen information to the user" is a system for notifying or displaying the latest allergen information to the user.

[1895] "Means for recognizing the user's emotions and making suggestions related to meal selection based on those emotions" refers to technology that analyzes the user's emotions from their facial expressions and voice and makes meal suggestions accordingly.

[1896] The "means for receiving a report from the user when an allergic reaction occurs" is a function for the user to report to the system any allergic reaction they have experienced.

[1897] The "means for analyzing the received report information and identifying the related menu or ingredient" is a process for analyzing the allergic reaction report from the user and identifying the menu or ingredient that caused it.

[1898] "Means for sharing analysis results with medical institutions" is a function for sending the analysis results of allergic reactions to the user's regular medical institution.

[1899] The system of the present invention is designed to suggest safe and comfortable meal options by combining food allergy and emotion recognition. Specifically, it is implemented using the following means.

[1900] 1. How to collect allergen information

[1901] The server collects menu information from each restaurant's website and dedicated API. The collected data includes the menu name, ingredient list, allergen information, etc. This makes it possible to always maintain the latest allergen information.

[1902] 2. Methods for analyzing allergen components

[1903] The server analyzes the collected menu information, specifically identifying allergens using an AI model and storing them in a database, where they can be later accessed by users.

[1904] 3. How to store and update allergen information

[1905] The identified allergens are stored in a database that is updated regularly, allowing users to access the most up-to-date allergen information.

[1906] 4. Means of Providing Allergen Information to Users

[1907] The user's device (such as a smartphone or PC) connects to the server and requests allergen information for the restaurant and menu. The server extracts the relevant allergen information from the database and returns it to the user's device.

[1908] 5. Emotion Recognition Suggestion Method

[1909] The emotion engine analyzes emotion data acquired from the user's device. It recognizes the user's emotions using a camera or voice input device, and makes suggestions related to meal selection based on the analysis results. These suggestions are intended to provide the optimal meal based on the user's emotional state.

[1910] 6. Allergic Reaction Reporting Method

[1911] If a user experiences an allergic reaction after eating, they can report the information to the server through a dedicated application. The server then analyzes the reported information and identifies the relevant menu items and ingredients.

[1912] 7. Linking to medical institutions

[1913] The server then sends the analysis results to the user's primary care physician, helping the physician to respond promptly if necessary.

[1914] Specific examples

[1915] For example, when a user visits "Restaurant X," they can use this system to search for "nut-free" menu items, and the camera can recognize their emotions to suggest menu items that include relaxing ingredients. If an allergic reaction occurs after eating, they can report this information to the app, which will promptly notify a medical institution.

[1916] Prompt Sentence Examples

[1917] An example of a prompt to be input to a generative AI model could be the following text:

[1918] "Design a smartphone app that uses an emotion engine to recognize the user's emotions and suggest menu items that match those emotions. This app also has the functionality to collect restaurant menu information and analyze allergens."

[1919] The above is an embodiment of the present invention.

[1920] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1921] Step 1:

[1922] The server collects food allergen information from stores.

[1923] Input: Menu information obtained from each store's website or dedicated API.

[1924] Data processing: Convert menu names, ingredient lists, and allergen information into an analyzable format.

[1925] Output: Menu information formatted in a format suitable for parsing.

[1926] Step 2:

[1927] The server analyzes the food allergen information collected and identifies the allergen components.

[1928] Input: Menu information collected and formatted in step 1.

[1929] Data processing: Using an AI model, allergen ingredients in each menu item are identified.

[1930] Output: Menu information linked to identified allergen ingredients.

[1931] Step 3:

[1932] The server stores the identified allergen components in a database.

[1933] Input: Menu information containing the allergen ingredients identified in step 2.

[1934] Data processing: Converting data into a format suitable for storing in a database.

[1935] Output: Menu and allergen information stored in a database.

[1936] Step 4:

[1937] The device connects to the server and requests allergen information for the store and menu.

[1938] Input: Store and menu information requested by the user.

[1939] Data processing: Extract the necessary menu information from the database.

[1940] Output: Providing relevant allergen information to the user's device.

[1941] Step 5:

[1942] The terminal acquires the user's emotional data, which is then analyzed by the server using an emotion engine.

[1943] Input: User's facial expressions and voice data.

[1944] Data processing: Analyze data using an emotion engine to identify emotional states.

[1945] Output: The analyzed emotional state of the user.

[1946] Step 6:

[1947] The server makes suggestions related to meal selection based on the analyzed emotional state.

[1948] Input: User's emotional state and menu information identified in step 5.

[1949] Data processing: Filter the optimal menu according to emotions and generate a list of suggestions.

[1950] Output: The optimal menu list proposed to the user's device.

[1951] Step 7:

[1952] The user reports an allergic reaction and the server receives the reported information.

[1953] Input: Allergic reaction report data from users.

[1954] Data processing: Analyze reported data and identify relevant menu items and ingredients.

[1955] Output: Parsed allergic reaction data and associated menu information.

[1956] Step 8:

[1957] The server shares the analysis results with the medical institution.

[1958] Input: Allergic reaction data analyzed in step 7.

[1959] Data processing: Converting data into a format that is easy for medical institutions to read.

[1960] Output: Allergic reaction data sent to healthcare provider.

[1961] The above are specific processing steps based on the application example.

[1962] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1963] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1964] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1965] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1966] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1967] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1968] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1969] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1970] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1971] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1972] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1973] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1974] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1975] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1976] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1977] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1978] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1979] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1980] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1981] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1982] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1983] The following is further disclosed regarding the above embodiment.

[1984] (Claim 1)

[1985] A means of collecting food allergen information from stores;

[1986] A means for analyzing the collected food allergen information and identifying allergen components;

[1987] means for storing the identified allergen components in a database;

[1988] a means for updating the allergen information in the database;

[1989] means for providing updated allergen information to a user;

[1990] a means for receiving a report from the user when an allergic reaction occurs;

[1991] A means for analyzing the received report information and identifying relevant menu items and ingredients;

[1992] A means of sharing the analysis results with your family doctor,

[1993] A system including:

[1994] (Claim 2)

[1995] 10. The system of claim 1, further comprising means for periodically updating the collected allergen information.

[1996] (Claim 3)

[1997] 2. The system according to claim 1, further comprising a user interface means for extracting and providing relevant allergen information from the database in response to a user request.

[1998] "Example 1"

[1999] (Claim 1)

[2000] A means to collect food allergen information from stores via websites and APIs, and

[2001] A means of analyzing the collected food allergen information using a generative AI model and identifying allergen components;

[2002] means for storing the identified allergen components in a relational database;

[2003] a means of regularly updating the allergen information in the database; and

[2004] means for providing updated allergen information to a user's terminal;

[2005] a means for receiving a report from the user when an allergic reaction occurs;

[2006] A means for analyzing the received report information and identifying relevant menu items and ingredients;

[2007] A means of sharing the analysis results with medical institutions,

[2008] A system including:

[2009] (Claim 2)

[2010] 10. The system of claim 1, further comprising means for analyzing the collected allergen information using a generative AI model.

[2011] (Claim 3)

[2012] 2. The system according to claim 1, further comprising an interface means for extracting and providing relevant allergen information from the database in response to a user request.

[2013] "Application Example 1"

[2014] (Claim 1)

[2015] A means of collecting food allergen information from stores;

[2016] A means for analyzing the collected food allergen information and identifying allergen components;

[2017] means for storing the identified allergen components in a database;

[2018] a means for updating the allergen information in the database;

[2019] means for providing updated allergen information to a user;

[2020] a means for receiving a report from the user when an allergic reaction occurs;

[2021] A means for analyzing the received report information and identifying relevant menu items and ingredients;

[2022] A means of sharing the analysis results with your family doctor,

[2023] A way to read the QR code on the food menu with a camera,

[2024] means for transmitting the read information to a server and acquiring allergen information;

[2025] a means for displaying the acquired allergen information on a head-up display;

[2026] A system including:

[2027] (Claim 2)

[2028] 10. The system of claim 1, further comprising means for periodically updating the collected allergen information.

[2029] (Claim 3)

[2030] 2. The system according to claim 1, further comprising a user interface means for extracting and providing relevant allergen information from the database in response to a user request.

[2031] "Example 2: Combining Emotion Engines"

[2032] (Claim 1)

[2033] A means of collecting food allergen information from stores;

[2034] a means for analyzing the collected food allergen information and using a natural language processing model to identify allergen components;

[2035] means for storing the identified allergen components in a database;

[2036] a means for updating the allergen information in the database;

[2037] means for providing updated allergen information to a user;

[2038] means for analyzing user emotion data;

[2039] a means for making suggestions based on the analyzed emotion data;

[2040] a means for receiving a report from the user when an allergic reaction occurs;

[2041] A means for analyzing the received report information and identifying relevant menu items and ingredients;

[2042] A means of sharing the analysis results with medical institutions,

[2043] A system including:

[2044] (Claim 2)

[2045] 10. The system of claim 1, further comprising means for periodically updating the collected allergen information.

[2046] (Claim 3)

[2047] 2. The system according to claim 1, further comprising a user interface means for extracting and providing relevant allergen information from the database in response to a user request.

[2048] "Application example 2 when combining emotion engines"

[2049] (Claim 1)

[2050] A means of collecting food allergen information from stores;

[2051] A means for analyzing the collected food allergen information and identifying allergen components;

[2052] means for storing the identified allergen components in a database;

[2053] a means for updating the allergen information in the database;

[2054] means for providing updated allergen information to a user;

[2055] means for recognizing a user's emotions and providing suggestions related to meal selection based on the emotions;

[2056] a means for receiving a report from the user when an allergic reaction occurs;

[2057] A means for analyzing the received report information and identifying relevant menu items and ingredients;

[2058] A means of sharing the analysis results with medical institutions,

[2059] A system including:

[2060] (Claim 2)

[2061] 10. The system of claim 1, further comprising means for periodically updating the collected allergen information.

[2062] (Claim 3)

[2063] 2. The system according to claim 1, further comprising a user interface means for extracting and providing relevant allergen information from the database in response to a user request. [Explanation of symbols]

[2064] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting food allergen information from stores; A means for analyzing the collected food allergen information and identifying allergen components; means for storing the identified allergen components in a database; a means for updating the allergen information in the database; means for providing updated allergen information to a user; a means for receiving a report from the user when an allergic reaction occurs; A means for analyzing the received report information and identifying relevant menu items and ingredients; A means of sharing the analysis results with your family doctor, A system including:

2. 10. The system of claim 1, further comprising means for periodically updating the collected allergen information.

3. 2. The system according to claim 1, further comprising a user interface means for extracting and providing relevant allergen information from the database in response to a user request.

Images