System

Abstract

A system is provided.SOLUTION: A system comprising: photographing means attached to a serving robot; data acquisition means for acquiring image data periodically photographed by the photographing means; analysis means for analyzing the image data; advice generation means for generating cooking advice based on an analysis result by the analysis means; and notification means for notifying a user of the cooking advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the food and beverage industry, maintaining and improving product quality and increasing employee work efficiency are important issues. Consistency and efficiency in quality control are especially important for companies that operate multiple stores or are expanding overseas. However, doing this manually is cumbersome and increases the burden on employees, so an effective automated system is needed. [Means for solving the problem]

[0005] The present invention includes a data acquisition means for periodically photographing products using a camera attached to a food delivery robot and acquiring image data of the photographs. It also includes an analysis means for analyzing the acquired image data and an advice generation means for generating cooking advice based on the analysis results. It also includes a notification means for notifying the user of the generated cooking advice. In this way, the present invention automatically checks product quality and provides appropriate advice, thereby streamlining quality control and reducing the burden on employees.

[0006] A "serving robot" is an automated device used in restaurants and eateries to deliver food and drinks to customers' tables.

[0007] "Photographing means" refers to means for obtaining images or videos of objects or scenes as electronic data using an optical device such as a camera.

[0008] The "data acquisition means" is a means for collecting and storing image data acquired by the imaging means and making it available within the system.

[0009] The "analysis means" is a means for analyzing data based on acquired image data in accordance with specified conditions and algorithms, and deriving results.

[0010] The "advice generating means" is a means for generating specific cooking advice and improvement suggestions based on the analysis results obtained by the analyzing means.

[0011] "Notification means" refers to a means for communicating generated advice or analysis results to users or employees. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, and a notification means.

[0034] Overview of program processing

[0035] System Setup

[0036] server

[0037] 1. The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system.

[0038] 2. The server performs a test shoot to verify that the camera settings were applied correctly.

[0039] Specific examples

[0040] The server receives instructions from the user such as "Set the camera resolution to 1080p and the shooting interval to 5 minutes," and reflects these instructions on the camera. It then takes a test shot to check whether the settings have been correctly reflected.

[0041] Data capture and acquisition

[0042] Terminal

[0043] 1. The device takes images and videos of the food being served at the specified interval (e.g., every 5 minutes).

[0044] 2. The device temporarily stores the captured data.

[0045] Specific examples

[0046] The device takes a photo of the food every five minutes and temporarily stores the photo data.

[0047] server

[0048] 1. The server receives the photo and video data sent from the device.

[0049] 2. The server stores the received data in a database and checks the integrity of the data.

[0050] Specific examples

[0051] The server receives the photos of the food sent from the device and stores the photo data in a database. It checks that there are no missing data.

[0052] Data analysis and advice generation

[0053] server

[0054] 1. The server passes the stored data to the AI ​​model and begins analysis.

[0055] 2. The server uses an AI model to evaluate the condition and quality of the food (color, shape, presentation, etc.).

[0056] 3. The server receives the analysis results and detects defects and areas for improvement as necessary.

[0057] Specific examples

[0058] The server uses an AI model to analyze the photos and detect problems such as "the food is overcooked" or "the presentation is messy."

[0059] server

[0060] 1. The server generates specific cooking advice based on the analysis results.

[0061] 2. The server formats the cooking advice into a format that can be communicated to the user.

[0062] Specific examples

[0063] Based on the analysis result that the food is "overcooked," the server generates specific advice such as "reduce the cooking time by one minute" and notifies the user.

[0064] Advice notification and response

[0065] server

[0066] 1. The server sends the generated advice to the user's terminal.

[0067] 2. The server ensures that the notification is displayed properly.

[0068] User

[0069] 1. The user receives and confirms the advice notification from the server.

[0070] 2. The user adjusts the cooking method according to the received advice.

[0071] Specific examples

[0072] The user receives a notification to "reduce the bake time by 1 minute" and reduces the bake time accordingly the next time they cook.

[0073] Order assistance function

[0074] server

[0075] 1. If the server detects a shortage of materials during AI analysis, it will activate the ordering assistance function.

[0076] 2. The server generates an order list of materials and sends it to the user.

[0077] User

[0078] 1. The user receives the order list, checks the required materials, and places an order.

[0079] Specific examples

[0080] The server detects that there is a shortage of fresh lemons, generates an order list, and sends it to the user, who then checks it and orders only the amount they need.

[0081] summary

[0082] This invention is a system that uses a food delivery robot to automatically check product quality and provide specific cooking advice based on the analysis results of AI. This will improve the efficiency of quality control in restaurants and reduce the burden on employees. In addition, if a shortage of ingredients is detected, an ordering assistance function will be provided, further improving the efficiency of ingredient management.

[0083] The processing flow will be explained below.

[0084] Step 1:

[0085] server

[0086] The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system. It also performs test shooting to confirm that the settings are correctly reflected.

[0087] Step 2:

[0088] Terminal

[0089] The device takes pictures and videos of the food being served at a specified interval (e.g., every 5 minutes), and temporarily stores the captured data in its internal storage.

[0090] Step 3:

[0091] Terminal

[0092] The device transmits the photos and videos stored in its internal storage to the server in real time, and receives feedback to confirm that the data transmission was successful.

[0093] Step 4:

[0094] server

[0095] The server receives the photo and video data sent from the device and stores it in a database. When saving, it checks the integrity of the data to ensure there are no missing or error data.

[0096] Step 5:

[0097] server

[0098] The server then passes the stored data to the AI ​​model, which then evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[0099] Step 6:

[0100] server

[0101] The server receives the analysis results from the AI ​​model and uses them to detect defects and areas for improvement, such as identifying problems like "overcooked" or "messy presentation."

[0102] Step 7:

[0103] server

[0104] The server generates specific cooking advice based on the analysis results, and the advice is formatted in a way that is easy for the user to understand.

[0105] Step 8:

[0106] server

[0107] The server sends the generated cooking advice to the user's device and monitors the delivery status to ensure that the notification is displayed properly.

[0108] Step 9:

[0109] User

[0110] The user receives the advice notification from the server, checks the contents, and, if necessary, modifies the cooking method based on the advice.

[0111] Step 10:

[0112] server

[0113] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function, generates an appropriate ordering list for materials, and sends it to the user.

[0114] Step 11:

[0115] User

[0116] The user receives the order list sent from the server, checks the required materials, and then takes steps to order the materials.

[0117] summary

[0118] Through these steps, the present invention utilizes a food delivery robot to monitor product availability in real time, provide specific cooking advice based on AI analysis results, and detect ingredient shortages and assist with ordering, thereby improving quality control and operational efficiency.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] Traditionally, quality control processes in restaurants have often been performed manually, resulting in issues such as wasted human resources and inaccurate quality assessment. Furthermore, there was a lack of systems for early detection of defects and providing specific advice for improvement, making it difficult to maintain consistent quality. Furthermore, there was also a lack of systems for timely detection of ingredient shortages and appropriate ordering, resulting in inefficient ingredient management.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes a setting application means that receives settings for the image capture means from the user and applies them to the camera and analysis system, a captured data storage means that temporarily stores data, a database storage means that stores image data sent from the terminal in a database and checks consistency, and an ingredient detection means and ordering assistance means that detect ingredient shortages during analysis and provide an ordering assistance function. This automates and streamlines quality control in restaurants, enables the provision of specific cooking advice, and enables timely detection of ingredient shortages and effective ingredient management.

[0124] The "photography means" is a device that is attached to the food delivery robot and periodically takes pictures and videos.

[0125] The "data acquisition means" is a mechanism for collecting and storing image data acquired by the imaging means.

[0126] An "analysis means" is a system or algorithm for analyzing acquired image data and evaluating its content.

[0127] The "advice generating means" is a function that generates specific cooking advice based on the analysis results of the analyzing means.

[0128] The "notification means" is a system for notifying and displaying the generated cooking advice on the user's terminal.

[0129] The "setting application means" is a mechanism that reflects the camera setting parameters received from the user in the food delivery robot and analysis system.

[0130] The "photographing data storage means" is a storage system for temporarily storing image data obtained by the photographing means.

[0131] The "database storage means" is a mechanism that stores image data sent from the terminal in a database and checks its consistency.

[0132] "Material detection means" means a system or algorithm for detecting material shortages during analysis.

[0133] The "order assist means" is a function that creates an order list based on the shortage materials detected by the material detection means and prompts the user to place an order.

[0134] The present invention is a system that analyzes image data captured by a food delivery robot, provides cooking advice based on the results, and also manages ingredients. Specifically, this system is comprised of an image capture means, data acquisition means, analysis means, advice generation means, notification means, setting application means, captured data storage means, database storage means, ingredient detection means, and ordering assistance means attached to the food delivery robot.

[0135] server

[0136] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval) and applies them to the camera and AI analysis system. This ensures that the shooting method operates with the correct settings. The server then takes a test shot and checks the results to ensure the settings were applied properly.

[0137] Terminal

[0138] The device periodically captures images and videos of the food being served according to the specified interval. For example, the device takes a photo of the food every five minutes and temporarily stores the photo data in local storage. The captured data is periodically sent to the server.

[0139] Data storage and analysis

[0140] The server receives the photo and video data sent from the device and stores it in a database. At this time, the data is checked for consistency and to ensure there are no missing pieces. The saved data is then passed to the AI ​​model, which begins analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and returns the results to the server. Specifically, it may give an evaluation such as "the food is overcooked" or "the presentation is messy."

[0141] Cooking advice generation and notification

[0142] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. For example, the advice might be, "Reduce the cooking time by one minute." The generated cooking advice is then sent to the user's device. The user receives the notification, checks the content, and adjusts the cooking method accordingly for the next time.

[0143] Material detection and ordering assistance

[0144] Furthermore, if a shortage of ingredients is detected during AI analysis, the server will activate the ordering assistance function. For example, if it detects that there is a shortage of fresh lemons, a list of ingredients to order will be generated and sent to the user. The user can then check the list and order the necessary ingredients.

[0145] As described above, this system automates and streamlines quality control in restaurants. It also provides specific cooking advice, which is expected to improve cooking quality. Furthermore, as soon as a shortage of ingredients is detected, appropriate action can be taken immediately, which also improves the efficiency of ingredient management.

[0146] Prompt Sentence Examples

[0147] "We would like to build a system that can analyze photos of food taken by a food delivery robot and evaluate their quality. Based on that, it can provide cooking advice based on the needs of the customer. The system should evaluate factors such as cooking time, presentation, and color, and generate specific advice."

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Step 1: Apply camera settings

[0150] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval). The input is the camera parameters set by the user, and the output is the state in which the settings have been applied to the camera. The server sends the settings via the camera's API, and the settings are changed. Specifically, the server receives a request for camera settings and sends an HTTP request to the corresponding endpoint to reflect the settings.

[0151] Step 2: Conduct a test shoot

[0152] The server takes a test shot to confirm that the camera settings have been applied correctly and checks the results. The input is the state with the camera settings applied, and the output is the test image taken. The server sends a command to the camera to take a test shot, displays the acquired image in real time, and checks whether the settings have been applied.

[0153] Step 3: Conduct regular photo shoots

[0154] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes). The input is the interval based on the internal clock, and the output is the captured image and video data. Specifically, the device sets a timer, starts the camera to take a photo each time the timer expires, and saves the captured data in local storage.

[0155] Step 4: Sending and Receiving Data

[0156] The device temporarily stores the captured data and then sends it to the server. The server receives the data sent from the device, stores it in a database, and checks its consistency. The input is the image data generated by the device, and the output is stored in the database within the server. Specifically, the device sends data to the server using an HTTP POST request, and the server receives the data and inserts it into the database.

[0157] Step 5: Analysis by AI model

[0158] The server passes the stored data to the AI ​​model and begins analysis. The input is the image data stored in the database, and the output is the analysis results returned by the AI ​​model. The server sends the data to the AI ​​model's API endpoint, and the model evaluates the condition and quality of the food (color, shape, presentation, etc.), and receives the results.

[0159] Step 6: Generate cooking advice

[0160] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. The input is the analysis results of the AI ​​model, and the output is the cooking advice to be provided to the user. Specifically, the server uses a template to interpret the analysis results and generates specific advice such as "Please reduce the cooking time by one minute."

[0161] Step 7: Advice Notification

[0162] The server sends the generated advice to the user's device. The user checks that the notification is displayed properly and modifies the cooking method according to the received advice. The input is the generated cooking advice, and the output is the notification displayed on the user's device. Specifically, the server sends the advice via email or app notification, and the user checks the notification.

[0163] Step 8: Assist with ordering materials

[0164] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function. The input is the analysis results of the AI ​​model and material inventory data, and the output is an ordering list provided to the user. Specifically, the server generates an ordering list based on the material detection results and sends it to the user. The user receives the ordering list, checks the required materials, and places an order.

[0165] Through these steps, the system can automatically check product quality, provide specific cooking advice, and streamline ingredient management.

[0166] (Application example 1)

[0167] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0168] In restaurants using conventional food delivery robots, product quality management and cooking improvements were sometimes not properly carried out. It was also difficult to respond quickly to ingredient shortages or provide specific advice to staff. As a result, problems arose, such as a decrease in customer satisfaction and a loss of store operational efficiency. The present invention aims to solve these problems and improve the efficiency of cooking quality management, the provision of improvement advice, and ingredient management in restaurants.

[0169] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0170] In this invention, the server includes a photographing means attached to the food delivery robot, a data acquisition means for periodically acquiring photographed image data, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results by the analysis means, a notification means for notifying the user of the cooking advice, an ordering assistance means for detecting ingredient shortages and generating an ordering list, a communication means for communicating with the server to send and receive food images, and an AI analysis means for performing image analysis using an AI model. This allows for efficient provision of advice to improve cooking quality and ingredient management, thereby improving restaurant operation efficiency and customer satisfaction.

[0171] The "photography means" is a device that is attached to the food delivery robot and periodically acquires image data.

[0172] The "data acquisition means" is a device or program that receives image data captured by the image capture means and stores or transmits the data.

[0173] The "analysis means" is a program or device for evaluating the condition and quality of a product using the acquired image data.

[0174] The "advice generating means" is a program or device that generates specific cooking advice based on the analysis results from the analyzing means.

[0175] The "notification means" refers to a device or program for transmitting the generated cooking advice to the user.

[0176] The "order assistance means" is a program or device for detecting shortages of materials and generating an order list for the necessary materials.

[0177] The "communication means" refers to devices and programs for transmitting and receiving cooking images to and from the server via communication.

[0178] "AI analysis means" refers to a program or device that uses an AI model to analyze image data and evaluate the condition and quality of a product.

[0179] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, a notification means, an order assistance means, a communication means, and an AI analysis means.

[0180] 1. Hardware and Software Used

[0181] Hardware

[0182] Smartphones (e.g. iPhone, Android devices)

[0183] Food delivery robot (equipped with a camera)

[0184] Server (data processing, analysis)

[0185] software

[0186] AI analysis models (e.g., TensorFlow, PyTorch)

[0187] Database (e.g. MySQL, PostgreSQL)

[0188] Communication protocol (e.g. HTTP, WebSocket)

[0189] 2. Specific Embodiments of the System

[0190] (1) Data capture and acquisition

[0191] The delivery robot periodically takes photos of the food. The image data is automatically acquired at specified intervals (e.g., every 5 minutes). The acquired image data is temporarily stored and then sent to the server.

[0192] Example: "Take a photo of the food every 5 minutes in 1080p resolution and send it to the server."

[0193] (2) Data storage and analysis

[0194] The server receives the transmitted image data and stores it in a database. The stored data is then analyzed by an AI analysis means. The analysis means uses an AI model (e.g., TensorFlow, PyTorch) to evaluate the condition and quality of the product based on the image data.

[0195] Example: "Analyze the received photo data using an AI model and evaluate the quality of the food. If there is a problem, notify us. Save it on the server."

[0196] (3) Advice generation

[0197] Based on the analysis results, the advice generating means generates specific cooking advice, which is then sent to the smartphone via the communication means.

[0198] Example: "Based on the analysis results, generate specific cooking advice. For example, 'Reduce the cooking time by 1 minute.'"

[0199] (4) Order assistance function

[0200] Furthermore, if the server detects a shortage of ingredients during the AI ​​analysis process, it will use the ordering assistance tool to generate a list of the ingredients that are in short supply, which will also be sent to the smartphone.

[0201] Example: "Detect shortages of materials and generate an order list to notify you."

[0202] 3. Specific Examples

[0203] For example, a smartphone app might send the following prompt to the user:

[0204] "Take a photo of your food every 5 minutes in 1080p resolution and send it to the server."

[0205] "The received photo data is analyzed using an AI model to evaluate the quality of the food. If there is a problem, we will notify you. Please save it on our server."

[0206] "Based on the analysis results, please generate specific cooking advice, such as 'Reduce the cooking time by one minute.'"

[0207] "Detect material shortages and generate an order list to notify you."

[0208] This allows the food delivery robot to periodically take photos of the food and send them to the server. The server then analyzes the image data, evaluates the cooking quality, and generates advice. It can also detect ingredient shortages and generate an ordering list for the necessary ingredients, improving restaurant operation efficiency and customer satisfaction.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] Data capture

[0212] The terminal (serving robot) takes pictures of food at a specified interval (e.g., every 5 minutes). The input is the interval and camera settings (resolution, focal length, etc.). The output is the captured food image data. Specifically, the serving robot automatically starts the camera and presses the capture button.

[0213] Step 2:

[0214] Temporary storage and transmission of data

[0215] The device temporarily stores the captured food image data and then sends it to the server. The input is the food image data acquired in step 1. The output is the food image data sent to the server. Specifically, the device stores the image data in local storage and then uploads the data to the server via the network.

[0216] Step 3:

[0217] Receiving and storing data

[0218] The server receives the food image data sent from the terminal. The input is the food image data sent from the terminal. The output is the food image data stored in the database. Specifically, the server receives the HTTP request and stores the image data in the database.

[0219] Step 4:

[0220] Data Integrity Check

[0221] The server checks the consistency of the stored image data. The input is the food image data stored in the database. The output is food image data whose consistency has been confirmed. Specifically, the server checks the size and format of the data to see if there are any inconsistencies.

[0222] Step 5:

[0223] Image data analysis

[0224] The server analyzes image data using an AI model. The input is food image data whose consistency has been confirmed. The output is the analysis results that evaluate the state and quality of the food. Specifically, the AI ​​analysis means uses TensorFlow and PyTorch to analyze the image and evaluate the color, shape, and presentation.

[0225] Step 6:

[0226] Advice Generation

[0227] The server generates specific cooking advice based on the analysis results. The input is the analysis results obtained in step 5. The output is the generated cooking advice. Specifically, specific points for improvement, such as "Please reduce the baking time by one minute," are generated in text format.

[0228] Step 7:

[0229] Advice Notice

[0230] The server notifies the user's smartphone of the generated cooking advice. The input is the cooking advice generated in step 6. The output is the cooking advice displayed on the smartphone. Specifically, the server sends the advice using a notification protocol, and a pop-up notification is displayed on the smartphone.

[0231] Step 8:

[0232] Activating the ordering assistance function

[0233] If the server detects a shortage of materials during the AI ​​analysis process, it activates the ordering assistance means. The input is the analysis results from step 5 and the materials database. The output is a materials ordering list. Specifically, the server detects a material shortage and automatically generates an ordering list.

[0234] Step 9:

[0235] Order list notification

[0236] The server notifies the user's smartphone of the generated order list. The input is the order list generated in step 8. The output is the order list displayed on the smartphone. Specifically, the server sends the order list using a notification protocol, and it is displayed on the smartphone.

[0237] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0238] This invention relates to a system that uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[0239] System Configuration

[0240] 1. Filming Method

[0241] Terminal

[0242] The camera is attached to the food delivery robot and positioned so that it can capture a wide range of the food delivery area. The camera takes photos and videos at specified intervals and generates the data.

[0243] 2. Data Acquisition Method

[0244] Terminal

[0245] Image and video data captured by the camera is temporarily stored in the device's internal storage, and then uploaded to a server in real time.

[0246] 3. Analysis method

[0247] server

[0248] The server passes the uploaded image data to an AI model for analysis, which evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[0249] 4. Advice Generation Methods

[0250] server

[0251] Based on the analysis results, the server generates specific cooking advice using specific algorithms and evaluation criteria to provide optimal suggestions for the user.

[0252] 5. Emotion Engine

[0253] server

[0254] The emotion engine analyzes sensor information from cameras, microphones, etc. to obtain user emotional data. It recognizes emotions from facial expressions, tone of voice, gestures, etc. and sends the data to the server.

[0255] 6. Means of notification

[0256] server

[0257] Based on the analysis results and emotional data, the server generates cooking advice in the most appropriate format and wording and notifies the user's device.

[0258] Overview of program processing

[0259] The program of this system is designed so that each means cooperates to acquire data, analyze it, generate advice, and notify it. The process flow is explained below with a concrete example.

[0260] 1. Data Acquisition

[0261] The device takes a photo of the food every five minutes and sends the image data in real time to a server, which stores the data in a database.

[0262] 2. Data Analysis

[0263] The server inputs the stored image data into the AI ​​model and analyzes the condition of the food. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that the food is overcooked.

[0264] 3. Emotion Data Acquisition

[0265] The emotion engine uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. For example, if it determines that the user is confused, it sends that data to the server.

[0266] 4. Advice Generation

[0267] The server combines the AI ​​analysis results with the emotion data from the emotion engine to generate cooking advice, such as recommending that users reduce the cooking time by one minute. Confused users are given advice in a polite and calm tone.

[0268] 5. Notification

[0269] The server sends the generated advice to the user's device. For example, a notification may be sent saying, "Please reduce the cooking time by one minute. If you need help, please contact support."

[0270] Application of specific examples

[0271] A restaurant chain is introducing a new food delivery robot system to reduce the workload of its chefs while consistently improving the quality of the food they serve. With this system, as the robot delivers food, it takes a photo of the food with a camera and uses AI to analyze it. Based on the analysis results, it generates specific advice on cooking times and methods, and even provides customized advice based on the chef's emotions. This allows the chef to work efficiently while maintaining a consistently high level of food quality.

[0272] This invention is a system that improves quality control and operational efficiency in restaurants by combining three elements: a food delivery robot, AI analysis, and an emotion engine. It is particularly useful for companies with multiple stores or those expanding overseas.

[0273] The processing flow will be explained below.

[0274] Step 1:

[0275] User

[0276] To perform the initial setup of the system, the user inputs the camera resolution, shooting interval, emotion engine setting parameters, etc. into the server. The user confirms that the initial setup is complete.

[0277] Step 2:

[0278] server

[0279] The server applies the setting parameters received from the user to the camera and AI analysis system, and then conducts a test shoot to confirm that the settings have been correctly applied.

[0280] Step 3:

[0281] Terminal

[0282] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes) and temporarily stores them in its internal storage.

[0283] Step 4:

[0284] Terminal

[0285] The device uploads stored image and video data to a server in real time, and receives feedback to confirm successful data transmission.

[0286] Step 5:

[0287] server

[0288] The server receives the image data sent from the device and stores it in a database. When saving, it checks the integrity of the data and makes sure there are no missing parts.

[0289] Step 6:

[0290] server

[0291] The server passes the stored image data to the AI ​​model to begin analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and outputs the results.

[0292] Step 7:

[0293] server

[0294] The server receives the results of the AI ​​analysis and records the condition of the food determined from the image data (e.g., overcooked, messy presentation, etc.) as the analysis result.

[0295] Step 8:

[0296] Terminal

[0297] The emotion engine installed in the device uses a camera and microphone to analyze the user's facial expressions and tone of voice in real time and generate emotion data.

[0298] Step 9:

[0299] Terminal

[0300] The device transmits the generated emotion data to the server, which conveys the user's current emotional state (e.g., confusion, joy, anger, etc.) to the server.

[0301] Step 10:

[0302] server

[0303] The server combines the AI ​​analysis results with emotional data to generate appropriate cooking advice. For example, if the user is confused, the server generates advice in a polite and calm tone. If the analysis results indicate that the cooking time should be shortened by one minute, this advice will also be included.

[0304] Step 11:

[0305] server

[0306] The generated cooking advice is sent to the user's device, and the wording and format of the notification are adjusted based on the emotion data.

[0307] Step 12:

[0308] User

[0309] The user receives the advice notification from the server, checks the content, and, if necessary, modifies the cooking method based on the advice.

[0310] Application of specific examples

[0311] For example, if a restaurant chain were to implement this system, while testing a new menu item, the delivery robot would take photos of the food every five minutes and send them to the server. The server would then analyze these photos using an AI model and determine whether the food was overcooked. Furthermore, an emotion engine would analyze the chef's facial expressions and detect whether the chef was confused. Based on this data, the server would generate a gentle message to the user, saying, "Please reduce the cooking time by one minute. If you have any questions, please contact support." The chef would then check the notification and shorten the cooking time the next time, thereby improving the quality of the food.

[0312] In this way, this system combines food delivery robots, AI analysis, and an emotion engine to significantly improve quality control and operational efficiency in restaurants.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] In restaurants, consistently improving the quality of the food served while reducing the workload of chefs is an important issue. Furthermore, when chefs or employees are stressed or overwhelmed, support is needed to reduce their workload and enable them to work efficiently. Current technology lacks a system that satisfies these needs, so a new system is needed that simultaneously manages food quality and provides psychological support to chefs.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0317] In this invention, the server includes a photographing means, a data acquisition means, an analysis means, an advice generation means, an emotion recognition means, and a notification means, which makes it possible to provide customized advice according to the emotional state of the chef or staff, in addition to evaluating the quality of the food and providing cooking advice.

[0318] The "photography means" is a device that is attached to the food delivery robot and periodically takes photos and videos of the food.

[0319] The "data acquisition means" is a function that records image data and video data acquired by the image capture means and transmits them to a server as necessary.

[0320] The "analysis means" refers to an analysis function that includes an AI model that evaluates the condition and quality of food based on image and video data uploaded to the server.

[0321] The "advice generating means" is a function that generates specific cooking advice for the user based on the analysis results by the analyzing means.

[0322] The "emotion recognition means" is a function that analyzes emotional data such as the user's facial expressions, tone of voice, and gestures, and customizes advice based on the analysis results.

[0323] The "notification means" is a function that notifies the user's terminal of the generated cooking advice.

[0324] This system uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[0325] First, the device uses its camera to periodically take photos and videos of the food. The camera is attached to the food delivery robot and is positioned so that it can capture a wide range of the food delivery area. Specifically, the camera takes photos every five minutes, and the captured images and video data are temporarily stored in the device's internal storage. The data is then uploaded to the server in real time.

[0326] The server stores the received image and video data in a database. The stored data is then input into an AI model to evaluate the state and quality of the food. The AI ​​model uses deep learning, for example, to analyze the color, shape, and presentation of the image. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that it is "overcooked."

[0327] The device then uses its camera and microphone to record the user's facial expressions and tone of voice, and sends the data in real time to a server. The server then uses an emotion engine to analyze the user's emotional state. For example, if the server detects that the user is confused, it will use that information.

[0328] As a result, the server combines the results of the food's condition analysis with the user's emotional data to generate specific cooking advice. For example, if the food is judged to be "overcooked," it generates advice such as "reduce the cooking time by one minute." Furthermore, if the user is confused, the advice will be more polite, such as "reduce the cooking time by one minute. If you have any questions, please contact support."

[0329] Finally, the server sends the generated advice to the user's device, which then provides the advice to the user using notification functions and alerts.

[0330] Specific examples

[0331] For example, a restaurant chain is introducing a new robotic food delivery system. This system reduces the workload of chefs and consistently improves the quality of the food they serve. A camera attached to the robot periodically photographs the food and sends the image data to a server in real time. The server uses an AI model to analyze the images and generate specific cooking advice. It also analyzes the chef's facial expressions and tone of voice to provide customized advice based on their emotions.

[0332] Prompt Sentence Examples

[0333] "Generate suggestions for today's doneness based on photos of food taken. If the user is confused about what to cook, please provide polite suggestions taking their situation into consideration."

[0334] This allows the system to reduce the workload of the chef while maintaining consistently high food quality.

[0335] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0336] Step 1: Data Acquisition

[0337] The device takes photos and videos of the food every five minutes. For example, it uses a built-in camera to take photos of the entire serving area. The input is image data and video data captured by the camera. The output is that this photographed data is temporarily saved in the built-in storage. Specifically, the camera shutter is automatically released and data is collected at specified intervals.

[0338] Step 2: Send data

[0339] The device sends the temporarily saved image and video data to the server. The input is the captured image data obtained in step 1. The output is the image and video data uploaded to the server. The specific operation is that the data is transferred to the server via an HTTP POST request.

[0340] Step 3: Save Data

[0341] The server stores the received image data and video data in a database. The input is the data sent in step 2. The output is the image data and video data organized and stored in the database. The specific operation is to store and organize the data through a database management system (DBMS).

[0342] Step 4: Data analysis

[0343] The server inputs the stored image and video data into the AI ​​model to analyze the condition and quality of the food. The input is the image and video data stored in the database. The output is the analysis results of the food condition (e.g., color, shape, presentation). Specifically, it uses a deep learning model to identify the characteristics of the image and make an evaluation based on that.

[0344] Step 5: Acquire emotion data

[0345] It uses the device's camera and microphone to record the user's facial expressions and tone of voice. The input is the user's facial and voice data. The output is emotion data that is sent to the server. Specifically, it uses advanced facial recognition algorithms and tone of voice analysis software to recognize emotions and transfers the data to the server.

[0346] Step 6: Sentiment Data Analysis

[0347] The server uses the emotion engine to analyze the user's emotional state. The input is the emotion data obtained in step 5. The output is the analyzed user's emotional state. Specifically, the server infers the user's psychological state from facial expression data and tone of voice, and analyzes this information using the analysis engine.

[0348] Step 7: Advice Generation

[0349] The server integrates the cooking state analysis results with the user's emotional data to generate cooking advice. The input is the analysis results from steps 4 and 6. The output is specific cooking advice to provide to the user. The specific operation is to analyze the analysis results using a specific algorithm and generate the optimal cooking method. For example, the advice generated is "Please reduce the cooking time by 1 minute."

[0350] Step 8: Advice Notification

[0351] The server sends the generated advice to the user's device. The input is the cooking advice generated in step 7. The output is a notification displayed on the user's device. The specific operation is to send the advice to the device via a communication protocol such as an HTTP POST request, and the device notifies the user. For example, the notification may say, "Please reduce the cooking time by one minute. If you need help, please contact support."

[0352] (Application example 2)

[0353] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Existing restaurant systems using food delivery robots have the problem of insufficient food quality control. They also lack specific cooking advice to maintain consistent food quality. Furthermore, they do not provide services that take into account the user's emotional state, making it difficult to improve the user experience.

[0355] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0356] In this invention, the server includes a camera attached to the food delivery robot, a data acquisition means for periodically acquiring image data captured by the camera, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results of the analysis means, a notification means for notifying the user of the cooking advice, an emotion analysis means for analyzing emotional data such as the user's facial expression and tone of voice and customizing the cooking advice, and an advice customization means for optimizing the cooking advice notified to the user's device to match the user's emotional state. This makes it possible to monitor the condition of the food in real time, detect quality defects early, and provide cooking advice tailored to the user's emotional state, improving the user experience.

[0357] A "serving robot" is an automated mechanical device designed to deliver food and drinks.

[0358] "Capture means" refers to the camera or sensor device used to capture images or videos.

[0359] The "data acquisition means" is a system for collecting and storing image data and video data obtained by the imaging means.

[0360] The "analysis means" refers to software or hardware for processing acquired image data or video data and extracting specific information.

[0361] The "advice generator" is a system that generates instructions to recommend specific actions or changes based on the results obtained by the analysis means.

[0362] The "notification means" is a communication means for conveying the generated advice or instructions to the user.

[0363] The "emotion analysis means" is a system for analyzing the user's facial expression, tone of voice, etc., to determine the user's emotional state.

[0364] The "advice customization means" is a system for optimizing the advice and instructions to be notified based on the user's emotional data obtained by the emotion analysis means.

[0365] A "user terminal" is a device used by a user to receive information, such as a smartphone or tablet.

[0366] This invention aims to improve quality control and user experience in the food delivery industry by using a camera attached to a food delivery robot, an AI analysis system, and an emotion analysis system. This system monitors the cooking status in real time and provides advice based on emotion analysis. Specific methods for hardware, software, data processing, and data calculation used to implement this invention are described below.

[0367] Hardware

[0368] Food delivery robot: an automated mechanical device for delivering food, equipped with cameras and other sensors.

[0369] Camera: A means of capturing images attached to the robot. For example, a webcam such as the Logitech C920.

[0370] Terminal: A device used to collect and process data and upload it to a server. An example is a single-board computer such as a Raspberry Pi.

[0371] Microphone: An input device for capturing the tone of a user's voice. For example, a high-quality microphone such as a Blue Yeti.

[0372] software

[0373] OpenCV: An open-source library for image capture and processing.

[0374] Requests: A Python library for communicating with the server.

[0375] TensorFlow / Keras: Machine learning frameworks that form the basis of the sentiment analysis model and cooking analysis AI model.

[0376] Custom Emotion Recognition Model: A model for recognizing a user's emotions from facial expressions and tone of voice.

[0377] Custom AI Model: An artificial intelligence model for analyzing the state of cooking and generating cooking advice.

[0378] Data processing and calculation

[0379] Data acquisition: The device periodically (e.g., every 5 minutes) takes pictures of the food using its camera, temporarily stores them in storage, and then uploads them to the server.

[0380] Data analysis: The server passes the uploaded image data to the analysis means to evaluate the state of the food (color, shape, presentation). The AI ​​model does this using specific algorithms.

[0381] Emotion data acquisition: The emotion analysis means uses a camera and microphone to collect the user's facial expressions and tone of voice, determine the user's emotional state, and send it to the server.

[0382] Advice generation: The server generates cooking advice based on the analysis results and emotion data. For example, it generates specific advice such as "We recommend shortening the cooking time by one minute."

[0383] Advice notification: The generated advice is sent to the user's device. If an emotion such as confusion is recognized, the advice is notified in a way that takes into consideration that emotion.

[0384] Specific examples

[0385] For example, a food delivery robot takes a photo of the food during delivery and sends it to a server, where an AI model analyzes it. Based on the analysis results, advice such as "reheating is necessary" is generated. If the user is confused, the emotion analysis system can detect this and provide additional support, such as "we'll explain in detail how to reheat it."

[0386] Prompt Sentence Examples

[0387] Below are some example prompts for the generative AI model and sentiment analysis system:

[0388] text

[0389] Provide a photo and generate optimal cooking advice based on emotional data from our emotion engine. We evaluate whether the food is properly cooked (color, shape, presentation, etc.) and take into account the user's feelings of confusion or dissatisfaction to customize the advice.

[0390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0391] Step 1:

[0392] The device uses the camera to take pictures of the food at regular intervals (e.g., every 5 minutes). The captured images are temporarily stored in the device's storage. The input is image data acquired from the camera, and the output is an image file saved in the storage.

[0393] Step 2:

[0394] The device uploads the temporarily saved image data to the server by executing an HTTP POST request using the Requests library. The input is the image file in storage, and the output is a notification to the server that the upload is complete.

[0395] Step 3:

[0396] The server receives the uploaded image data and stores it in a database. The input is the image file sent from the device, and the output is the image data stored in the database. Specifically, data is received through a RESTful API.

[0397] Step 4:

[0398] The server passes the stored image data to an AI model, which is the analytical tool, and evaluates the condition of the food (color, shape, presentation). The input is the image data in the database, and the output is the analysis result (e.g., overcooked, poor color, etc.). Specifically, the analysis is performed using a machine learning framework such as TensorFlow or Keras.

[0399] Step 5:

[0400] The server generates cooking advice based on the analysis results. This advice is calculated using a specific algorithm. The input is the analysis results, and the output is specific cooking advice (e.g., "reduce the cooking time by 1 minute").

[0401] Step 6:

[0402] At the same time, the server uses emotion analysis means to obtain the user's emotion data. It collects and analyzes the user's facial expressions and tone of voice through a camera or microphone. The input is the facial expression and voice data obtained from the camera or microphone, and the output is the user's emotional state (e.g., confusion, joy). Specific operations use an emotion analysis model.

[0403] Step 7:

[0404] The server generates customized cooking advice that takes emotion data into consideration. The inputs are the analysis results and emotion data, and the output is emotion-sensitive cooking advice (e.g., "I'll explain it carefully, but please reduce the cooking time by 1 minute").

[0405] Step 8:

[0406] The server notifies the user's terminal of the generated cooking advice. The notification means is used to send emotion-conscious advice. The input is customized cooking advice, and the output is a notification message to the user's terminal.

[0407] Step 9:

[0408] The user receives the advice on the device and adjusts the cooking accordingly, specifically by referring to the message displayed on the device.

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0412] [Second embodiment]

[0413] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0414] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0420] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0421] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0424] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0425] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, and a notification means.

[0426] Overview of program processing

[0427] System Setup

[0428] server

[0429] 1. The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system.

[0430] 2. The server performs a test shoot to verify that the camera settings were applied correctly.

[0431] Specific examples

[0432] The server receives instructions from the user such as "Set the camera resolution to 1080p and the shooting interval to 5 minutes," and reflects these instructions on the camera. It then takes a test shot to check whether the settings have been correctly reflected.

[0433] Data capture and acquisition

[0434] Terminal

[0435] 1. The device takes images and videos of the food being served at the specified interval (e.g., every 5 minutes).

[0436] 2. The device temporarily stores the captured data.

[0437] Specific examples

[0438] The device takes a photo of the food every five minutes and temporarily stores the photo data.

[0439] server

[0440] 1. The server receives the photo and video data sent from the device.

[0441] 2. The server stores the received data in a database and checks the integrity of the data.

[0442] Specific examples

[0443] The server receives the photos of the food sent from the device and stores the photo data in a database. It checks that there are no missing data.

[0444] Data analysis and advice generation

[0445] server

[0446] 1. The server passes the stored data to the AI ​​model and begins analysis.

[0447] 2. The server uses an AI model to evaluate the condition and quality of the food (color, shape, presentation, etc.).

[0448] 3. The server receives the analysis results and detects defects and areas for improvement as necessary.

[0449] Specific examples

[0450] The server uses an AI model to analyze the photos and detect problems such as "the food is overcooked" or "the presentation is messy."

[0451] server

[0452] 1. The server generates specific cooking advice based on the analysis results.

[0453] 2. The server formats the cooking advice into a format that can be communicated to the user.

[0454] Specific examples

[0455] Based on the analysis result that the food is "overcooked," the server generates specific advice such as "reduce the cooking time by one minute" and notifies the user.

[0456] Advice notification and response

[0457] server

[0458] 1. The server sends the generated advice to the user's terminal.

[0459] 2. The server ensures that the notification is displayed properly.

[0460] User

[0461] 1. The user receives and confirms the advice notification from the server.

[0462] 2. The user adjusts the cooking method according to the received advice.

[0463] Specific examples

[0464] The user receives a notification to "reduce the bake time by 1 minute" and reduces the bake time accordingly the next time they cook.

[0465] Order assistance function

[0466] server

[0467] 1. If the server detects a shortage of materials during AI analysis, it will activate the ordering assistance function.

[0468] 2. The server generates an order list of materials and sends it to the user.

[0469] User

[0470] 1. The user receives the order list, checks the required materials, and places an order.

[0471] Specific examples

[0472] The server detects that there is a shortage of fresh lemons, generates an order list, and sends it to the user, who then checks it and orders only the amount they need.

[0473] summary

[0474] This invention is a system that uses a food delivery robot to automatically check product quality and provide specific cooking advice based on the analysis results of AI. This will improve the efficiency of quality control in restaurants and reduce the burden on employees. In addition, if a shortage of ingredients is detected, an ordering assistance function will be provided, further improving the efficiency of ingredient management.

[0475] The processing flow will be explained below.

[0476] Step 1:

[0477] server

[0478] The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system. It also performs test shooting to confirm that the settings are correctly reflected.

[0479] Step 2:

[0480] Terminal

[0481] The device takes pictures and videos of the food being served at a specified interval (e.g., every 5 minutes), and temporarily stores the captured data in its internal storage.

[0482] Step 3:

[0483] Terminal

[0484] The device transmits the photos and videos stored in its internal storage to the server in real time, and receives feedback to confirm that the data transmission was successful.

[0485] Step 4:

[0486] server

[0487] The server receives the photo and video data sent from the device and stores it in a database. When saving, it checks the integrity of the data to ensure there are no missing or error data.

[0488] Step 5:

[0489] server

[0490] The server then passes the stored data to the AI ​​model, which then evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[0491] Step 6:

[0492] server

[0493] The server receives the analysis results from the AI ​​model and uses them to detect defects and areas for improvement, such as identifying problems like "overcooked" or "messy presentation."

[0494] Step 7:

[0495] server

[0496] The server generates specific cooking advice based on the analysis results, and the advice is formatted in a way that is easy for the user to understand.

[0497] Step 8:

[0498] server

[0499] The server sends the generated cooking advice to the user's device and monitors the delivery status to ensure that the notification is displayed properly.

[0500] Step 9:

[0501] User

[0502] The user receives the advice notification from the server, checks the contents, and, if necessary, modifies the cooking method based on the advice.

[0503] Step 10:

[0504] server

[0505] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function, generates an appropriate ordering list for materials, and sends it to the user.

[0506] Step 11:

[0507] User

[0508] The user receives the order list sent from the server, checks the required materials, and then takes steps to order the materials.

[0509] summary

[0510] Through these steps, the present invention utilizes a food delivery robot to monitor product availability in real time, provide specific cooking advice based on AI analysis results, and detect ingredient shortages and assist with ordering, thereby improving quality control and operational efficiency.

[0511] Example 1

[0512] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0513] Traditionally, quality control processes in restaurants have often been performed manually, resulting in issues such as wasted human resources and inaccurate quality assessment. Furthermore, there was a lack of systems for early detection of defects and providing specific advice for improvement, making it difficult to maintain consistent quality. Furthermore, there was also a lack of systems for timely detection of ingredient shortages and appropriate ordering, resulting in inefficient ingredient management.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0515] In this invention, the server includes a setting application means that receives settings for the image capture means from the user and applies them to the camera and analysis system, a captured data storage means that temporarily stores data, a database storage means that stores image data sent from the terminal in a database and checks consistency, and an ingredient detection means and ordering assistance means that detect ingredient shortages during analysis and provide an ordering assistance function. This automates and streamlines quality control in restaurants, enables the provision of specific cooking advice, and enables timely detection of ingredient shortages and effective ingredient management.

[0516] The "photography means" is a device that is attached to the food delivery robot and periodically takes pictures and videos.

[0517] The "data acquisition means" is a mechanism for collecting and storing image data acquired by the imaging means.

[0518] An "analysis means" is a system or algorithm for analyzing acquired image data and evaluating its content.

[0519] The "advice generating means" is a function that generates specific cooking advice based on the analysis results of the analyzing means.

[0520] The "notification means" is a system for notifying and displaying the generated cooking advice on the user's terminal.

[0521] The "setting application means" is a mechanism that reflects the camera setting parameters received from the user in the food delivery robot and analysis system.

[0522] The "photographing data storage means" is a storage system for temporarily storing image data obtained by the photographing means.

[0523] The "database storage means" is a mechanism that stores image data sent from the terminal in a database and checks its consistency.

[0524] "Material detection means" means a system or algorithm for detecting material shortages during analysis.

[0525] The "order assist means" is a function that creates an order list based on the shortage materials detected by the material detection means and prompts the user to place an order.

[0526] The present invention is a system that analyzes image data captured by a food delivery robot, provides cooking advice based on the results, and also manages ingredients. Specifically, this system is comprised of an image capture means, data acquisition means, analysis means, advice generation means, notification means, setting application means, captured data storage means, database storage means, ingredient detection means, and ordering assistance means attached to the food delivery robot.

[0527] server

[0528] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval) and applies them to the camera and AI analysis system. This ensures that the shooting method operates with the correct settings. The server then takes a test shot and checks the results to ensure the settings were applied properly.

[0529] Terminal

[0530] The device periodically captures images and videos of the food being served according to the specified interval. For example, the device takes a photo of the food every five minutes and temporarily stores the photo data in local storage. The captured data is periodically sent to the server.

[0531] Data storage and analysis

[0532] The server receives the photo and video data sent from the device and stores it in a database. At this time, the data is checked for consistency and to ensure there are no missing pieces. The saved data is then passed to the AI ​​model, which begins analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and returns the results to the server. Specifically, it may give an evaluation such as "the food is overcooked" or "the presentation is messy."

[0533] Cooking advice generation and notification

[0534] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. For example, the advice might be, "Reduce the cooking time by one minute." The generated cooking advice is then sent to the user's device. The user receives the notification, checks the content, and adjusts the cooking method accordingly for the next time.

[0535] Material detection and ordering assistance

[0536] Furthermore, if a shortage of ingredients is detected during AI analysis, the server will activate the ordering assistance function. For example, if it detects that there is a shortage of fresh lemons, a list of ingredients to order will be generated and sent to the user. The user can then check the list and order the necessary ingredients.

[0537] As described above, this system automates and streamlines quality control in restaurants. It also provides specific cooking advice, which is expected to improve cooking quality. Furthermore, as soon as a shortage of ingredients is detected, appropriate action can be taken immediately, which also improves the efficiency of ingredient management.

[0538] Prompt Sentence Examples

[0539] "We would like to build a system that can analyze photos of food taken by a food delivery robot and evaluate their quality. Based on that, it can provide cooking advice based on the needs of the customer. The system should evaluate factors such as cooking time, presentation, and color, and generate specific advice."

[0540] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0541] Step 1: Apply camera settings

[0542] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval). The input is the camera parameters set by the user, and the output is the state in which the settings have been applied to the camera. The server sends the settings via the camera's API, and the settings are changed. Specifically, the server receives a request for camera settings and sends an HTTP request to the corresponding endpoint to reflect the settings.

[0543] Step 2: Conduct a test shoot

[0544] The server takes a test shot to confirm that the camera settings have been applied correctly and checks the results. The input is the state with the camera settings applied, and the output is the test image taken. The server sends a command to the camera to take a test shot, displays the acquired image in real time, and checks whether the settings have been applied.

[0545] Step 3: Conduct regular photo shoots

[0546] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes). The input is the interval based on the internal clock, and the output is the captured image and video data. Specifically, the device sets a timer, starts the camera to take a photo each time the timer expires, and saves the captured data in local storage.

[0547] Step 4: Sending and Receiving Data

[0548] The device temporarily stores the captured data and then sends it to the server. The server receives the data sent from the device, stores it in a database, and checks its consistency. The input is the image data generated by the device, and the output is stored in the database within the server. Specifically, the device sends data to the server using an HTTP POST request, and the server receives the data and inserts it into the database.

[0549] Step 5: Analysis by AI model

[0550] The server passes the stored data to the AI ​​model and begins analysis. The input is the image data stored in the database, and the output is the analysis results returned by the AI ​​model. The server sends the data to the AI ​​model's API endpoint, and the model evaluates the condition and quality of the food (color, shape, presentation, etc.), and receives the results.

[0551] Step 6: Generate cooking advice

[0552] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. The input is the analysis results of the AI ​​model, and the output is the cooking advice to be provided to the user. Specifically, the server uses a template to interpret the analysis results and generates specific advice such as "Please reduce the cooking time by one minute."

[0553] Step 7: Advice Notification

[0554] The server sends the generated advice to the user's device. The user checks that the notification is displayed properly and modifies the cooking method according to the received advice. The input is the generated cooking advice, and the output is the notification displayed on the user's device. Specifically, the server sends the advice via email or app notification, and the user checks the notification.

[0555] Step 8: Assist with ordering materials

[0556] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function. The input is the analysis results of the AI ​​model and material inventory data, and the output is an ordering list provided to the user. Specifically, the server generates an ordering list based on the material detection results and sends it to the user. The user receives the ordering list, checks the required materials, and places an order.

[0557] Through these steps, the system can automatically check product quality, provide specific cooking advice, and streamline ingredient management.

[0558] (Application example 1)

[0559] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0560] In restaurants using conventional food delivery robots, product quality management and cooking improvements were sometimes not properly carried out. It was also difficult to respond quickly to ingredient shortages or provide specific advice to staff. As a result, problems arose, such as a decrease in customer satisfaction and a loss of store operational efficiency. The present invention aims to solve these problems and improve the efficiency of cooking quality management, the provision of improvement advice, and ingredient management in restaurants.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0562] In this invention, the server includes a photographing means attached to the food delivery robot, a data acquisition means for periodically acquiring photographed image data, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results by the analysis means, a notification means for notifying the user of the cooking advice, an ordering assistance means for detecting ingredient shortages and generating an ordering list, a communication means for communicating with the server to send and receive food images, and an AI analysis means for performing image analysis using an AI model. This allows for efficient provision of advice to improve cooking quality and ingredient management, thereby improving restaurant operation efficiency and customer satisfaction.

[0563] The "photography means" is a device that is attached to the food delivery robot and periodically acquires image data.

[0564] The "data acquisition means" is a device or program that receives image data captured by the image capture means and stores or transmits the data.

[0565] The "analysis means" is a program or device for evaluating the condition and quality of a product using the acquired image data.

[0566] The "advice generating means" is a program or device that generates specific cooking advice based on the analysis results from the analyzing means.

[0567] The "notification means" refers to a device or program for transmitting the generated cooking advice to the user.

[0568] The "order assistance means" is a program or device for detecting shortages of materials and generating an order list for the necessary materials.

[0569] The "communication means" refers to devices and programs for transmitting and receiving cooking images to and from the server via communication.

[0570] "AI analysis means" refers to a program or device that uses an AI model to analyze image data and evaluate the condition and quality of a product.

[0571] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, a notification means, an order assistance means, a communication means, and an AI analysis means.

[0572] 1. Hardware and Software Used

[0573] Hardware

[0574] Smartphones (e.g. iPhone, Android devices)

[0575] Food delivery robot (equipped with a camera)

[0576] Server (data processing, analysis)

[0577] software

[0578] AI analysis models (e.g., TensorFlow, PyTorch)

[0579] Database (e.g. MySQL, PostgreSQL)

[0580] Communication protocol (e.g. HTTP, WebSocket)

[0581] 2. Specific Embodiments of the System

[0582] (1) Data capture and acquisition

[0583] The delivery robot periodically takes photos of the food. The image data is automatically acquired at specified intervals (e.g., every 5 minutes). The acquired image data is temporarily stored and then sent to the server.

[0584] Example: "Take a photo of the food every 5 minutes in 1080p resolution and send it to the server."

[0585] (2) Data storage and analysis

[0586] The server receives the transmitted image data and stores it in a database. The stored data is then analyzed by an AI analysis means. The analysis means uses an AI model (e.g., TensorFlow, PyTorch) to evaluate the condition and quality of the product based on the image data.

[0587] Example: "Analyze the received photo data using an AI model and evaluate the quality of the food. If there is a problem, notify us. Save it on the server."

[0588] (3) Advice generation

[0589] Based on the analysis results, the advice generating means generates specific cooking advice, which is then sent to the smartphone via the communication means.

[0590] Example: "Based on the analysis results, generate specific cooking advice. For example, 'Reduce the cooking time by 1 minute.'"

[0591] (4) Order assistance function

[0592] Furthermore, if the server detects a shortage of ingredients during the AI ​​analysis process, it will use the ordering assistance tool to generate a list of the ingredients that are in short supply, which will also be sent to the smartphone.

[0593] Example: "Detect shortages of materials and generate an order list to notify you."

[0594] 3. Specific Examples

[0595] For example, a smartphone app might send the following prompt to the user:

[0596] "Take a photo of your food every 5 minutes in 1080p resolution and send it to the server."

[0597] "The received photo data is analyzed using an AI model to evaluate the quality of the food. If there is a problem, we will notify you. Please save it on our server."

[0598] "Based on the analysis results, please generate specific cooking advice, such as 'Reduce the cooking time by one minute.'"

[0599] "Detect material shortages and generate an order list to notify you."

[0600] This allows the food delivery robot to periodically take photos of the food and send them to the server. The server then analyzes the image data, evaluates the cooking quality, and generates advice. It can also detect ingredient shortages and generate an ordering list for the necessary ingredients, improving restaurant operation efficiency and customer satisfaction.

[0601] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0602] Step 1:

[0603] Data capture

[0604] The terminal (serving robot) takes pictures of food at a specified interval (e.g., every 5 minutes). The input is the interval and camera settings (resolution, focal length, etc.). The output is the captured food image data. Specifically, the serving robot automatically starts the camera and presses the capture button.

[0605] Step 2:

[0606] Temporary storage and transmission of data

[0607] The device temporarily stores the captured food image data and then sends it to the server. The input is the food image data acquired in step 1. The output is the food image data sent to the server. Specifically, the device stores the image data in local storage and then uploads the data to the server via the network.

[0608] Step 3:

[0609] Receiving and storing data

[0610] The server receives the food image data sent from the terminal. The input is the food image data sent from the terminal. The output is the food image data stored in the database. Specifically, the server receives the HTTP request and stores the image data in the database.

[0611] Step 4:

[0612] Data Integrity Check

[0613] The server checks the consistency of the stored image data. The input is the food image data stored in the database. The output is food image data whose consistency has been confirmed. Specifically, the server checks the size and format of the data to see if there are any inconsistencies.

[0614] Step 5:

[0615] Image data analysis

[0616] The server analyzes image data using an AI model. The input is food image data whose consistency has been confirmed. The output is the analysis results that evaluate the state and quality of the food. Specifically, the AI ​​analysis means uses TensorFlow and PyTorch to analyze the image and evaluate the color, shape, and presentation.

[0617] Step 6:

[0618] Advice Generation

[0619] The server generates specific cooking advice based on the analysis results. The input is the analysis results obtained in step 5. The output is the generated cooking advice. Specifically, specific points for improvement, such as "Please reduce the baking time by one minute," are generated in text format.

[0620] Step 7:

[0621] Advice Notice

[0622] The server notifies the user's smartphone of the generated cooking advice. The input is the cooking advice generated in step 6. The output is the cooking advice displayed on the smartphone. Specifically, the server sends the advice using a notification protocol, and a pop-up notification is displayed on the smartphone.

[0623] Step 8:

[0624] Activating the ordering assistance function

[0625] If the server detects a shortage of materials during the AI ​​analysis process, it activates the ordering assistance means. The input is the analysis results from step 5 and the materials database. The output is a materials ordering list. Specifically, the server detects a material shortage and automatically generates an ordering list.

[0626] Step 9:

[0627] Order list notification

[0628] The server notifies the user's smartphone of the generated order list. The input is the order list generated in step 8. The output is the order list displayed on the smartphone. Specifically, the server sends the order list using a notification protocol, and it is displayed on the smartphone.

[0629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0630] This invention relates to a system that uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[0631] System Configuration

[0632] 1. Filming Method

[0633] Terminal

[0634] The camera is attached to the food delivery robot and positioned so that it can capture a wide range of the food delivery area. The camera takes photos and videos at specified intervals and generates the data.

[0635] 2. Data Acquisition Method

[0636] Terminal

[0637] Image and video data captured by the camera is temporarily stored in the device's internal storage, and then uploaded to a server in real time.

[0638] 3. Analysis method

[0639] server

[0640] The server passes the uploaded image data to an AI model for analysis, which evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[0641] 4. Advice Generation Methods

[0642] server

[0643] Based on the analysis results, the server generates specific cooking advice using specific algorithms and evaluation criteria to provide optimal suggestions for the user.

[0644] 5. Emotion Engine

[0645] server

[0646] The emotion engine analyzes sensor information from cameras, microphones, etc. to obtain user emotional data. It recognizes emotions from facial expressions, tone of voice, gestures, etc. and sends the data to the server.

[0647] 6. Means of notification

[0648] server

[0649] Based on the analysis results and emotional data, the server generates cooking advice in the most appropriate format and wording and notifies the user's device.

[0650] Overview of program processing

[0651] The program of this system is designed so that each means cooperates to acquire data, analyze it, generate advice, and notify it. The process flow is explained below with a concrete example.

[0652] 1. Data Acquisition

[0653] The device takes a photo of the food every five minutes and sends the image data in real time to a server, which stores the data in a database.

[0654] 2. Data Analysis

[0655] The server inputs the stored image data into the AI ​​model and analyzes the condition of the food. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that the food is overcooked.

[0656] 3. Emotion Data Acquisition

[0657] The emotion engine uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. For example, if it determines that the user is confused, it sends that data to the server.

[0658] 4. Advice Generation

[0659] The server combines the AI ​​analysis results with the emotion data from the emotion engine to generate cooking advice, such as recommending that users reduce the cooking time by one minute. Confused users are given advice in a polite and calm tone.

[0660] 5. Notification

[0661] The server sends the generated advice to the user's device. For example, a notification may be sent saying, "Please reduce the cooking time by one minute. If you need help, please contact support."

[0662] Application of specific examples

[0663] A restaurant chain is introducing a new food delivery robot system to reduce the workload of its chefs while consistently improving the quality of the food they serve. With this system, as the robot delivers food, it takes a photo of the food with a camera and uses AI to analyze it. Based on the analysis results, it generates specific advice on cooking times and methods, and even provides customized advice based on the chef's emotions. This allows the chef to work efficiently while maintaining a consistently high level of food quality.

[0664] This invention is a system that improves quality control and operational efficiency in restaurants by combining three elements: a food delivery robot, AI analysis, and an emotion engine. It is particularly useful for companies with multiple stores or those expanding overseas.

[0665] The processing flow will be explained below.

[0666] Step 1:

[0667] User

[0668] To perform the initial setup of the system, the user inputs the camera resolution, shooting interval, emotion engine setting parameters, etc. into the server. The user confirms that the initial setup is complete.

[0669] Step 2:

[0670] server

[0671] The server applies the setting parameters received from the user to the camera and AI analysis system, and then conducts a test shoot to confirm that the settings have been correctly applied.

[0672] Step 3:

[0673] Terminal

[0674] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes) and temporarily stores them in its internal storage.

[0675] Step 4:

[0676] Terminal

[0677] The device uploads stored image and video data to a server in real time, and receives feedback to confirm successful data transmission.

[0678] Step 5:

[0679] server

[0680] The server receives the image data sent from the device and stores it in a database. When saving, it checks the integrity of the data and makes sure there are no missing parts.

[0681] Step 6:

[0682] server

[0683] The server passes the stored image data to the AI ​​model to begin analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and outputs the results.

[0684] Step 7:

[0685] server

[0686] The server receives the results of the AI ​​analysis and records the condition of the food determined from the image data (e.g., overcooked, messy presentation, etc.) as the analysis result.

[0687] Step 8:

[0688] Terminal

[0689] The emotion engine installed in the device uses a camera and microphone to analyze the user's facial expressions and tone of voice in real time and generate emotion data.

[0690] Step 9:

[0691] Terminal

[0692] The device transmits the generated emotion data to the server, which conveys the user's current emotional state (e.g., confusion, joy, anger, etc.) to the server.

[0693] Step 10:

[0694] server

[0695] The server combines the AI ​​analysis results with emotional data to generate appropriate cooking advice. For example, if the user is confused, the server generates advice in a polite and calm tone. If the analysis results indicate that the cooking time should be shortened by one minute, this advice will also be included.

[0696] Step 11:

[0697] server

[0698] The generated cooking advice is sent to the user's device, and the wording and format of the notification are adjusted based on the emotion data.

[0699] Step 12:

[0700] User

[0701] The user receives the advice notification from the server, checks the content, and, if necessary, modifies the cooking method based on the advice.

[0702] Application of specific examples

[0703] For example, if a restaurant chain were to implement this system, while testing a new menu item, the delivery robot would take photos of the food every five minutes and send them to the server. The server would then analyze these photos using an AI model and determine whether the food was overcooked. Furthermore, an emotion engine would analyze the chef's facial expressions and detect whether the chef was confused. Based on this data, the server would generate a gentle message to the user, saying, "Please reduce the cooking time by one minute. If you have any questions, please contact support." The chef would then check the notification and shorten the cooking time the next time, thereby improving the quality of the food.

[0704] In this way, this system combines food delivery robots, AI analysis, and an emotion engine to significantly improve quality control and operational efficiency in restaurants.

[0705] Example 2

[0706] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0707] In restaurants, consistently improving the quality of the food served while reducing the workload of chefs is an important issue. Furthermore, when chefs or employees are stressed or overwhelmed, support is needed to reduce their workload and enable them to work efficiently. Current technology lacks a system that satisfies these needs, so a new system is needed that simultaneously manages food quality and provides psychological support to chefs.

[0708] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0709] In this invention, the server includes a photographing means, a data acquisition means, an analysis means, an advice generation means, an emotion recognition means, and a notification means, which makes it possible to provide customized advice according to the emotional state of the chef or staff, in addition to evaluating the quality of the food and providing cooking advice.

[0710] The "photography means" is a device that is attached to the food delivery robot and periodically takes photos and videos of the food.

[0711] The "data acquisition means" is a function that records image data and video data acquired by the image capture means and transmits them to a server as necessary.

[0712] The "analysis means" refers to an analysis function that includes an AI model that evaluates the condition and quality of food based on image and video data uploaded to the server.

[0713] The "advice generating means" is a function that generates specific cooking advice for the user based on the analysis results by the analyzing means.

[0714] The "emotion recognition means" is a function that analyzes emotional data such as the user's facial expressions, tone of voice, and gestures, and customizes advice based on the analysis results.

[0715] The "notification means" is a function that notifies the user's terminal of the generated cooking advice.

[0716] This system uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[0717] First, the device uses its camera to periodically take photos and videos of the food. The camera is attached to the food delivery robot and is positioned so that it can capture a wide range of the food delivery area. Specifically, the camera takes photos every five minutes, and the captured images and video data are temporarily stored in the device's internal storage. The data is then uploaded to the server in real time.

[0718] The server stores the received image and video data in a database. The stored data is then input into an AI model to evaluate the state and quality of the food. The AI ​​model uses deep learning, for example, to analyze the color, shape, and presentation of the image. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that it is "overcooked."

[0719] The device then uses its camera and microphone to record the user's facial expressions and tone of voice, and sends the data in real time to a server. The server then uses an emotion engine to analyze the user's emotional state. For example, if the server detects that the user is confused, it will use that information.

[0720] As a result, the server combines the results of the food's condition analysis with the user's emotional data to generate specific cooking advice. For example, if the food is judged to be "overcooked," it generates advice such as "reduce the cooking time by one minute." Furthermore, if the user is confused, the advice will be more polite, such as "reduce the cooking time by one minute. If you have any questions, please contact support."

[0721] Finally, the server sends the generated advice to the user's device, which then provides the advice to the user using notification functions and alerts.

[0722] Specific examples

[0723] For example, a restaurant chain is introducing a new robotic food delivery system. This system reduces the workload of chefs and consistently improves the quality of the food they serve. A camera attached to the robot periodically photographs the food and sends the image data to a server in real time. The server uses an AI model to analyze the images and generate specific cooking advice. It also analyzes the chef's facial expressions and tone of voice to provide customized advice based on their emotions.

[0724] Prompt Sentence Examples

[0725] "Generate suggestions for today's doneness based on photos of food taken. If the user is confused about what to cook, please provide polite suggestions taking their situation into consideration."

[0726] This allows the system to reduce the workload of the chef while maintaining consistently high food quality.

[0727] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0728] Step 1: Data Acquisition

[0729] The device takes photos and videos of the food every five minutes. For example, it uses a built-in camera to take photos of the entire serving area. The input is image data and video data captured by the camera. The output is that this photographed data is temporarily saved in the built-in storage. Specifically, the camera shutter is automatically released and data is collected at specified intervals.

[0730] Step 2: Send data

[0731] The device sends the temporarily saved image and video data to the server. The input is the captured image data obtained in step 1. The output is the image and video data uploaded to the server. The specific operation is that the data is transferred to the server via an HTTP POST request.

[0732] Step 3: Save Data

[0733] The server stores the received image data and video data in a database. The input is the data sent in step 2. The output is the image data and video data organized and stored in the database. The specific operation is to store and organize the data through a database management system (DBMS).

[0734] Step 4: Data analysis

[0735] The server inputs the stored image and video data into the AI ​​model to analyze the condition and quality of the food. The input is the image and video data stored in the database. The output is the analysis results of the food condition (e.g., color, shape, presentation). Specifically, it uses a deep learning model to identify the characteristics of the image and make an evaluation based on that.

[0736] Step 5: Acquire emotion data

[0737] It uses the device's camera and microphone to record the user's facial expressions and tone of voice. The input is the user's facial and voice data. The output is emotion data that is sent to the server. Specifically, it uses advanced facial recognition algorithms and tone of voice analysis software to recognize emotions and transfers the data to the server.

[0738] Step 6: Sentiment Data Analysis

[0739] The server uses the emotion engine to analyze the user's emotional state. The input is the emotion data obtained in step 5. The output is the analyzed user's emotional state. Specifically, the server infers the user's psychological state from facial expression data and tone of voice, and analyzes this information using the analysis engine.

[0740] Step 7: Advice Generation

[0741] The server integrates the cooking state analysis results with the user's emotional data to generate cooking advice. The input is the analysis results from steps 4 and 6. The output is specific cooking advice to provide to the user. The specific operation is to analyze the analysis results using a specific algorithm and generate the optimal cooking method. For example, the advice generated is "Please reduce the cooking time by 1 minute."

[0742] Step 8: Advice Notification

[0743] The server sends the generated advice to the user's device. The input is the cooking advice generated in step 7. The output is a notification displayed on the user's device. The specific operation is to send the advice to the device via a communication protocol such as an HTTP POST request, and the device notifies the user. For example, the notification may say, "Please reduce the cooking time by one minute. If you need help, please contact support."

[0744] (Application example 2)

[0745] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Existing restaurant systems using food delivery robots have the problem of insufficient food quality control. They also lack specific cooking advice to maintain consistent food quality. Furthermore, they do not provide services that take into account the user's emotional state, making it difficult to improve the user experience.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0748] In this invention, the server includes a camera attached to the food delivery robot, a data acquisition means for periodically acquiring image data captured by the camera, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results of the analysis means, a notification means for notifying the user of the cooking advice, an emotion analysis means for analyzing emotional data such as the user's facial expression and tone of voice and customizing the cooking advice, and an advice customization means for optimizing the cooking advice notified to the user's device to match the user's emotional state. This makes it possible to monitor the condition of the food in real time, detect quality defects early, and provide cooking advice tailored to the user's emotional state, improving the user experience.

[0749] A "serving robot" is an automated mechanical device designed to deliver food and drinks.

[0750] "Capture means" refers to the camera or sensor device used to capture images or videos.

[0751] The "data acquisition means" is a system for collecting and storing image data and video data obtained by the imaging means.

[0752] The "analysis means" refers to software or hardware for processing acquired image data or video data and extracting specific information.

[0753] The "advice generator" is a system that generates instructions to recommend specific actions or changes based on the results obtained by the analysis means.

[0754] The "notification means" is a communication means for conveying the generated advice or instructions to the user.

[0755] The "emotion analysis means" is a system for analyzing the user's facial expression, tone of voice, etc., to determine the user's emotional state.

[0756] The "advice customization means" is a system for optimizing the advice and instructions to be notified based on the user's emotional data obtained by the emotion analysis means.

[0757] A "user terminal" is a device used by a user to receive information, such as a smartphone or tablet.

[0758] This invention aims to improve quality control and user experience in the food delivery industry by using a camera attached to a food delivery robot, an AI analysis system, and an emotion analysis system. This system monitors the cooking status in real time and provides advice based on emotion analysis. Specific methods for hardware, software, data processing, and data calculation used to implement this invention are described below.

[0759] Hardware

[0760] Food delivery robot: an automated mechanical device for delivering food, equipped with cameras and other sensors.

[0761] Camera: A means of capturing images attached to the robot. For example, a webcam such as the Logitech C920.

[0762] Terminal: A device used to collect and process data and upload it to a server. An example is a single-board computer such as a Raspberry Pi.

[0763] Microphone: An input device for capturing the tone of a user's voice. For example, a high-quality microphone such as a Blue Yeti.

[0764] software

[0765] OpenCV: An open-source library for image capture and processing.

[0766] Requests: A Python library for communicating with the server.

[0767] TensorFlow / Keras: Machine learning frameworks that form the basis of the sentiment analysis model and cooking analysis AI model.

[0768] Custom Emotion Recognition Model: A model for recognizing a user's emotions from facial expressions and tone of voice.

[0769] Custom AI Model: An artificial intelligence model for analyzing the state of cooking and generating cooking advice.

[0770] Data processing and calculation

[0771] Data acquisition: The device periodically (e.g., every 5 minutes) takes pictures of the food using its camera, temporarily stores them in storage, and then uploads them to the server.

[0772] Data analysis: The server passes the uploaded image data to the analysis means to evaluate the state of the food (color, shape, presentation). The AI ​​model does this using specific algorithms.

[0773] Emotion data acquisition: The emotion analysis means uses a camera and microphone to collect the user's facial expressions and tone of voice, determine the user's emotional state, and send it to the server.

[0774] Advice generation: The server generates cooking advice based on the analysis results and emotion data. For example, it generates specific advice such as "We recommend shortening the cooking time by one minute."

[0775] Advice notification: The generated advice is sent to the user's device. If an emotion such as confusion is recognized, the advice is notified in a way that takes into consideration that emotion.

[0776] Specific examples

[0777] For example, a food delivery robot takes a photo of the food during delivery and sends it to a server, where an AI model analyzes it. Based on the analysis results, advice such as "reheating is necessary" is generated. If the user is confused, the emotion analysis system can detect this and provide additional support, such as "we'll explain in detail how to reheat it."

[0778] Prompt Sentence Examples

[0779] Below are some example prompts for the generative AI model and sentiment analysis system:

[0780] text

[0781] Provide a photo and generate optimal cooking advice based on emotional data from our emotion engine. We evaluate whether the food is properly cooked (color, shape, presentation, etc.) and take into account the user's feelings of confusion or dissatisfaction to customize the advice.

[0782] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0783] Step 1:

[0784] The device uses the camera to take pictures of the food at regular intervals (e.g., every 5 minutes). The captured images are temporarily stored in the device's storage. The input is image data acquired from the camera, and the output is an image file saved in the storage.

[0785] Step 2:

[0786] The device uploads the temporarily saved image data to the server by executing an HTTP POST request using the Requests library. The input is the image file in storage, and the output is a notification to the server that the upload is complete.

[0787] Step 3:

[0788] The server receives the uploaded image data and stores it in a database. The input is the image file sent from the device, and the output is the image data stored in the database. Specifically, data is received through a RESTful API.

[0789] Step 4:

[0790] The server passes the stored image data to an AI model, which is the analytical tool, and evaluates the condition of the food (color, shape, presentation). The input is the image data in the database, and the output is the analysis result (e.g., overcooked, poor color, etc.). Specifically, the analysis is performed using a machine learning framework such as TensorFlow or Keras.

[0791] Step 5:

[0792] The server generates cooking advice based on the analysis results. This advice is calculated using a specific algorithm. The input is the analysis results, and the output is specific cooking advice (e.g., "reduce the cooking time by 1 minute").

[0793] Step 6:

[0794] At the same time, the server uses emotion analysis means to obtain the user's emotion data. It collects and analyzes the user's facial expressions and tone of voice through a camera or microphone. The input is the facial expression and voice data obtained from the camera or microphone, and the output is the user's emotional state (e.g., confusion, joy). Specific operations use an emotion analysis model.

[0795] Step 7:

[0796] The server generates customized cooking advice that takes emotion data into consideration. The inputs are the analysis results and emotion data, and the output is emotion-sensitive cooking advice (e.g., "I'll explain it carefully, but please reduce the cooking time by 1 minute").

[0797] Step 8:

[0798] The server notifies the user's terminal of the generated cooking advice. The notification means is used to send emotion-conscious advice. The input is customized cooking advice, and the output is a notification message to the user's terminal.

[0799] Step 9:

[0800] The user receives the advice on the device and adjusts the cooking accordingly, specifically by referring to the message displayed on the device.

[0801] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0802] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0803] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0804] [Third embodiment]

[0805] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0806] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0807] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0808] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0809] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0810] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0811] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0812] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0813] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0814] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0815] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0816] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0817] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, and a notification means.

[0818] Overview of program processing

[0819] System Setup

[0820] server

[0821] 1. The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system.

[0822] 2. The server performs a test shoot to verify that the camera settings were applied correctly.

[0823] Specific examples

[0824] The server receives instructions from the user such as "Set the camera resolution to 1080p and the shooting interval to 5 minutes," and reflects these instructions on the camera. It then takes a test shot to check whether the settings have been correctly reflected.

[0825] Data capture and acquisition

[0826] Terminal

[0827] 1. The device takes images and videos of the food being served at the specified interval (e.g., every 5 minutes).

[0828] 2. The device temporarily stores the captured data.

[0829] Specific examples

[0830] The device takes a photo of the food every five minutes and temporarily stores the photo data.

[0831] server

[0832] 1. The server receives the photo and video data sent from the device.

[0833] 2. The server stores the received data in a database and checks the integrity of the data.

[0834] Specific examples

[0835] The server receives the photos of the food sent from the device and stores the photo data in a database. It checks that there are no missing data.

[0836] Data analysis and advice generation

[0837] server

[0838] 1. The server passes the stored data to the AI ​​model and begins analysis.

[0839] 2. The server uses an AI model to evaluate the condition and quality of the food (color, shape, presentation, etc.).

[0840] 3. The server receives the analysis results and detects defects and areas for improvement as necessary.

[0841] Specific examples

[0842] The server uses an AI model to analyze the photos and detect problems such as "the food is overcooked" or "the presentation is messy."

[0843] server

[0844] 1. The server generates specific cooking advice based on the analysis results.

[0845] 2. The server formats the cooking advice into a format that can be communicated to the user.

[0846] Specific examples

[0847] Based on the analysis result that the food is "overcooked," the server generates specific advice such as "reduce the cooking time by one minute" and notifies the user.

[0848] Advice notification and response

[0849] server

[0850] 1. The server sends the generated advice to the user's terminal.

[0851] 2. The server ensures that the notification is displayed properly.

[0852] User

[0853] 1. The user receives and confirms the advice notification from the server.

[0854] 2. The user adjusts the cooking method according to the received advice.

[0855] Specific examples

[0856] The user receives a notification to "reduce the bake time by 1 minute" and reduces the bake time accordingly the next time they cook.

[0857] Order assistance function

[0858] server

[0859] 1. If the server detects a shortage of materials during AI analysis, it will activate the ordering assistance function.

[0860] 2. The server generates an order list of materials and sends it to the user.

[0861] User

[0862] 1. The user receives the order list, checks the required materials, and places an order.

[0863] Specific examples

[0864] The server detects that there is a shortage of fresh lemons, generates an order list, and sends it to the user, who then checks it and orders only the amount they need.

[0865] summary

[0866] This invention is a system that uses a food delivery robot to automatically check product quality and provide specific cooking advice based on the analysis results of AI. This will improve the efficiency of quality control in restaurants and reduce the burden on employees. In addition, if a shortage of ingredients is detected, an ordering assistance function will be provided, further improving the efficiency of ingredient management.

[0867] The processing flow will be explained below.

[0868] Step 1:

[0869] server

[0870] The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system. It also performs test shooting to confirm that the settings are correctly reflected.

[0871] Step 2:

[0872] Terminal

[0873] The device takes pictures and videos of the food being served at a specified interval (e.g., every 5 minutes), and temporarily stores the captured data in its internal storage.

[0874] Step 3:

[0875] Terminal

[0876] The device transmits the photos and videos stored in its internal storage to the server in real time, and receives feedback to confirm that the data transmission was successful.

[0877] Step 4:

[0878] server

[0879] The server receives the photo and video data sent from the device and stores it in a database. When saving, it checks the integrity of the data to ensure there are no missing or error data.

[0880] Step 5:

[0881] server

[0882] The server then passes the stored data to the AI ​​model, which then evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[0883] Step 6:

[0884] server

[0885] The server receives the analysis results from the AI ​​model and uses them to detect defects and areas for improvement, such as identifying problems like "overcooked" or "messy presentation."

[0886] Step 7:

[0887] server

[0888] The server generates specific cooking advice based on the analysis results, and the advice is formatted in a way that is easy for the user to understand.

[0889] Step 8:

[0890] server

[0891] The server sends the generated cooking advice to the user's device and monitors the delivery status to ensure that the notification is displayed properly.

[0892] Step 9:

[0893] User

[0894] The user receives the advice notification from the server, checks the contents, and, if necessary, modifies the cooking method based on the advice.

[0895] Step 10:

[0896] server

[0897] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function, generates an appropriate ordering list for materials, and sends it to the user.

[0898] Step 11:

[0899] User

[0900] The user receives the order list sent from the server, checks the required materials, and then takes steps to order the materials.

[0901] summary

[0902] Through these steps, the present invention utilizes a food delivery robot to monitor product availability in real time, provide specific cooking advice based on AI analysis results, and detect ingredient shortages and assist with ordering, thereby improving quality control and operational efficiency.

[0903] Example 1

[0904] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0905] Traditionally, quality control processes in restaurants have often been performed manually, resulting in issues such as wasted human resources and inaccurate quality assessment. Furthermore, there was a lack of systems for early detection of defects and providing specific advice for improvement, making it difficult to maintain consistent quality. Furthermore, there was also a lack of systems for timely detection of ingredient shortages and appropriate ordering, resulting in inefficient ingredient management.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0907] In this invention, the server includes a setting application means that receives settings for the image capture means from the user and applies them to the camera and analysis system, a captured data storage means that temporarily stores data, a database storage means that stores image data sent from the terminal in a database and checks consistency, and an ingredient detection means and ordering assistance means that detect ingredient shortages during analysis and provide an ordering assistance function. This automates and streamlines quality control in restaurants, enables the provision of specific cooking advice, and enables timely detection of ingredient shortages and effective ingredient management.

[0908] The "photography means" is a device that is attached to the food delivery robot and periodically takes pictures and videos.

[0909] The "data acquisition means" is a mechanism for collecting and storing image data acquired by the imaging means.

[0910] An "analysis means" is a system or algorithm for analyzing acquired image data and evaluating its content.

[0911] The "advice generating means" is a function that generates specific cooking advice based on the analysis results of the analyzing means.

[0912] The "notification means" is a system for notifying and displaying the generated cooking advice on the user's terminal.

[0913] The "setting application means" is a mechanism that reflects the camera setting parameters received from the user in the food delivery robot and analysis system.

[0914] The "photographing data storage means" is a storage system for temporarily storing image data obtained by the photographing means.

[0915] The "database storage means" is a mechanism that stores image data sent from the terminal in a database and checks its consistency.

[0916] "Material detection means" means a system or algorithm for detecting material shortages during analysis.

[0917] The "order assist means" is a function that creates an order list based on the shortage materials detected by the material detection means and prompts the user to place an order.

[0918] The present invention is a system that analyzes image data captured by a food delivery robot, provides cooking advice based on the results, and also manages ingredients. Specifically, this system is comprised of an image capture means, data acquisition means, analysis means, advice generation means, notification means, setting application means, captured data storage means, database storage means, ingredient detection means, and ordering assistance means attached to the food delivery robot.

[0919] server

[0920] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval) and applies them to the camera and AI analysis system. This ensures that the shooting method operates with the correct settings. The server then takes a test shot and checks the results to ensure the settings were applied properly.

[0921] Terminal

[0922] The device periodically captures images and videos of the food being served according to the specified interval. For example, the device takes a photo of the food every five minutes and temporarily stores the photo data in local storage. The captured data is periodically sent to the server.

[0923] Data storage and analysis

[0924] The server receives the photo and video data sent from the device and stores it in a database. At this time, the data is checked for consistency and to ensure there are no missing pieces. The saved data is then passed to the AI ​​model, which begins analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and returns the results to the server. Specifically, it may give an evaluation such as "the food is overcooked" or "the presentation is messy."

[0925] Cooking advice generation and notification

[0926] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. For example, the advice might be, "Reduce the cooking time by one minute." The generated cooking advice is then sent to the user's device. The user receives the notification, checks the content, and adjusts the cooking method accordingly for the next time.

[0927] Material detection and ordering assistance

[0928] Furthermore, if a shortage of ingredients is detected during AI analysis, the server will activate the ordering assistance function. For example, if it detects that there is a shortage of fresh lemons, a list of ingredients to order will be generated and sent to the user. The user can then check the list and order the necessary ingredients.

[0929] As described above, this system automates and streamlines quality control in restaurants. It also provides specific cooking advice, which is expected to improve cooking quality. Furthermore, as soon as a shortage of ingredients is detected, appropriate action can be taken immediately, which also improves the efficiency of ingredient management.

[0930] Prompt Sentence Examples

[0931] "We would like to build a system that can analyze photos of food taken by a food delivery robot and evaluate their quality. Based on that, it can provide cooking advice based on the needs of the customer. The system should evaluate factors such as cooking time, presentation, and color, and generate specific advice."

[0932] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0933] Step 1: Apply camera settings

[0934] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval). The input is the camera parameters set by the user, and the output is the state in which the settings have been applied to the camera. The server sends the settings via the camera's API, and the settings are changed. Specifically, the server receives a request for camera settings and sends an HTTP request to the corresponding endpoint to reflect the settings.

[0935] Step 2: Conduct a test shoot

[0936] The server takes a test shot to confirm that the camera settings have been applied correctly and checks the results. The input is the state with the camera settings applied, and the output is the test image taken. The server sends a command to the camera to take a test shot, displays the acquired image in real time, and checks whether the settings have been applied.

[0937] Step 3: Conduct regular photo shoots

[0938] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes). The input is the interval based on the internal clock, and the output is the captured image and video data. Specifically, the device sets a timer, starts the camera to take a photo each time the timer expires, and saves the captured data in local storage.

[0939] Step 4: Sending and Receiving Data

[0940] The device temporarily stores the captured data and then sends it to the server. The server receives the data sent from the device, stores it in a database, and checks its consistency. The input is the image data generated by the device, and the output is stored in the database within the server. Specifically, the device sends data to the server using an HTTP POST request, and the server receives the data and inserts it into the database.

[0941] Step 5: Analysis by AI model

[0942] The server passes the stored data to the AI ​​model and begins analysis. The input is the image data stored in the database, and the output is the analysis results returned by the AI ​​model. The server sends the data to the AI ​​model's API endpoint, and the model evaluates the condition and quality of the food (color, shape, presentation, etc.), and receives the results.

[0943] Step 6: Generate cooking advice

[0944] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. The input is the analysis results of the AI ​​model, and the output is the cooking advice to be provided to the user. Specifically, the server uses a template to interpret the analysis results and generates specific advice such as "Please reduce the cooking time by one minute."

[0945] Step 7: Advice Notification

[0946] The server sends the generated advice to the user's device. The user checks that the notification is displayed properly and modifies the cooking method according to the received advice. The input is the generated cooking advice, and the output is the notification displayed on the user's device. Specifically, the server sends the advice via email or app notification, and the user checks the notification.

[0947] Step 8: Assist with ordering materials

[0948] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function. The input is the analysis results of the AI ​​model and material inventory data, and the output is an ordering list provided to the user. Specifically, the server generates an ordering list based on the material detection results and sends it to the user. The user receives the ordering list, checks the required materials, and places an order.

[0949] Through these steps, the system can automatically check product quality, provide specific cooking advice, and streamline ingredient management.

[0950] (Application example 1)

[0951] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0952] In restaurants using conventional food delivery robots, product quality management and cooking improvements were sometimes not properly carried out. It was also difficult to respond quickly to ingredient shortages or provide specific advice to staff. As a result, problems arose, such as a decrease in customer satisfaction and a loss of store operational efficiency. The present invention aims to solve these problems and improve the efficiency of cooking quality management, the provision of improvement advice, and ingredient management in restaurants.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0954] In this invention, the server includes a photographing means attached to the food delivery robot, a data acquisition means for periodically acquiring photographed image data, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results by the analysis means, a notification means for notifying the user of the cooking advice, an ordering assistance means for detecting ingredient shortages and generating an ordering list, a communication means for communicating with the server to send and receive food images, and an AI analysis means for performing image analysis using an AI model. This allows for efficient provision of advice to improve cooking quality and ingredient management, thereby improving restaurant operation efficiency and customer satisfaction.

[0955] The "photography means" is a device that is attached to the food delivery robot and periodically acquires image data.

[0956] The "data acquisition means" is a device or program that receives image data captured by the image capture means and stores or transmits the data.

[0957] The "analysis means" is a program or device for evaluating the condition and quality of a product using the acquired image data.

[0958] The "advice generating means" is a program or device that generates specific cooking advice based on the analysis results from the analyzing means.

[0959] The "notification means" refers to a device or program for transmitting the generated cooking advice to the user.

[0960] The "order assistance means" is a program or device for detecting shortages of materials and generating an order list for the necessary materials.

[0961] The "communication means" refers to devices and programs for transmitting and receiving cooking images to and from the server via communication.

[0962] "AI analysis means" refers to a program or device that uses an AI model to analyze image data and evaluate the condition and quality of a product.

[0963] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, a notification means, an order assistance means, a communication means, and an AI analysis means.

[0964] 1. Hardware and Software Used

[0965] Hardware

[0966] Smartphones (e.g. iPhone, Android devices)

[0967] Food delivery robot (equipped with a camera)

[0968] Server (data processing, analysis)

[0969] software

[0970] AI analysis models (e.g., TensorFlow, PyTorch)

[0971] Database (e.g. MySQL, PostgreSQL)

[0972] Communication protocol (e.g. HTTP, WebSocket)

[0973] 2. Specific Embodiments of the System

[0974] (1) Data capture and acquisition

[0975] The delivery robot periodically takes photos of the food. The image data is automatically acquired at specified intervals (e.g., every 5 minutes). The acquired image data is temporarily stored and then sent to the server.

[0976] Example: "Take a photo of the food every 5 minutes in 1080p resolution and send it to the server."

[0977] (2) Data storage and analysis

[0978] The server receives the transmitted image data and stores it in a database. The stored data is then analyzed by an AI analysis means. The analysis means uses an AI model (e.g., TensorFlow, PyTorch) to evaluate the condition and quality of the product based on the image data.

[0979] Example: "Analyze the received photo data using an AI model and evaluate the quality of the food. If there is a problem, notify us. Save it on the server."

[0980] (3) Advice generation

[0981] Based on the analysis results, the advice generating means generates specific cooking advice, which is then sent to the smartphone via the communication means.

[0982] Example: "Based on the analysis results, generate specific cooking advice. For example, 'Reduce the cooking time by 1 minute.'"

[0983] (4) Order assistance function

[0984] Furthermore, if the server detects a shortage of ingredients during the AI ​​analysis process, it will use the ordering assistance tool to generate a list of the ingredients that are in short supply, which will also be sent to the smartphone.

[0985] Example: "Detect shortages of materials and generate an order list to notify you."

[0986] 3. Specific Examples

[0987] For example, a smartphone app might send the following prompt to the user:

[0988] "Take a photo of your food every 5 minutes in 1080p resolution and send it to the server."

[0989] "The received photo data is analyzed using an AI model to evaluate the quality of the food. If there is a problem, we will notify you. Please save it on our server."

[0990] "Based on the analysis results, please generate specific cooking advice, such as 'Reduce the cooking time by one minute.'"

[0991] "Detect material shortages and generate an order list to notify you."

[0992] This allows the food delivery robot to periodically take photos of the food and send them to the server. The server then analyzes the image data, evaluates the cooking quality, and generates advice. It can also detect ingredient shortages and generate an ordering list for the necessary ingredients, improving restaurant operation efficiency and customer satisfaction.

[0993] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0994] Step 1:

[0995] Data capture

[0996] The terminal (serving robot) takes pictures of food at a specified interval (e.g., every 5 minutes). The input is the interval and camera settings (resolution, focal length, etc.). The output is the captured food image data. Specifically, the serving robot automatically starts the camera and presses the capture button.

[0997] Step 2:

[0998] Temporary storage and transmission of data

[0999] The device temporarily stores the captured food image data and then sends it to the server. The input is the food image data acquired in step 1. The output is the food image data sent to the server. Specifically, the device stores the image data in local storage and then uploads the data to the server via the network.

[1000] Step 3:

[1001] Receiving and storing data

[1002] The server receives the food image data sent from the terminal. The input is the food image data sent from the terminal. The output is the food image data stored in the database. Specifically, the server receives the HTTP request and stores the image data in the database.

[1003] Step 4:

[1004] Data Integrity Check

[1005] The server checks the consistency of the stored image data. The input is the food image data stored in the database. The output is food image data whose consistency has been confirmed. Specifically, the server checks the size and format of the data to see if there are any inconsistencies.

[1006] Step 5:

[1007] Image data analysis

[1008] The server analyzes image data using an AI model. The input is food image data whose consistency has been confirmed. The output is the analysis results that evaluate the state and quality of the food. Specifically, the AI ​​analysis means uses TensorFlow and PyTorch to analyze the image and evaluate the color, shape, and presentation.

[1009] Step 6:

[1010] Advice Generation

[1011] The server generates specific cooking advice based on the analysis results. The input is the analysis results obtained in step 5. The output is the generated cooking advice. Specifically, specific points for improvement, such as "Please reduce the baking time by one minute," are generated in text format.

[1012] Step 7:

[1013] Advice Notice

[1014] The server notifies the user's smartphone of the generated cooking advice. The input is the cooking advice generated in step 6. The output is the cooking advice displayed on the smartphone. Specifically, the server sends the advice using a notification protocol, and a pop-up notification is displayed on the smartphone.

[1015] Step 8:

[1016] Activating the ordering assistance function

[1017] If the server detects a shortage of materials during the AI ​​analysis process, it activates the ordering assistance means. The input is the analysis results from step 5 and the materials database. The output is a materials ordering list. Specifically, the server detects a material shortage and automatically generates an ordering list.

[1018] Step 9:

[1019] Order list notification

[1020] The server notifies the user's smartphone of the generated order list. The input is the order list generated in step 8. The output is the order list displayed on the smartphone. Specifically, the server sends the order list using a notification protocol, and it is displayed on the smartphone.

[1021] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1022] This invention relates to a system that uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[1023] System Configuration

[1024] 1. Filming Method

[1025] Terminal

[1026] The camera is attached to the food delivery robot and positioned so that it can capture a wide range of the food delivery area. The camera takes photos and videos at specified intervals and generates the data.

[1027] 2. Data Acquisition Method

[1028] Terminal

[1029] Image and video data captured by the camera is temporarily stored in the device's internal storage, and then uploaded to a server in real time.

[1030] 3. Analysis method

[1031] server

[1032] The server passes the uploaded image data to an AI model for analysis, which evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[1033] 4. Advice Generation Methods

[1034] server

[1035] Based on the analysis results, the server generates specific cooking advice using specific algorithms and evaluation criteria to provide optimal suggestions for the user.

[1036] 5. Emotion Engine

[1037] server

[1038] The emotion engine analyzes sensor information from cameras, microphones, etc. to obtain user emotional data. It recognizes emotions from facial expressions, tone of voice, gestures, etc. and sends the data to the server.

[1039] 6. Means of notification

[1040] server

[1041] Based on the analysis results and emotional data, the server generates cooking advice in the most appropriate format and wording and notifies the user's device.

[1042] Overview of program processing

[1043] The program of this system is designed so that each means cooperates to acquire data, analyze it, generate advice, and notify it. The process flow is explained below with a concrete example.

[1044] 1. Data Acquisition

[1045] The device takes a photo of the food every five minutes and sends the image data in real time to a server, which stores the data in a database.

[1046] 2. Data Analysis

[1047] The server inputs the stored image data into the AI ​​model and analyzes the condition of the food. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that the food is overcooked.

[1048] 3. Emotion Data Acquisition

[1049] The emotion engine uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. For example, if it determines that the user is confused, it sends that data to the server.

[1050] 4. Advice Generation

[1051] The server combines the AI ​​analysis results with the emotion data from the emotion engine to generate cooking advice, such as recommending that users reduce the cooking time by one minute. Confused users are given advice in a polite and calm tone.

[1052] 5. Notification

[1053] The server sends the generated advice to the user's device. For example, a notification may be sent saying, "Please reduce the cooking time by one minute. If you need help, please contact support."

[1054] Application of specific examples

[1055] A restaurant chain is introducing a new food delivery robot system to reduce the workload of its chefs while consistently improving the quality of the food they serve. With this system, as the robot delivers food, it takes a photo of the food with a camera and uses AI to analyze it. Based on the analysis results, it generates specific advice on cooking times and methods, and even provides customized advice based on the chef's emotions. This allows the chef to work efficiently while maintaining a consistently high level of food quality.

[1056] This invention is a system that improves quality control and operational efficiency in restaurants by combining three elements: a food delivery robot, AI analysis, and an emotion engine. It is particularly useful for companies with multiple stores or those expanding overseas.

[1057] The processing flow will be explained below.

[1058] Step 1:

[1059] User

[1060] To perform the initial setup of the system, the user inputs the camera resolution, shooting interval, emotion engine setting parameters, etc. into the server. The user confirms that the initial setup is complete.

[1061] Step 2:

[1062] server

[1063] The server applies the setting parameters received from the user to the camera and AI analysis system, and then conducts a test shoot to confirm that the settings have been correctly applied.

[1064] Step 3:

[1065] Terminal

[1066] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes) and temporarily stores them in its internal storage.

[1067] Step 4:

[1068] Terminal

[1069] The device uploads stored image and video data to a server in real time, and receives feedback to confirm successful data transmission.

[1070] Step 5:

[1071] server

[1072] The server receives the image data sent from the device and stores it in a database. When saving, it checks the integrity of the data and makes sure there are no missing parts.

[1073] Step 6:

[1074] server

[1075] The server passes the stored image data to the AI ​​model to begin analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and outputs the results.

[1076] Step 7:

[1077] server

[1078] The server receives the results of the AI ​​analysis and records the condition of the food determined from the image data (e.g., overcooked, messy presentation, etc.) as the analysis result.

[1079] Step 8:

[1080] Terminal

[1081] The emotion engine installed in the device uses a camera and microphone to analyze the user's facial expressions and tone of voice in real time and generate emotion data.

[1082] Step 9:

[1083] Terminal

[1084] The device transmits the generated emotion data to the server, which conveys the user's current emotional state (e.g., confusion, joy, anger, etc.) to the server.

[1085] Step 10:

[1086] server

[1087] The server combines the AI ​​analysis results with emotional data to generate appropriate cooking advice. For example, if the user is confused, the server generates advice in a polite and calm tone. If the analysis results indicate that the cooking time should be shortened by one minute, this advice will also be included.

[1088] Step 11:

[1089] server

[1090] The generated cooking advice is sent to the user's device, and the wording and format of the notification are adjusted based on the emotion data.

[1091] Step 12:

[1092] User

[1093] The user receives the advice notification from the server, checks the content, and, if necessary, modifies the cooking method based on the advice.

[1094] Application of specific examples

[1095] For example, if a restaurant chain were to implement this system, while testing a new menu item, the delivery robot would take photos of the food every five minutes and send them to the server. The server would then analyze these photos using an AI model and determine whether the food was overcooked. Furthermore, an emotion engine would analyze the chef's facial expressions and detect whether the chef was confused. Based on this data, the server would generate a gentle message to the user, saying, "Please reduce the cooking time by one minute. If you have any questions, please contact support." The chef would then check the notification and shorten the cooking time the next time, thereby improving the quality of the food.

[1096] In this way, this system combines food delivery robots, AI analysis, and an emotion engine to significantly improve quality control and operational efficiency in restaurants.

[1097] Example 2

[1098] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1099] In restaurants, consistently improving the quality of the food served while reducing the workload of chefs is an important issue. Furthermore, when chefs or employees are stressed or overwhelmed, support is needed to reduce their workload and enable them to work efficiently. Current technology lacks a system that satisfies these needs, so a new system is needed that simultaneously manages food quality and provides psychological support to chefs.

[1100] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1101] In this invention, the server includes a photographing means, a data acquisition means, an analysis means, an advice generation means, an emotion recognition means, and a notification means, which makes it possible to provide customized advice according to the emotional state of the chef or staff, in addition to evaluating the quality of the food and providing cooking advice.

[1102] The "photography means" is a device that is attached to the food delivery robot and periodically takes photos and videos of the food.

[1103] The "data acquisition means" is a function that records image data and video data acquired by the image capture means and transmits them to a server as necessary.

[1104] The "analysis means" refers to an analysis function that includes an AI model that evaluates the condition and quality of food based on image and video data uploaded to the server.

[1105] The "advice generating means" is a function that generates specific cooking advice for the user based on the analysis results by the analyzing means.

[1106] The "emotion recognition means" is a function that analyzes emotional data such as the user's facial expressions, tone of voice, and gestures, and customizes advice based on the analysis results.

[1107] The "notification means" is a function that notifies the user's terminal of the generated cooking advice.

[1108] This system uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[1109] First, the device uses its camera to periodically take photos and videos of the food. The camera is attached to the food delivery robot and is positioned so that it can capture a wide range of the food delivery area. Specifically, the camera takes photos every five minutes, and the captured images and video data are temporarily stored in the device's internal storage. The data is then uploaded to the server in real time.

[1110] The server stores the received image and video data in a database. The stored data is then input into an AI model to evaluate the state and quality of the food. The AI ​​model uses deep learning, for example, to analyze the color, shape, and presentation of the image. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that it is "overcooked."

[1111] The device then uses its camera and microphone to record the user's facial expressions and tone of voice, and sends the data in real time to a server. The server then uses an emotion engine to analyze the user's emotional state. For example, if the server detects that the user is confused, it will use that information.

[1112] As a result, the server combines the results of the food's condition analysis with the user's emotional data to generate specific cooking advice. For example, if the food is judged to be "overcooked," it generates advice such as "reduce the cooking time by one minute." Furthermore, if the user is confused, the advice will be more polite, such as "reduce the cooking time by one minute. If you have any questions, please contact support."

[1113] Finally, the server sends the generated advice to the user's device, which then provides the advice to the user using notification functions and alerts.

[1114] Specific examples

[1115] For example, a restaurant chain is introducing a new robotic food delivery system. This system reduces the workload of chefs and consistently improves the quality of the food they serve. A camera attached to the robot periodically photographs the food and sends the image data to a server in real time. The server uses an AI model to analyze the images and generate specific cooking advice. It also analyzes the chef's facial expressions and tone of voice to provide customized advice based on their emotions.

[1116] Prompt Sentence Examples

[1117] "Generate suggestions for today's doneness based on photos of food taken. If the user is confused about what to cook, please provide polite suggestions taking their situation into consideration."

[1118] This allows the system to reduce the workload of the chef while maintaining consistently high food quality.

[1119] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1120] Step 1: Data Acquisition

[1121] The device takes photos and videos of the food every five minutes. For example, it uses a built-in camera to take photos of the entire serving area. The input is image data and video data captured by the camera. The output is that this photographed data is temporarily saved in the built-in storage. Specifically, the camera shutter is automatically released and data is collected at specified intervals.

[1122] Step 2: Send data

[1123] The device sends the temporarily saved image and video data to the server. The input is the captured image data obtained in step 1. The output is the image and video data uploaded to the server. The specific operation is that the data is transferred to the server via an HTTP POST request.

[1124] Step 3: Save Data

[1125] The server stores the received image data and video data in a database. The input is the data sent in step 2. The output is the image data and video data organized and stored in the database. The specific operation is to store and organize the data through a database management system (DBMS).

[1126] Step 4: Data analysis

[1127] The server inputs the stored image and video data into the AI ​​model to analyze the condition and quality of the food. The input is the image and video data stored in the database. The output is the analysis results of the food condition (e.g., color, shape, presentation). Specifically, it uses a deep learning model to identify the characteristics of the image and make an evaluation based on that.

[1128] Step 5: Acquire emotion data

[1129] It uses the device's camera and microphone to record the user's facial expressions and tone of voice. The input is the user's facial and voice data. The output is emotion data that is sent to the server. Specifically, it uses advanced facial recognition algorithms and tone of voice analysis software to recognize emotions and transfers the data to the server.

[1130] Step 6: Sentiment Data Analysis

[1131] The server uses the emotion engine to analyze the user's emotional state. The input is the emotion data obtained in step 5. The output is the analyzed user's emotional state. Specifically, the server infers the user's psychological state from facial expression data and tone of voice, and analyzes this information using the analysis engine.

[1132] Step 7: Advice Generation

[1133] The server integrates the cooking state analysis results with the user's emotional data to generate cooking advice. The input is the analysis results from steps 4 and 6. The output is specific cooking advice to provide to the user. The specific operation is to analyze the analysis results using a specific algorithm and generate the optimal cooking method. For example, the advice generated is "Please reduce the cooking time by 1 minute."

[1134] Step 8: Advice Notification

[1135] The server sends the generated advice to the user's device. The input is the cooking advice generated in step 7. The output is a notification displayed on the user's device. The specific operation is to send the advice to the device via a communication protocol such as an HTTP POST request, and the device notifies the user. For example, the notification may say, "Please reduce the cooking time by one minute. If you need help, please contact support."

[1136] (Application example 2)

[1137] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1138] Existing restaurant systems using food delivery robots have the problem of insufficient food quality control. They also lack specific cooking advice to maintain consistent food quality. Furthermore, they do not provide services that take into account the user's emotional state, making it difficult to improve the user experience.

[1139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1140] In this invention, the server includes a camera attached to the food delivery robot, a data acquisition means for periodically acquiring image data captured by the camera, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results of the analysis means, a notification means for notifying the user of the cooking advice, an emotion analysis means for analyzing emotional data such as the user's facial expression and tone of voice and customizing the cooking advice, and an advice customization means for optimizing the cooking advice notified to the user's device to match the user's emotional state. This makes it possible to monitor the condition of the food in real time, detect quality defects early, and provide cooking advice tailored to the user's emotional state, improving the user experience.

[1141] A "serving robot" is an automated mechanical device designed to deliver food and drinks.

[1142] "Capture means" refers to the camera or sensor device used to capture images or videos.

[1143] The "data acquisition means" is a system for collecting and storing image data and video data obtained by the imaging means.

[1144] The "analysis means" refers to software or hardware for processing acquired image data or video data and extracting specific information.

[1145] The "advice generator" is a system that generates instructions to recommend specific actions or changes based on the results obtained by the analysis means.

[1146] The "notification means" is a communication means for conveying the generated advice or instructions to the user.

[1147] The "emotion analysis means" is a system for analyzing the user's facial expression, tone of voice, etc., to determine the user's emotional state.

[1148] The "advice customization means" is a system for optimizing the advice and instructions to be notified based on the user's emotional data obtained by the emotion analysis means.

[1149] A "user terminal" is a device used by a user to receive information, such as a smartphone or tablet.

[1150] This invention aims to improve quality control and user experience in the food delivery industry by using a camera attached to a food delivery robot, an AI analysis system, and an emotion analysis system. This system monitors the cooking status in real time and provides advice based on emotion analysis. Specific methods for hardware, software, data processing, and data calculation used to implement this invention are described below.

[1151] Hardware

[1152] Food delivery robot: an automated mechanical device for delivering food, equipped with cameras and other sensors.

[1153] Camera: A means of capturing images attached to the robot. For example, a webcam such as the Logitech C920.

[1154] Terminal: A device used to collect and process data and upload it to a server. An example is a single-board computer such as a Raspberry Pi.

[1155] Microphone: An input device for capturing the tone of a user's voice. For example, a high-quality microphone such as a Blue Yeti.

[1156] software

[1157] OpenCV: An open-source library for image capture and processing.

[1158] Requests: A Python library for communicating with the server.

[1159] TensorFlow / Keras: Machine learning frameworks that form the basis of the sentiment analysis model and cooking analysis AI model.

[1160] Custom Emotion Recognition Model: A model for recognizing a user's emotions from facial expressions and tone of voice.

[1161] Custom AI Model: An artificial intelligence model for analyzing the state of cooking and generating cooking advice.

[1162] Data processing and calculation

[1163] Data acquisition: The device periodically (e.g., every 5 minutes) takes pictures of the food using its camera, temporarily stores them in storage, and then uploads them to the server.

[1164] Data analysis: The server passes the uploaded image data to the analysis means to evaluate the state of the food (color, shape, presentation). The AI ​​model does this using specific algorithms.

[1165] Emotion data acquisition: The emotion analysis means uses a camera and microphone to collect the user's facial expressions and tone of voice, determine the user's emotional state, and send it to the server.

[1166] Advice generation: The server generates cooking advice based on the analysis results and emotion data. For example, it generates specific advice such as "We recommend shortening the cooking time by one minute."

[1167] Advice notification: The generated advice is sent to the user's device. If an emotion such as confusion is recognized, the advice is notified in a way that takes into consideration that emotion.

[1168] Specific examples

[1169] For example, a food delivery robot takes a photo of the food during delivery and sends it to a server, where an AI model analyzes it. Based on the analysis results, advice such as "reheating is necessary" is generated. If the user is confused, the emotion analysis system can detect this and provide additional support, such as "we'll explain in detail how to reheat it."

[1170] Prompt Sentence Examples

[1171] Below are some example prompts for the generative AI model and sentiment analysis system:

[1172] text

[1173] Provide a photo and generate optimal cooking advice based on emotional data from our emotion engine. We evaluate whether the food is properly cooked (color, shape, presentation, etc.) and take into account the user's feelings of confusion or dissatisfaction to customize the advice.

[1174] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1175] Step 1:

[1176] The device uses the camera to take pictures of the food at regular intervals (e.g., every 5 minutes). The captured images are temporarily stored in the device's storage. The input is image data acquired from the camera, and the output is an image file saved in the storage.

[1177] Step 2:

[1178] The device uploads the temporarily saved image data to the server by executing an HTTP POST request using the Requests library. The input is the image file in storage, and the output is a notification to the server that the upload is complete.

[1179] Step 3:

[1180] The server receives the uploaded image data and stores it in a database. The input is the image file sent from the device, and the output is the image data stored in the database. Specifically, data is received through a RESTful API.

[1181] Step 4:

[1182] The server passes the stored image data to an AI model, which is the analytical tool, and evaluates the condition of the food (color, shape, presentation). The input is the image data in the database, and the output is the analysis result (e.g., overcooked, poor color, etc.). Specifically, the analysis is performed using a machine learning framework such as TensorFlow or Keras.

[1183] Step 5:

[1184] The server generates cooking advice based on the analysis results. This advice is calculated using a specific algorithm. The input is the analysis results, and the output is specific cooking advice (e.g., "reduce the cooking time by 1 minute").

[1185] Step 6:

[1186] At the same time, the server uses emotion analysis means to obtain the user's emotion data. It collects and analyzes the user's facial expressions and tone of voice through a camera or microphone. The input is the facial expression and voice data obtained from the camera or microphone, and the output is the user's emotional state (e.g., confusion, joy). Specific operations use an emotion analysis model.

[1187] Step 7:

[1188] The server generates customized cooking advice that takes emotion data into consideration. The inputs are the analysis results and emotion data, and the output is emotion-sensitive cooking advice (e.g., "I'll explain it carefully, but please reduce the cooking time by 1 minute").

[1189] Step 8:

[1190] The server notifies the user's terminal of the generated cooking advice. The notification means is used to send emotion-conscious advice. The input is customized cooking advice, and the output is a notification message to the user's terminal.

[1191] Step 9:

[1192] The user receives the advice on the device and adjusts the cooking accordingly, specifically by referring to the message displayed on the device.

[1193] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1194] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1195] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1196] [Fourth embodiment]

[1197] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1198] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1199] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1200] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1201] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1202] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1203] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1204] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1205] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1206] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1207] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1208] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1209] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1210] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, and a notification means.

[1211] Overview of program processing

[1212] System Setup

[1213] server

[1214] 1. The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system.

[1215] 2. The server performs a test shoot to verify that the camera settings were applied correctly.

[1216] Specific examples

[1217] The server receives instructions from the user such as "Set the camera resolution to 1080p and the shooting interval to 5 minutes," and reflects these instructions on the camera. It then takes a test shot to check whether the settings have been correctly reflected.

[1218] Data capture and acquisition

[1219] Terminal

[1220] 1. The device takes images and videos of the food being served at the specified interval (e.g., every 5 minutes).

[1221] 2. The device temporarily stores the captured data.

[1222] Specific examples

[1223] The device takes a photo of the food every five minutes and temporarily stores the photo data.

[1224] server

[1225] 1. The server receives the photo and video data sent from the device.

[1226] 2. The server stores the received data in a database and checks the integrity of the data.

[1227] Specific examples

[1228] The server receives the photos of the food sent from the device and stores the photo data in a database. It checks that there are no missing data.

[1229] Data analysis and advice generation

[1230] server

[1231] 1. The server passes the stored data to the AI ​​model and begins analysis.

[1232] 2. The server uses an AI model to evaluate the condition and quality of the food (color, shape, presentation, etc.).

[1233] 3. The server receives the analysis results and detects defects and areas for improvement as necessary.

[1234] Specific examples

[1235] The server uses an AI model to analyze the photos and detect problems such as "the food is overcooked" or "the presentation is messy."

[1236] server

[1237] 1. The server generates specific cooking advice based on the analysis results.

[1238] 2. The server formats the cooking advice into a format that can be communicated to the user.

[1239] Specific examples

[1240] Based on the analysis result that the food is "overcooked," the server generates specific advice such as "reduce the cooking time by one minute" and notifies the user.

[1241] Advice notification and response

[1242] server

[1243] 1. The server sends the generated advice to the user's terminal.

[1244] 2. The server ensures that the notification is displayed properly.

[1245] User

[1246] 1. The user receives and confirms the advice notification from the server.

[1247] 2. The user adjusts the cooking method according to the received advice.

[1248] Specific examples

[1249] The user receives a notification to "reduce the bake time by 1 minute" and reduces the bake time accordingly the next time they cook.

[1250] Order assistance function

[1251] server

[1252] 1. If the server detects a shortage of materials during AI analysis, it will activate the ordering assistance function.

[1253] 2. The server generates an order list of materials and sends it to the user.

[1254] User

[1255] 1. The user receives the order list, checks the required materials, and places an order.

[1256] Specific examples

[1257] The server detects that there is a shortage of fresh lemons, generates an order list, and sends it to the user, who then checks it and orders only the amount they need.

[1258] summary

[1259] This invention is a system that uses a food delivery robot to automatically check product quality and provide specific cooking advice based on the analysis results of AI. This will improve the efficiency of quality control in restaurants and reduce the burden on employees. In addition, if a shortage of ingredients is detected, an ordering assistance function will be provided, further improving the efficiency of ingredient management.

[1260] The processing flow will be explained below.

[1261] Step 1:

[1262] server

[1263] The server receives camera setting parameters (e.g., resolution, shooting interval) from the user and applies them to the camera and AI analysis system. It also performs test shooting to confirm that the settings are correctly reflected.

[1264] Step 2:

[1265] Terminal

[1266] The device takes pictures and videos of the food being served at a specified interval (e.g., every 5 minutes), and temporarily stores the captured data in its internal storage.

[1267] Step 3:

[1268] Terminal

[1269] The device transmits the photos and videos stored in its internal storage to the server in real time, and receives feedback to confirm that the data transmission was successful.

[1270] Step 4:

[1271] server

[1272] The server receives the photo and video data sent from the device and stores it in a database. When saving, it checks the integrity of the data to ensure there are no missing or error data.

[1273] Step 5:

[1274] server

[1275] The server then passes the stored data to the AI ​​model, which then evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[1276] Step 6:

[1277] server

[1278] The server receives the analysis results from the AI ​​model and uses them to detect defects and areas for improvement, such as identifying problems like "overcooked" or "messy presentation."

[1279] Step 7:

[1280] server

[1281] The server generates specific cooking advice based on the analysis results, and the advice is formatted in a way that is easy for the user to understand.

[1282] Step 8:

[1283] server

[1284] The server sends the generated cooking advice to the user's device and monitors the delivery status to ensure that the notification is displayed properly.

[1285] Step 9:

[1286] User

[1287] The user receives the advice notification from the server, checks the contents, and, if necessary, modifies the cooking method based on the advice.

[1288] Step 10:

[1289] server

[1290] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function, generates an appropriate ordering list for materials, and sends it to the user.

[1291] Step 11:

[1292] User

[1293] The user receives the order list sent from the server, checks the required materials, and then takes steps to order the materials.

[1294] summary

[1295] Through these steps, the present invention utilizes a food delivery robot to monitor product availability in real time, provide specific cooking advice based on AI analysis results, and detect ingredient shortages and assist with ordering, thereby improving quality control and operational efficiency.

[1296] Example 1

[1297] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1298] Traditionally, quality control processes in restaurants have often been performed manually, resulting in issues such as wasted human resources and inaccurate quality assessment. Furthermore, there was a lack of systems for early detection of defects and providing specific advice for improvement, making it difficult to maintain consistent quality. Furthermore, there was also a lack of systems for timely detection of ingredient shortages and appropriate ordering, resulting in inefficient ingredient management.

[1299] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1300] In this invention, the server includes a setting application means that receives settings for the image capture means from the user and applies them to the camera and analysis system, a captured data storage means that temporarily stores data, a database storage means that stores image data sent from the terminal in a database and checks consistency, and an ingredient detection means and ordering assistance means that detect ingredient shortages during analysis and provide an ordering assistance function. This automates and streamlines quality control in restaurants, enables the provision of specific cooking advice, and enables timely detection of ingredient shortages and effective ingredient management.

[1301] The "photography means" is a device that is attached to the food delivery robot and periodically takes pictures and videos.

[1302] The "data acquisition means" is a mechanism for collecting and storing image data acquired by the imaging means.

[1303] An "analysis means" is a system or algorithm for analyzing acquired image data and evaluating its content.

[1304] The "advice generating means" is a function that generates specific cooking advice based on the analysis results of the analyzing means.

[1305] The "notification means" is a system for notifying and displaying the generated cooking advice on the user's terminal.

[1306] The "setting application means" is a mechanism that reflects the camera setting parameters received from the user in the food delivery robot and analysis system.

[1307] The "photographing data storage means" is a storage system for temporarily storing image data obtained by the photographing means.

[1308] The "database storage means" is a mechanism that stores image data sent from the terminal in a database and checks its consistency.

[1309] "Material detection means" means a system or algorithm for detecting material shortages during analysis.

[1310] The "order assist means" is a function that creates an order list based on the shortage materials detected by the material detection means and prompts the user to place an order.

[1311] The present invention is a system that analyzes image data captured by a food delivery robot, provides cooking advice based on the results, and also manages ingredients. Specifically, this system is comprised of an image capture means, data acquisition means, analysis means, advice generation means, notification means, setting application means, captured data storage means, database storage means, ingredient detection means, and ordering assistance means attached to the food delivery robot.

[1312] server

[1313] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval) and applies them to the camera and AI analysis system. This ensures that the shooting method operates with the correct settings. The server then takes a test shot and checks the results to ensure the settings were applied properly.

[1314] Terminal

[1315] The device periodically captures images and videos of the food being served according to the specified interval. For example, the device takes a photo of the food every five minutes and temporarily stores the photo data in local storage. The captured data is periodically sent to the server.

[1316] Data storage and analysis

[1317] The server receives the photo and video data sent from the device and stores it in a database. At this time, the data is checked for consistency and to ensure there are no missing pieces. The saved data is then passed to the AI ​​model, which begins analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and returns the results to the server. Specifically, it may give an evaluation such as "the food is overcooked" or "the presentation is messy."

[1318] Cooking advice generation and notification

[1319] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. For example, the advice might be, "Reduce the cooking time by one minute." The generated cooking advice is then sent to the user's device. The user receives the notification, checks the content, and adjusts the cooking method accordingly for the next time.

[1320] Material detection and ordering assistance

[1321] Furthermore, if a shortage of ingredients is detected during AI analysis, the server will activate the ordering assistance function. For example, if it detects that there is a shortage of fresh lemons, a list of ingredients to order will be generated and sent to the user. The user can then check the list and order the necessary ingredients.

[1322] As described above, this system automates and streamlines quality control in restaurants. It also provides specific cooking advice, which is expected to improve cooking quality. Furthermore, as soon as a shortage of ingredients is detected, appropriate action can be taken immediately, which also improves the efficiency of ingredient management.

[1323] Prompt Sentence Examples

[1324] "We would like to build a system that can analyze photos of food taken by a food delivery robot and evaluate their quality. Based on that, it can provide cooking advice based on the needs of the customer. The system should evaluate factors such as cooking time, presentation, and color, and generate specific advice."

[1325] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1326] Step 1: Apply camera settings

[1327] The server receives camera setting parameters from the user (e.g., 1080p resolution, 5-minute shooting interval). The input is the camera parameters set by the user, and the output is the state in which the settings have been applied to the camera. The server sends the settings via the camera's API, and the settings are changed. Specifically, the server receives a request for camera settings and sends an HTTP request to the corresponding endpoint to reflect the settings.

[1328] Step 2: Conduct a test shoot

[1329] The server takes a test shot to confirm that the camera settings have been applied correctly and checks the results. The input is the state with the camera settings applied, and the output is the test image taken. The server sends a command to the camera to take a test shot, displays the acquired image in real time, and checks whether the settings have been applied.

[1330] Step 3: Conduct regular photo shoots

[1331] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes). The input is the interval based on the internal clock, and the output is the captured image and video data. Specifically, the device sets a timer, starts the camera to take a photo each time the timer expires, and saves the captured data in local storage.

[1332] Step 4: Sending and Receiving Data

[1333] The device temporarily stores the captured data and then sends it to the server. The server receives the data sent from the device, stores it in a database, and checks its consistency. The input is the image data generated by the device, and the output is stored in the database within the server. Specifically, the device sends data to the server using an HTTP POST request, and the server receives the data and inserts it into the database.

[1334] Step 5: Analysis by AI model

[1335] The server passes the stored data to the AI ​​model and begins analysis. The input is the image data stored in the database, and the output is the analysis results returned by the AI ​​model. The server sends the data to the AI ​​model's API endpoint, and the model evaluates the condition and quality of the food (color, shape, presentation, etc.), and receives the results.

[1336] Step 6: Generate cooking advice

[1337] The server generates specific cooking advice based on the analysis results obtained from the AI ​​model. The input is the analysis results of the AI ​​model, and the output is the cooking advice to be provided to the user. Specifically, the server uses a template to interpret the analysis results and generates specific advice such as "Please reduce the cooking time by one minute."

[1338] Step 7: Advice Notification

[1339] The server sends the generated advice to the user's device. The user checks that the notification is displayed properly and modifies the cooking method according to the received advice. The input is the generated cooking advice, and the output is the notification displayed on the user's device. Specifically, the server sends the advice via email or app notification, and the user checks the notification.

[1340] Step 8: Assist with ordering materials

[1341] If the server detects a shortage of materials during AI analysis, it activates the ordering assistance function. The input is the analysis results of the AI ​​model and material inventory data, and the output is an ordering list provided to the user. Specifically, the server generates an ordering list based on the material detection results and sends it to the user. The user receives the ordering list, checks the required materials, and places an order.

[1342] Through these steps, the system can automatically check product quality, provide specific cooking advice, and streamline ingredient management.

[1343] (Application example 1)

[1344] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1345] In restaurants using conventional food delivery robots, product quality management and cooking improvements were sometimes not properly carried out. It was also difficult to respond quickly to ingredient shortages or provide specific advice to staff. As a result, problems arose, such as a decrease in customer satisfaction and a loss of store operational efficiency. The present invention aims to solve these problems and improve the efficiency of cooking quality management, the provision of improvement advice, and ingredient management in restaurants.

[1346] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1347] In this invention, the server includes a photographing means attached to the food delivery robot, a data acquisition means for periodically acquiring photographed image data, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results by the analysis means, a notification means for notifying the user of the cooking advice, an ordering assistance means for detecting ingredient shortages and generating an ordering list, a communication means for communicating with the server to send and receive food images, and an AI analysis means for performing image analysis using an AI model. This allows for efficient provision of advice to improve cooking quality and ingredient management, thereby improving restaurant operation efficiency and customer satisfaction.

[1348] The "photography means" is a device that is attached to the food delivery robot and periodically acquires image data.

[1349] The "data acquisition means" is a device or program that receives image data captured by the image capture means and stores or transmits the data.

[1350] The "analysis means" is a program or device for evaluating the condition and quality of a product using the acquired image data.

[1351] The "advice generating means" is a program or device that generates specific cooking advice based on the analysis results from the analyzing means.

[1352] The "notification means" refers to a device or program for transmitting the generated cooking advice to the user.

[1353] The "order assistance means" is a program or device for detecting shortages of materials and generating an order list for the necessary materials.

[1354] The "communication means" refers to devices and programs for transmitting and receiving cooking images to and from the server via communication.

[1355] "AI analysis means" refers to a program or device that uses an AI model to analyze image data and evaluate the condition and quality of a product.

[1356] This invention relates to a system that checks product quality and provides cooking advice by installing a camera on a food delivery robot and analyzing periodically captured product images using AI. This system is composed of a food delivery robot, a photography means, a data acquisition means, an analysis means, an advice generation means, a notification means, an order assistance means, a communication means, and an AI analysis means.

[1357] 1. Hardware and Software Used

[1358] Hardware

[1359] Smartphones (e.g. iPhone, Android devices)

[1360] Food delivery robot (equipped with a camera)

[1361] Server (data processing, analysis)

[1362] software

[1363] AI analysis models (e.g., TensorFlow, PyTorch)

[1364] Database (e.g. MySQL, PostgreSQL)

[1365] Communication protocol (e.g. HTTP, WebSocket)

[1366] 2. Specific Embodiments of the System

[1367] (1) Data capture and acquisition

[1368] The delivery robot periodically takes photos of the food. The image data is automatically acquired at specified intervals (e.g., every 5 minutes). The acquired image data is temporarily stored and then sent to the server.

[1369] Example: "Take a photo of the food every 5 minutes in 1080p resolution and send it to the server."

[1370] (2) Data storage and analysis

[1371] The server receives the transmitted image data and stores it in a database. The stored data is then analyzed by an AI analysis means. The analysis means uses an AI model (e.g., TensorFlow, PyTorch) to evaluate the condition and quality of the product based on the image data.

[1372] Example: "Analyze the received photo data using an AI model and evaluate the quality of the food. If there is a problem, notify us. Save it on the server."

[1373] (3) Advice generation

[1374] Based on the analysis results, the advice generating means generates specific cooking advice, which is then sent to the smartphone via the communication means.

[1375] Example: "Based on the analysis results, generate specific cooking advice. For example, 'Reduce the cooking time by 1 minute.'"

[1376] (4) Order assistance function

[1377] Furthermore, if the server detects a shortage of ingredients during the AI ​​analysis process, it will use the ordering assistance tool to generate a list of the ingredients that are in short supply, which will also be sent to the smartphone.

[1378] Example: "Detect shortages of materials and generate an order list to notify you."

[1379] 3. Specific Examples

[1380] For example, a smartphone app might send the following prompt to the user:

[1381] "Take a photo of your food every 5 minutes in 1080p resolution and send it to the server."

[1382] "The received photo data is analyzed using an AI model to evaluate the quality of the food. If there is a problem, we will notify you. Please save it on our server."

[1383] "Based on the analysis results, please generate specific cooking advice, such as 'Reduce the cooking time by one minute.'"

[1384] "Detect material shortages and generate an order list to notify you."

[1385] This allows the food delivery robot to periodically take photos of the food and send them to the server. The server then analyzes the image data, evaluates the cooking quality, and generates advice. It can also detect ingredient shortages and generate an ordering list for the necessary ingredients, improving restaurant operation efficiency and customer satisfaction.

[1386] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1387] Step 1:

[1388] Data capture

[1389] The terminal (serving robot) takes pictures of food at a specified interval (e.g., every 5 minutes). The input is the interval and camera settings (resolution, focal length, etc.). The output is the captured food image data. Specifically, the serving robot automatically starts the camera and presses the capture button.

[1390] Step 2:

[1391] Temporary storage and transmission of data

[1392] The device temporarily stores the captured food image data and then sends it to the server. The input is the food image data acquired in step 1. The output is the food image data sent to the server. Specifically, the device stores the image data in local storage and then uploads the data to the server via the network.

[1393] Step 3:

[1394] Receiving and storing data

[1395] The server receives the food image data sent from the terminal. The input is the food image data sent from the terminal. The output is the food image data stored in the database. Specifically, the server receives the HTTP request and stores the image data in the database.

[1396] Step 4:

[1397] Data Integrity Check

[1398] The server checks the consistency of the stored image data. The input is the food image data stored in the database. The output is food image data whose consistency has been confirmed. Specifically, the server checks the size and format of the data to see if there are any inconsistencies.

[1399] Step 5:

[1400] Image data analysis

[1401] The server analyzes image data using an AI model. The input is food image data whose consistency has been confirmed. The output is the analysis results that evaluate the state and quality of the food. Specifically, the AI ​​analysis means uses TensorFlow and PyTorch to analyze the image and evaluate the color, shape, and presentation.

[1402] Step 6:

[1403] Advice Generation

[1404] The server generates specific cooking advice based on the analysis results. The input is the analysis results obtained in step 5. The output is the generated cooking advice. Specifically, specific points for improvement, such as "Please reduce the baking time by one minute," are generated in text format.

[1405] Step 7:

[1406] Advice Notice

[1407] The server notifies the user's smartphone of the generated cooking advice. The input is the cooking advice generated in step 6. The output is the cooking advice displayed on the smartphone. Specifically, the server sends the advice using a notification protocol, and a pop-up notification is displayed on the smartphone.

[1408] Step 8:

[1409] Activating the ordering assistance function

[1410] If the server detects a shortage of materials during the AI ​​analysis process, it activates the ordering assistance means. The input is the analysis results from step 5 and the materials database. The output is a materials ordering list. Specifically, the server detects a material shortage and automatically generates an ordering list.

[1411] Step 9:

[1412] Order list notification

[1413] The server notifies the user's smartphone of the generated order list. The input is the order list generated in step 8. The output is the order list displayed on the smartphone. Specifically, the server sends the order list using a notification protocol, and it is displayed on the smartphone.

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] This invention relates to a system that uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[1416] System Configuration

[1417] 1. Filming Method

[1418] Terminal

[1419] The camera is attached to the food delivery robot and positioned so that it can capture a wide range of the food delivery area. The camera takes photos and videos at specified intervals and generates the data.

[1420] 2. Data Acquisition Method

[1421] Terminal

[1422] Image and video data captured by the camera is temporarily stored in the device's internal storage, and then uploaded to a server in real time.

[1423] 3. Analysis method

[1424] server

[1425] The server passes the uploaded image data to an AI model for analysis, which evaluates the condition and quality of the food (e.g., color, shape, and presentation).

[1426] 4. Advice Generation Methods

[1427] server

[1428] Based on the analysis results, the server generates specific cooking advice using specific algorithms and evaluation criteria to provide optimal suggestions for the user.

[1429] 5. Emotion Engine

[1430] server

[1431] The emotion engine analyzes sensor information from cameras, microphones, etc. to obtain user emotional data. It recognizes emotions from facial expressions, tone of voice, gestures, etc. and sends the data to the server.

[1432] 6. Means of notification

[1433] server

[1434] Based on the analysis results and emotional data, the server generates cooking advice in the most appropriate format and wording and notifies the user's device.

[1435] Overview of program processing

[1436] The program of this system is designed so that each means cooperates to acquire data, analyze it, generate advice, and notify it. The process flow is explained below with a concrete example.

[1437] 1. Data Acquisition

[1438] The device takes a photo of the food every five minutes and sends the image data in real time to a server, which stores the data in a database.

[1439] 2. Data Analysis

[1440] The server inputs the stored image data into the AI ​​model and analyzes the condition of the food. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that the food is overcooked.

[1441] 3. Emotion Data Acquisition

[1442] The emotion engine uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. For example, if it determines that the user is confused, it sends that data to the server.

[1443] 4. Advice Generation

[1444] The server combines the AI ​​analysis results with the emotion data from the emotion engine to generate cooking advice, such as recommending that users reduce the cooking time by one minute. Confused users are given advice in a polite and calm tone.

[1445] 5. Notification

[1446] The server sends the generated advice to the user's device. For example, a notification may be sent saying, "Please reduce the cooking time by one minute. If you need help, please contact support."

[1447] Application of specific examples

[1448] A restaurant chain is introducing a new food delivery robot system to reduce the workload of its chefs while consistently improving the quality of the food they serve. With this system, as the robot delivers food, it takes a photo of the food with a camera and uses AI to analyze it. Based on the analysis results, it generates specific advice on cooking times and methods, and even provides customized advice based on the chef's emotions. This allows the chef to work efficiently while maintaining a consistently high level of food quality.

[1449] This invention is a system that improves quality control and operational efficiency in restaurants by combining three elements: a food delivery robot, AI analysis, and an emotion engine. It is particularly useful for companies with multiple stores or those expanding overseas.

[1450] The processing flow will be explained below.

[1451] Step 1:

[1452] User

[1453] To perform the initial setup of the system, the user inputs the camera resolution, shooting interval, emotion engine setting parameters, etc. into the server. The user confirms that the initial setup is complete.

[1454] Step 2:

[1455] server

[1456] The server applies the setting parameters received from the user to the camera and AI analysis system, and then conducts a test shoot to confirm that the settings have been correctly applied.

[1457] Step 3:

[1458] Terminal

[1459] The device takes images and videos of the food being served at a specified interval (e.g., every 5 minutes) and temporarily stores them in its internal storage.

[1460] Step 4:

[1461] Terminal

[1462] The device uploads stored image and video data to a server in real time, and receives feedback to confirm successful data transmission.

[1463] Step 5:

[1464] server

[1465] The server receives the image data sent from the device and stores it in a database. When saving, it checks the integrity of the data and makes sure there are no missing parts.

[1466] Step 6:

[1467] server

[1468] The server passes the stored image data to the AI ​​model to begin analysis. The AI ​​model evaluates the condition and quality of the food (color, shape, presentation, etc.) and outputs the results.

[1469] Step 7:

[1470] server

[1471] The server receives the results of the AI ​​analysis and records the condition of the food determined from the image data (e.g., overcooked, messy presentation, etc.) as the analysis result.

[1472] Step 8:

[1473] Terminal

[1474] The emotion engine installed in the device uses a camera and microphone to analyze the user's facial expressions and tone of voice in real time and generate emotion data.

[1475] Step 9:

[1476] Terminal

[1477] The device transmits the generated emotion data to the server, which conveys the user's current emotional state (e.g., confusion, joy, anger, etc.) to the server.

[1478] Step 10:

[1479] server

[1480] The server combines the AI ​​analysis results with emotional data to generate appropriate cooking advice. For example, if the user is confused, the server generates advice in a polite and calm tone. If the analysis results indicate that the cooking time should be shortened by one minute, this advice will also be included.

[1481] Step 11:

[1482] server

[1483] The generated cooking advice is sent to the user's device, and the wording and format of the notification are adjusted based on the emotion data.

[1484] Step 12:

[1485] User

[1486] The user receives the advice notification from the server, checks the content, and, if necessary, modifies the cooking method based on the advice.

[1487] Application of specific examples

[1488] For example, if a restaurant chain were to implement this system, while testing a new menu item, the delivery robot would take photos of the food every five minutes and send them to the server. The server would then analyze these photos using an AI model and determine whether the food was overcooked. Furthermore, an emotion engine would analyze the chef's facial expressions and detect whether the chef was confused. Based on this data, the server would generate a gentle message to the user, saying, "Please reduce the cooking time by one minute. If you have any questions, please contact support." The chef would then check the notification and shorten the cooking time the next time, thereby improving the quality of the food.

[1489] In this way, this system combines food delivery robots, AI analysis, and an emotion engine to significantly improve quality control and operational efficiency in restaurants.

[1490] Example 2

[1491] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1492] In restaurants, consistently improving the quality of the food served while reducing the workload of chefs is an important issue. Furthermore, when chefs or employees are stressed or overwhelmed, support is needed to reduce their workload and enable them to work efficiently. Current technology lacks a system that satisfies these needs, so a new system is needed that simultaneously manages food quality and provides psychological support to chefs.

[1493] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1494] In this invention, the server includes a photographing means, a data acquisition means, an analysis means, an advice generation means, an emotion recognition means, and a notification means, which makes it possible to provide customized advice according to the emotional state of the chef or staff, in addition to evaluating the quality of the food and providing cooking advice.

[1495] The "photography means" is a device that is attached to the food delivery robot and periodically takes photos and videos of the food.

[1496] The "data acquisition means" is a function that records image data and video data acquired by the image capture means and transmits them to a server as necessary.

[1497] The "analysis means" refers to an analysis function that includes an AI model that evaluates the condition and quality of food based on image and video data uploaded to the server.

[1498] The "advice generating means" is a function that generates specific cooking advice for the user based on the analysis results by the analyzing means.

[1499] The "emotion recognition means" is a function that analyzes emotional data such as the user's facial expressions, tone of voice, and gestures, and customizes advice based on the analysis results.

[1500] The "notification means" is a function that notifies the user's terminal of the generated cooking advice.

[1501] This system uses a camera attached to a food delivery robot and an AI analysis system to periodically take photos and videos of the food being served, analyzes the data, and generates cooking advice and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the cooking advice can be customized based on the user's emotions.

[1502] First, the device uses its camera to periodically take photos and videos of the food. The camera is attached to the food delivery robot and is positioned so that it can capture a wide range of the food delivery area. Specifically, the camera takes photos every five minutes, and the captured images and video data are temporarily stored in the device's internal storage. The data is then uploaded to the server in real time.

[1503] The server stores the received image and video data in a database. The stored data is then input into an AI model to evaluate the state and quality of the food. The AI ​​model uses deep learning, for example, to analyze the color, shape, and presentation of the image. For example, if the color indicates that the food is overcooked, the AI ​​model will determine that it is "overcooked."

[1504] The device then uses its camera and microphone to record the user's facial expressions and tone of voice, and sends the data in real time to a server. The server then uses an emotion engine to analyze the user's emotional state. For example, if the server detects that the user is confused, it will use that information.

[1505] As a result, the server combines the results of the food's condition analysis with the user's emotional data to generate specific cooking advice. For example, if the food is judged to be "overcooked," it generates advice such as "reduce the cooking time by one minute." Furthermore, if the user is confused, the advice will be more polite, such as "reduce the cooking time by one minute. If you have any questions, please contact support."

[1506] Finally, the server sends the generated advice to the user's device, which then provides the advice to the user using notification functions and alerts.

[1507] Specific examples

[1508] For example, a restaurant chain is introducing a new robotic food delivery system. This system reduces the workload of chefs and consistently improves the quality of the food they serve. A camera attached to the robot periodically photographs the food and sends the image data to a server in real time. The server uses an AI model to analyze the images and generate specific cooking advice. It also analyzes the chef's facial expressions and tone of voice to provide customized advice based on their emotions.

[1509] Prompt Sentence Examples

[1510] "Generate suggestions for today's doneness based on photos of food taken. If the user is confused about what to cook, please provide polite suggestions taking their situation into consideration."

[1511] This allows the system to reduce the workload of the chef while maintaining consistently high food quality.

[1512] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1513] Step 1: Data Acquisition

[1514] The device takes photos and videos of the food every five minutes. For example, it uses a built-in camera to take photos of the entire serving area. The input is image data and video data captured by the camera. The output is that this photographed data is temporarily saved in the built-in storage. Specifically, the camera shutter is automatically released and data is collected at specified intervals.

[1515] Step 2: Send data

[1516] The device sends the temporarily saved image and video data to the server. The input is the captured image data obtained in step 1. The output is the image and video data uploaded to the server. The specific operation is that the data is transferred to the server via an HTTP POST request.

[1517] Step 3: Save Data

[1518] The server stores the received image data and video data in a database. The input is the data sent in step 2. The output is the image data and video data organized and stored in the database. The specific operation is to store and organize the data through a database management system (DBMS).

[1519] Step 4: Data analysis

[1520] The server inputs the stored image and video data into the AI ​​model to analyze the condition and quality of the food. The input is the image and video data stored in the database. The output is the analysis results of the food condition (e.g., color, shape, presentation). Specifically, it uses a deep learning model to identify the characteristics of the image and make an evaluation based on that.

[1521] Step 5: Acquire emotion data

[1522] It uses the device's camera and microphone to record the user's facial expressions and tone of voice. The input is the user's facial and voice data. The output is emotion data that is sent to the server. Specifically, it uses advanced facial recognition algorithms and tone of voice analysis software to recognize emotions and transfers the data to the server.

[1523] Step 6: Sentiment Data Analysis

[1524] The server uses the emotion engine to analyze the user's emotional state. The input is the emotion data obtained in step 5. The output is the analyzed user's emotional state. Specifically, the server infers the user's psychological state from facial expression data and tone of voice, and analyzes this information using the analysis engine.

[1525] Step 7: Advice Generation

[1526] The server integrates the cooking state analysis results with the user's emotional data to generate cooking advice. The input is the analysis results from steps 4 and 6. The output is specific cooking advice to provide to the user. The specific operation is to analyze the analysis results using a specific algorithm and generate the optimal cooking method. For example, the advice generated is "Please reduce the cooking time by 1 minute."

[1527] Step 8: Advice Notification

[1528] The server sends the generated advice to the user's device. The input is the cooking advice generated in step 7. The output is a notification displayed on the user's device. The specific operation is to send the advice to the device via a communication protocol such as an HTTP POST request, and the device notifies the user. For example, the notification may say, "Please reduce the cooking time by one minute. If you need help, please contact support."

[1529] (Application example 2)

[1530] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1531] Existing restaurant systems using food delivery robots have the problem of insufficient food quality control. They also lack specific cooking advice to maintain consistent food quality. Furthermore, they do not provide services that take into account the user's emotional state, making it difficult to improve the user experience.

[1532] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1533] In this invention, the server includes a camera attached to the food delivery robot, a data acquisition means for periodically acquiring image data captured by the camera, an analysis means for analyzing the image data, an advice generation means for generating cooking advice based on the analysis results of the analysis means, a notification means for notifying the user of the cooking advice, an emotion analysis means for analyzing emotional data such as the user's facial expression and tone of voice and customizing the cooking advice, and an advice customization means for optimizing the cooking advice notified to the user's device to match the user's emotional state. This makes it possible to monitor the condition of the food in real time, detect quality defects early, and provide cooking advice tailored to the user's emotional state, improving the user experience.

[1534] A "serving robot" is an automated mechanical device designed to deliver food and drinks.

[1535] "Capture means" refers to the camera or sensor device used to capture images or videos.

[1536] The "data acquisition means" is a system for collecting and storing image data and video data obtained by the imaging means.

[1537] The "analysis means" refers to software or hardware for processing acquired image data or video data and extracting specific information.

[1538] The "advice generator" is a system that generates instructions to recommend specific actions or changes based on the results obtained by the analysis means.

[1539] The "notification means" is a communication means for conveying the generated advice or instructions to the user.

[1540] The "emotion analysis means" is a system for analyzing the user's facial expression, tone of voice, etc., to determine the user's emotional state.

[1541] The "advice customization means" is a system for optimizing the advice and instructions to be notified based on the user's emotional data obtained by the emotion analysis means.

[1542] A "user terminal" is a device used by a user to receive information, such as a smartphone or tablet.

[1543] This invention aims to improve quality control and user experience in the food delivery industry by using a camera attached to a food delivery robot, an AI analysis system, and an emotion analysis system. This system monitors the cooking status in real time and provides advice based on emotion analysis. Specific methods for hardware, software, data processing, and data calculation used to implement this invention are described below.

[1544] Hardware

[1545] Food delivery robot: an automated mechanical device for delivering food, equipped with cameras and other sensors.

[1546] Camera: A means of capturing images attached to the robot. For example, a webcam such as the Logitech C920.

[1547] Terminal: A device used to collect and process data and upload it to a server. An example is a single-board computer such as a Raspberry Pi.

[1548] Microphone: An input device for capturing the tone of a user's voice. For example, a high-quality microphone such as a Blue Yeti.

[1549] software

[1550] OpenCV: An open-source library for image capture and processing.

[1551] Requests: A Python library for communicating with the server.

[1552] TensorFlow / Keras: Machine learning frameworks that form the basis of the sentiment analysis model and cooking analysis AI model.

[1553] Custom Emotion Recognition Model: A model for recognizing a user's emotions from facial expressions and tone of voice.

[1554] Custom AI Model: An artificial intelligence model for analyzing the state of cooking and generating cooking advice.

[1555] Data processing and calculation

[1556] Data acquisition: The device periodically (e.g., every 5 minutes) takes pictures of the food using its camera, temporarily stores them in storage, and then uploads them to the server.

[1557] Data analysis: The server passes the uploaded image data to the analysis means to evaluate the state of the food (color, shape, presentation). The AI ​​model does this using specific algorithms.

[1558] Emotion data acquisition: The emotion analysis means uses a camera and microphone to collect the user's facial expressions and tone of voice, determine the user's emotional state, and send it to the server.

[1559] Advice generation: The server generates cooking advice based on the analysis results and emotion data. For example, it generates specific advice such as "We recommend shortening the cooking time by one minute."

[1560] Advice notification: The generated advice is sent to the user's device. If an emotion such as confusion is recognized, the advice is notified in a way that takes into consideration that emotion.

[1561] Specific examples

[1562] For example, a food delivery robot takes a photo of the food during delivery and sends it to a server, where an AI model analyzes it. Based on the analysis results, advice such as "reheating is necessary" is generated. If the user is confused, the emotion analysis system can detect this and provide additional support, such as "we'll explain in detail how to reheat it."

[1563] Prompt Sentence Examples

[1564] Below are some example prompts for the generative AI model and sentiment analysis system:

[1565] text

[1566] Provide a photo and generate optimal cooking advice based on emotional data from our emotion engine. We evaluate whether the food is properly cooked (color, shape, presentation, etc.) and take into account the user's feelings of confusion or dissatisfaction to customize the advice.

[1567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1568] Step 1:

[1569] The device uses the camera to take pictures of the food at regular intervals (e.g., every 5 minutes). The captured images are temporarily stored in the device's storage. The input is image data acquired from the camera, and the output is an image file saved in the storage.

[1570] Step 2:

[1571] The device uploads the temporarily saved image data to the server by executing an HTTP POST request using the Requests library. The input is the image file in storage, and the output is a notification to the server that the upload is complete.

[1572] Step 3:

[1573] The server receives the uploaded image data and stores it in a database. The input is the image file sent from the device, and the output is the image data stored in the database. Specifically, data is received through a RESTful API.

[1574] Step 4:

[1575] The server passes the stored image data to an AI model, which is the analytical tool, and evaluates the condition of the food (color, shape, presentation). The input is the image data in the database, and the output is the analysis result (e.g., overcooked, poor color, etc.). Specifically, the analysis is performed using a machine learning framework such as TensorFlow or Keras.

[1576] Step 5:

[1577] The server generates cooking advice based on the analysis results. This advice is calculated using a specific algorithm. The input is the analysis results, and the output is specific cooking advice (e.g., "reduce the cooking time by 1 minute").

[1578] Step 6:

[1579] At the same time, the server uses emotion analysis means to obtain the user's emotion data. It collects and analyzes the user's facial expressions and tone of voice through a camera or microphone. The input is the facial expression and voice data obtained from the camera or microphone, and the output is the user's emotional state (e.g., confusion, joy). Specific operations use an emotion analysis model.

[1580] Step 7:

[1581] The server generates customized cooking advice that takes emotion data into consideration. The inputs are the analysis results and emotion data, and the output is emotion-sensitive cooking advice (e.g., "I'll explain it carefully, but please reduce the cooking time by 1 minute").

[1582] Step 8:

[1583] The server notifies the user's terminal of the generated cooking advice. The notification means is used to send emotion-conscious advice. The input is customized cooking advice, and the output is a notification message to the user's terminal.

[1584] Step 9:

[1585] The user receives the advice on the device and adjusts the cooking accordingly, specifically by referring to the message displayed on the device.

[1586] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1587] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1588] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1589] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1590] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1591] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1592] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1593] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1594] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1595] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1596] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1597] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1598] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1599] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1600] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1601] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1602] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1603] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1604] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1605] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1606] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1607] The following is further disclosed regarding the above embodiment.

[1608] (Claim 1)

[1609] A photographing means attached to the food delivery robot;

[1610] data acquisition means for periodically acquiring image data captured by the imaging means;

[1611] analysis means for analyzing the image data;

[1612] an advice generating means for generating cooking advice based on the analysis result by the analyzing means;

[1613] The system further includes a notification means for notifying the user of the cooking advice.

[1614] (Claim 2)

[1615] 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot.

[1616] (Claim 3)

[1617] 2. The system according to claim 1, wherein the analysis means evaluates the condition and quality of the product from the image data, detects defects, and generates improvement advice based on the evaluation results.

[1618] "Example 1"

[1619] (Claim 1)

[1620] A photographing means attached to the food delivery robot;

[1621] data acquisition means for periodically acquiring image data captured by the imaging means;

[1622] analysis means for analyzing the image data;

[1623] an advice generating means for generating cooking advice based on the analysis result by the analyzing means;

[1624] a notification means for notifying the user of the cooking advice on a terminal;

[1625] a setting application means for receiving settings of the photographing means from a user and applying the settings to the camera and the analysis system;

[1626] a photographic data storage means for temporarily storing data;

[1627] a database storage means for storing the image data transmitted from the terminal in a database and checking the consistency;

[1628] The system includes a material detection means and an ordering assistance means for detecting a shortage of material during analysis and providing an ordering assistance function.

[1629] (Claim 2)

[1630] 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot, and settings such as the photographing interval can be adjusted by the user.

[1631] (Claim 3)

[1632] The system of claim 1, wherein the analysis means evaluates the condition and quality (color, shape, presentation, etc.) of the product from image data, detects defects (overcooking, messy presentation, etc.) and generates improvement advice based on the evaluation results, and further has the function of detecting shortages of ingredients.

[1633] "Application Example 1"

[1634] (Claim 1)

[1635] A photographing means attached to the food delivery robot;

[1636] data acquisition means for periodically acquiring image data captured by the imaging means;

[1637] analysis means for analyzing the image data;

[1638] an advice generating means for generating cooking advice based on the analysis result by the analyzing means;

[1639] notification means for notifying the user of the cooking advice;

[1640] an ordering assistance means for detecting shortages of materials and generating an ordering list;

[1641] a communication means for communicating with a server to send and receive cooking images;

[1642] An AI analysis means for performing image analysis using an AI model;

[1643] A system including:

[1644] (Claim 2)

[1645] 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot.

[1646] (Claim 3)

[1647] 2. The system according to claim 1, wherein the analysis means evaluates the condition and quality of the product from the image data, detects defects, and generates improvement advice based on the evaluation results.

[1648] "Example 2: Combining Emotion Engines"

[1649] (Claim 1)

[1650] A photographing means attached to the food delivery robot;

[1651] data acquisition means for periodically acquiring image data captured by the imaging means;

[1652] analysis means for analyzing the image data;

[1653] an advice generating means for generating cooking advice based on the analysis result by the analyzing means;

[1654] emotion recognition means for customizing cooking advice based on emotion data of a user;

[1655] The system further includes a notification means for notifying the user of the cooking advice.

[1656] (Claim 2)

[1657] 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot.

[1658] (Claim 3)

[1659] 2. The system according to claim 1, wherein the analysis means evaluates the condition and quality of the product from the image data, detects defects and generates improvement advice based on the evaluation results.

[1660] "Application example 2 when combining emotion engines"

[1661] (Claim 1)

[1662] A photographing means attached to the food delivery robot;

[1663] data acquisition means for periodically acquiring image data captured by the imaging means;

[1664] analysis means for analyzing the image data;

[1665] an advice generating means for generating cooking advice based on the analysis result by the analyzing means;

[1666] notification means for notifying the user of the cooking advice;

[1667] An emotion analysis means for analyzing emotion data such as the user's facial expressions and tone of voice to customize cooking advice;

[1668] an advice customization means for optimizing cooking advice notified to the user's terminal in accordance with the user's emotional state;

[1669] A system including:

[1670] (Claim 2)

[1671] 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot.

[1672] (Claim 3)

[1673] 2. The system according to claim 1, wherein the analysis means evaluates the condition and quality of the product from the image data, detects defects, and generates improvement advice based on the evaluation results. [Explanation of symbols]

[1674] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A photographing means attached to the food delivery robot; data acquisition means for periodically acquiring image data captured by the imaging means; analysis means for analyzing the image data; an advice generating means for generating cooking advice based on the analysis result by the analyzing means; The system further includes a notification means for notifying the user of the cooking advice.

2. 2. The system according to claim 1, wherein the photographing means is arranged so as to be able to photograph the serving area of ​​the serving robot.

3. 2. The system according to claim 1, wherein the analysis means evaluates the condition and quality of the product from the image data, detects defects, and generates improvement advice based on the evaluation results.

Images