Information processing system

Abstract

The application provides an information processing system. An information processing system, characterized by comprising: a processor; wherein the processor is configured to: receive a user condition through an interface for receiving a condition input from a user; acquire food material information in a refrigerator by using a sensor; and generate prompt information for instructing a generative artificial intelligence model to generate a menu on the basis of the acquired food material information and the user condition.

Description

Technical Field

[0001] The technology disclosed herein relates to an information processing system. Background Technology

[0002] Japanese Patent Application Publication No. 2022-180282 discloses a method for controlling a role-based chatbot executed by at least one processor. The method includes the following steps: receiving a user's speech; adding the user's speech to a prompt word, the prompt word containing instruction statements associated with an explanation of the chatbot's role; encoding the prompt word; and inputting the encoded prompt word into a language model to generate a chatbot response to the user's speech.

[0003] As residents' living standards improve, their demands for health, convenience, and affordability in daily diets are constantly increasing. However, in existing technologies, users typically need to manually check the ingredients in their refrigerator, search for recipes, and determine whether they meet health requirements and time constraints when creating daily meal plans. This is not only time-consuming and labor-intensive but also prone to overlooking the shelf life of ingredients, leading to food waste. Furthermore, existing meal plan recommendation systems often rely on preset rules or simple recipe matching, making it difficult to reflect users' personalized health conditions in a timely and comprehensive manner. They also struggle to comprehensively optimize meal plans by considering ingredient shelf life and power consumption, failing to effectively help users reduce food waste and achieve economic benefits in terms of electricity consumption while meeting their healthy dietary needs. Therefore, it is necessary to provide a system that can automatically acquire information about ingredients in the refrigerator, combine it with user conditions, and use a generative artificial intelligence model to generate meal plans, while simultaneously reducing food waste and optimizing power consumption, to solve the aforementioned problems. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides an information processing system comprising a processor configured to: receive user conditions via an interface for receiving condition input from a user; acquire food information within a refrigerator using sensors; and, based on the acquired food information and the user conditions, generate prompts instructing a generative artificial intelligence model to generate a suitable meal, thereby enabling the generative artificial intelligence model to automatically generate appropriate meals based on real-time food status and the user's personalized conditions. Furthermore, the processor is further configured to: reflect health-related conditions from the user in the prompts to generate prompts instructing the generative artificial intelligence model to generate healthy meals, thus ensuring that the generated meals better meet the user's requirements for healthy eating. Moreover, the processor is further configured to: consider the shelf life of food within the refrigerator when generating meals, prioritizing recipes corresponding to food nearing its expiration date to reduce food waste; and optimize power consumption by comprehensively considering factors such as cooling load, door opening / closing frequency, and usage time based on meal arrangements, thereby providing economic benefits to the user while ensuring food quality and health needs. Through the above-described structure, the present invention can effectively solve the problems of cumbersome manual management of ingredients and difficulty in balancing health needs and economy in the prior art, based on automatic identification of ingredients and intelligent generation of ingredients.

[0005] "System" refers to an overall device or combination of devices that includes at least one processor and hardware and software modules that interact with the processor to perform the functions described in this invention, and may include servers, terminal devices, sensor devices, and storage media, etc.

[0006] A "processor" is a computing unit that can execute program instructions, process input data, and output processing results. It can be a single physical processor, a collection of multiple physical processors, or a central processing unit, microcontroller, signal processor, etc., integrated into a server, terminal, or dedicated control board.

[0007] An "interface" refers to a hardware or software structure used to transmit data or control information between a user and a system, or between different modules within a system. These include graphical user interfaces, touchscreen interfaces, voice input interfaces, application programming interfaces (APIs), and network communication interfaces.

[0008] "User conditions" refers to a set of parameters input by the user through the interface to constrain or specify the requirements for the creation of the offering, including but not limited to health-related conditions, cooking time limits, taste preferences, dietary restrictions, and nutritional needs.

[0009] "Sensor" refers to a hardware device used to detect and output information related to the status of food in the refrigerator, including but not limited to cameras, weight sensors, temperature sensors, humidity sensors, radio frequency identification (RFID) readers, and barcode / QR code reading devices.

[0010] "Information on food inside the refrigerator" refers to data obtained through sensors and processed by the system, which can characterize the types, quantities, storage locations, and shelf lives of food inside the refrigerator.

[0011] "Generative artificial intelligence model" refers to a model trained based on machine learning or deep learning techniques that can automatically generate text or structured output based on input prompts. In this invention, it is used to generate the proposal and its related descriptions.

[0012] "Prompt information" refers to text or structured data generated by the processor based on user conditions and information about the food in the refrigerator, which serves as input to the generative artificial intelligence model. It is used to instruct and constrain the generative artificial intelligence model to generate offerings that meet predetermined requirements.

[0013] "Prepared meal plan" refers to a meal combination plan generated by the system and provided to the user, including the name of the dishes for at least one or more meals, the required ingredients, the amount, the cooking steps, and optional nutritional or energy information.

[0014] "Health-related conditions" refer to conditions set by users based on their health status or health goals to limit the scope of donation generation, including but not limited to requirements such as low fat, low salt, low sugar, high protein, controlled calorie intake, and suitability for specific diseases or allergies.

[0015] "Health Contribution" refers to a contribution that meets the user's basic dietary needs, fulfills the user's set health-related conditions, and achieves predetermined health standards in terms of nutritional balance, energy intake, and specific nutrient ratios.

[0016] "Shelf life" refers to information that characterizes the time limit within which food in the refrigerator can be safely and healthily consumed, including but not limited to shelf life, best-before date, or remaining usable time estimated by the system.

[0017] "Food waste" refers to situations where edible ingredients are discarded or rendered inedible due to reasons such as exceeding their expiration date, spoilage, or long-term non-use.

[0018] "Electricity consumption" refers to the total amount of electrical energy consumed by the system during operation, as well as by functions related to refrigerator refrigeration, lighting, and temperature regulation.

[0019] "Economic benefits" refers to the beneficial effects that the use of the system of the present invention brings to users in terms of cost savings or improved resource utilization efficiency, including but not limited to the cost savings in food ingredients due to reduced food waste and the reduction in electricity expenses due to optimized power consumption. Attached Figure Description

[0020] Figure 1 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the first embodiment.

[0021] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0022] Figure 3 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the second embodiment.

[0023] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0024] Figure 5 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the third embodiment.

[0025] Figure 6 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and head-mounted terminal according to the third embodiment.

[0026] Figure 7 This is a conceptual diagram illustrating an example of the configuration of the data processing system according to the fourth embodiment.

[0027] Figure 8 This is a conceptual diagram illustrating an example of the main functions of the data processing device and robot according to the fourth embodiment.

[0028] Figure 9 This represents an emotion map that maps multiple emotions.

[0029] Figure 10 This represents an emotion map that maps multiple emotions.

[0030] Figure 11 This is a sequence diagram illustrating the processing flow of the data processing system of the first embodiment.

[0031] Figure 12 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 1.

[0032] Figure 13 This is a sequence diagram illustrating the processing flow of the data processing system of the second embodiment.

[0033] Figure 14 This is a sequence diagram illustrating the processing flow of the data processing system in Application Example 2. Detailed Implementation

[0034] Hereinafter, an example of an implementation of the system according to the present disclosure will be described with reference to the accompanying drawings.

[0035] First, let me explain the terminology used in the following instructions.

[0036] In the following embodiments, the processor (hereinafter referred to as "processor") with reference numerals may be a single computing device or a combination of multiple computing devices. Furthermore, the processor may be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0037] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory that temporarily stores information and is used as working memory by the processor.

[0038] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices that store various programs and parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disks (e.g., hard disks), or magnetic tapes.

[0039] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface that includes a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. As an example of a communication specification applicable to the communication I / F, wireless communication specifications such as 5G (5th Generation Mobile Communication System), Wi-Fi (wireless fidelity) (registered trademark), or Bluetooth (registered trademark) can be listed.

[0040] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects to express more than three items, the same interpretation as "A and / or B" applies.

[0041] First Implementation Method Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0042] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. A server can be cited as an example of the data processing device 12.

[0043] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0044] The smart device 14 includes a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. In addition, the receiving device 38, output device 40, camera 42, and communication I / F 44 are also connected to the bus 52.

[0045] The receiving device 38 includes a touchscreen 38A and a microphone 38B, and receives user input. The touchscreen 38A receives user input via touch by detecting contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input via sound by detecting the user's voice. The control unit 46A in the processor 46 sends data representing the user input received by the touchscreen 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data representing the user input.

[0046] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting data in a form perceptible to the user 20 (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0047] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0048] Figure 2 The diagram shows an example of the main functions of the data processing device 12 and the smart device 14.

[0049] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0050] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0051] In the smart device 14, the processor 46 performs the acceptance output processing. The memory 50 stores the acceptance output program 60. The acceptance output program 60 is used in conjunction with the data processing system 10 and the specific processing program 56. The processor 46 reads the acceptance output program 60 from the memory 50 and executes the read acceptance output program 60 on the RAM 48. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48. Furthermore, the smart device 14 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290. The acceptance output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance output program 60 executed on the RAM 48.

[0052] Alternatively, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-held terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing of the data processing system 10 of the first embodiment will be described.

[0053] Example 1 The flow of a specific process in Example 1 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. Furthermore, the data processing device 12 is referred to as the "server," and the smart device 14 is referred to as the "terminal."

[0054] In technologies that utilize computer technology to generate menu schemes, user-inputted conditions are typically passed directly to a generative artificial intelligence model using simple rules or fixed templates to obtain recommendation results. However, this approach suffers from the following technical problems: First, information processing devices often fail to perform unified modeling and in-depth analysis of data from multiple sources. For example, there is a lack of unified internal computer representation and comprehensive processing mechanism between the free text conditions, structured health conditions, and time constraints input by users through terminals and the type information, expiration date information, and user historical selection record information of objects (such as food ingredients) in the storage container. This results in insufficient and inaccurate expression of prompts received by generative artificial intelligence models, making it difficult for the generated results to take into account health, time constraints, preferences, and resource utilization efficiency.

[0055] Second, existing systems generally treat generative AI models as "black box recommendation engines," lacking a programmatic construction and optimization process for prompt statements on the server side. They do not explicitly embed health constraint parameters, time constraint parameters, preference constraint parameters, and object usage priority parameters in the prompt statements. Therefore, generative AI models cannot make full use of these structured constraints during inference, resulting in poor adaptability of menu generation to individual user differences. It is difficult to provide high-quality personalized menus while reducing food waste and resource consumption.

[0056] Third, many menu generation systems do not implement automatic evaluation and filtering mechanisms based on usage period information and historical selection records on the server side. Instead, they directly return the raw results output by the generative artificial intelligence model to the terminal, leaving users to manually judge their feasibility. This approach not only increases the user's workload but also lacks a reusable and scalable evaluation and filtering algorithm component within the computer system, failing to systematically reduce food waste and resource consumption from a computer technology perspective.

[0057] Fourth, from the perspective of computer system architecture, existing solutions often simply overlay a generative artificial intelligence interface call on top of the traditional application layer, lacking optimized design for the integrated pipeline of "prompt statement construction - model call - secondary result filtering". As a result, information processing devices cannot fully utilize the potential of generative artificial intelligence models in multi-constraint, multi-objective optimization problems, and it is also difficult to reflect substantial improvements to the computer data processing flow.

[0058] Therefore, it is necessary to provide a new server-based data processing system and method. By introducing functional components such as a condition information receiving unit, an object information acquisition unit, a prompt statement generation unit, a menu information acquisition unit, and a menu information selection unit into the server, an end-to-end computer processing flow is constructed, from multi-source data acquisition, condition parsing, fine construction of prompt statements to automatic result filtering. This improves the accuracy, efficiency, and resource utilization optimization capabilities of menu generation and processing at the computer technology level.

[0059] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 1 is achieved by the following means.

[0060] In this invention, the server includes a condition information receiving unit, an object information acquisition unit, a prompt statement generation unit, a menu information acquisition unit, and a menu information selection unit. This allows for unified modeling and phased processing of user condition information, storage container object information, and historical selection record information within the information processing device. By programmatically constructing prompt statements containing health conditions, time conditions, preference conditions, and object usage priority conditions, multi-source constraint information is densely encoded into the input of the generative artificial intelligence model. After obtaining multiple menu results, the server automatically performs evaluation and filtering based on usage period and resource consumption, thereby achieving personalized, low-waste, and high-resource-utilization-efficiency menu generation through an improved computer data processing flow.

[0061] "Information processing device" refers to an electronic device with a processor, memory and communication interface, which is a hardware platform used to execute programs and perform calculations and control on input data. It may include servers, terminal devices or combinations thereof.

[0062] "Information input / output unit" refers to an interface component on an information processing device used to receive user input information and output information to the user. It may include hardware such as touch screen, display, keyboard, microphone, speaker, network interface and user interface software that work with it.

[0063] "Conditional information" refers to the constraints and requirements data related to cooking or menu generation that are input by the user through the information input / output section, including but not limited to health conditions, time conditions, ingredient preference conditions, allergy restrictions, and free text descriptions.

[0064] "Storage container" refers to a physical container or device used to store objects (such as food or items) and capable of working in conjunction with detection elements. It may include refrigeration equipment, storage boxes, warehousing equipment, etc.

[0065] "Object information" refers to data related to objects within a storage container, including object type information, quantity information, usage period information, storage location, and other attributes that can be used for menu generation and resource management.

[0066] "Object category information" refers to classification data used to identify the category of an object, such as the major category to which the food belongs (vegetables, meat, grains, etc.), general product name, or unified code, etc.

[0067] "Usage period information" refers to time-limited data related to the availability of an object, including shelf life, best-before date, expiration date, or recommended usage period estimated based on sensor detection.

[0068] "Detection element" refers to a sensing component used to acquire information about objects within a storage container, including but not limited to image sensors, weight sensors, temperature and humidity sensors, radio frequency identification readers, and environmental monitoring sensors.

[0069] "Historical selection record information" refers to the data set stored in the information processing device that reflects the user's past selection behavior of menus or dishes, including menus used by the user, collection records, rejection records, and evaluation results.

[0070] "Generative artificial intelligence models" refer to artificial intelligence models that are based on machine learning and deep learning technologies and can automatically generate text output based on input prompts, especially models that use natural language processing structures to reason about input conditions and output menu solutions.

[0071] "Prompt statements" refer to text or text sequences constructed by the server and input into the generative artificial intelligence model. They are used to trigger the reasoning process within the model and constrain the generated results, so that the generated content meets the comprehensive requirements of conditional information and object information.

[0072] The "prompt statement generation unit" refers to a functional module in an information processing device that, through program execution, parses, combines, and formats conditional and object information to construct the prompt statements required for a generative artificial intelligence model.

[0073] "Menu information" refers to a structured or semi-structured data set containing one or more cooking schemes, generated by a generative artificial intelligence model based on prompts and obtained by the server. This data includes information such as the dish name, main ingredients, cooking steps, and required cooking time.

[0074] The "menu information acquisition unit" refers to a functional module in an information processing device used to send prompts to generative artificial intelligence models and receive the menu information they generate, including the model call interface and the logic for parsing the returned results.

[0075] The “menu information selection unit” refers to a functional module in an information processing device that evaluates and filters multiple candidate menu information based on usage period information, historical selection record information, and resource consumption indicators, in order to select the menu information that meets the predetermined optimization objectives.

[0076] The "menu information presentation unit" refers to a functional module that provides the selected menu information to the user in a visual or auditory form through the information input / output unit, including graphical interface display, voice broadcast or other output methods.

[0077] "Health conditions" refer to the constraint parameters or preference information related to health, such as nutrition, energy intake, fat and salt control, in the condition information, which are used to guide generative artificial intelligence models to generate menu schemes with health orientations.

[0078] "Time conditions" refers to the constraint parameters in the conditions information that are related to cooking time, preparation time, or overall completion time, and are used to limit the estimated time of the menu plan.

[0079] "Preference conditions" refer to conditional information that reflects a user's taste preferences, commonly used ingredients, foods to avoid, and dietary habits. This includes preferences actively entered by the user and preferences inferred from historical selection records.

[0080] "Object usage priority conditions" refers to the constraint information used to determine the priority order of each object in the menu, which is inferred based on the object's usage period, storage duration, inventory status, etc.

[0081] "Cooking suggestion information" refers to a set of text content output by a generative artificial intelligence model after receiving a prompt statement, which is used to guide actual cooking operations. This includes suggested dish combinations, ingredient pairings, and step instructions.

[0082] "Resource consumption" refers to the degree of physical resources and energy used in the process of generating and executing menu schemes, including quantifiable or estimable consumption indicators such as electricity consumption, gas consumption, and water resource usage.

[0083] The embodiments of the present invention will be described in detail with reference to the system configuration defined in the claims, and will focus on the specific data structure, algorithm flow and generative artificial intelligence model structure and learning method within the information processing device, so that those skilled in the art can implement the present invention accordingly.

[0084] I. Overall System Composition As the core node of an information processing device, the server can be deployed on general-purpose computing equipment in a data center, such as a rack server equipped with a central processing unit (CPU) for general-purpose computing and a graphics processing unit (GPU) for deep learning inference. The server can run the various functional modules of this invention using a general-purpose operating system (such as a Unix-based server operating system) and general-purpose middleware (such as web server software and application server software).

[0085] A terminal can be a smartphone, tablet, laptop, or desktop computer, running a mobile or desktop operating system. It includes hardware such as a display, touch input or keyboard, microphone, speaker, and network communication unit. The applications running on the terminal can be native applications or web applications running in a browser.

[0086] Users input cooking-related information through the terminal's user interface and view the menu information generated and returned by the server. The server exchanges structured data with the terminal via a network connection (such as HTTPS communication based on the TCP / IP protocol).

[0087] II. Internal Module Composition and Data Structure of the Server 1. Conditional Information Receiving Unit The server includes a conditional information receiving unit. At the application layer, the server uses a network framework (such as a web application framework) to parse requests from the terminal, converting the text data in the request body into internal data structures.

[0088] The server represents condition information uniformly as condition objects. A condition object can include key-value pairs of the following fields (abstracted into a higher-level data structure): - Health Conditions field: Indicates constraints on fat, salt, sugar, energy, etc.; - Time Conditions field: Indicates the maximum acceptable cooking time; - Preference criteria field: Indicates the categories of food that the user likes or dislikes; - Meal type field: indicates a general category such as breakfast, lunch, or dinner; - Free text description field: Represents other requirements described by the user in natural language.

[0089] The server stores condition objects in memory in an associated container (such as a hash map or dictionary structure) and can attach metadata such as timestamps and user identifiers.

[0090] 2. Object Information Acquisition Unit The server includes an object information acquisition unit. The server acquires information about objects within the storage container through an interface program that works in conjunction with detection elements on the storage container side. Detection elements may include image sensors, weight sensors, RFID readers, etc. The detection elements send raw signals to a local control device, which then transmits them to the server via the network.

[0091] The server represents object information as a collection of object records. Each object record contains: - Object category identifier: Encodes abstract categories (such as "meat" or "vegetables"); - General name identifier: The common name of the ingredient; - Usage period information: including expiration date or remaining days; - Storage location labeling: such as refrigerator compartment, freezer compartment, and other general location categories; - Quantity or weight information: expressed in uniform units.

[0092] The server stores multiple object records in a database (such as a relational database or a document database) and loads them into a list or tabular data structure in memory at runtime for subsequent analysis.

[0093] 3. Acquisition and Modeling of Historical Selection Record Information The server can further store and manage users' historical selection records. The server generates a historical record entry for each user's selection of menu items or dishes. These entries include: - User ID; - General labeling of dishes; - Behavior category (e.g., accept, save, reject); - Timestamp of occurrence.

[0094] The server can calculate user preference feature vectors in memory using statistical functions. For example, the server can count the number of times users accept or reject each food category, and normalize these counts into preference weights, thus forming a "category-weight" mapping structure.

[0095] III. Collaboration between the prompt statement generation unit and the generative artificial intelligence model 1. Data processing of the prompt statement generation unit The server includes a prompt statement generation unit. The server performs a series of text processing and rule operations on the CPU, integrating condition objects, object information, and historical selection record information into prompt statements.

[0096] The server first parses the health condition, time condition, and preference condition in the condition object. For example: - If the health condition is "low fat, low salt", the server maps it to a description of constraints on fat and sodium intake.

[0097] - If the time condition is "within 20 minutes", the server will convert it into an explicit time constraint in the text.

[0098] - If preferences and history show that a user has repeatedly accepted menus containing chicken and leafy greens and repeatedly rejected menus containing a certain type of herb, the server encodes this as a description of "preferring certain ingredients" and "avoiding certain ingredients".

[0099] The server then prioritizes the object information. For example, by comparing the current date and usage period information, the server calculates the "remaining days" for each object and maps it to a priority level (such as high, medium, or low). The server categorizes high-priority objects into "food categories that should be consumed first" and reflects this in the prompt message.

[0100] The server uses a template structure when constructing the prompt statement, embedding the above health conditions, time conditions, preference conditions, and object usage priority conditions into a complete description in natural language.

[0101] For example, the prompt message generated by the server can be: Please recommend a dinner menu for a user based on the following criteria: 1. Dishes should be as healthy as possible, with less oil and salt; 2. The dishes should be suitable for children's tastes, not too spicy, and should not use ingredients that are difficult to chew; 3. The overall cooking time should be kept within 20 minutes; 4. Prioritize using vegetables and poultry that need to be consumed as soon as possible; 5. Avoid using strongly scented ingredients that users have repeatedly rejected.

[0102] Please provide three suitable dishes. Each dish should include: the name of the dish, its main ingredients, a brief description of the cooking steps (no more than five steps), and the approximate cooking time. The server can modify the number of lines and expression of the template according to different implementation forms, but the overall structure follows the design principle of explicitly embedding multiple constraint parameters into the prompt statement.

[0103] 2. Structure and Learning Methods of Generative Artificial Intelligence Models The server retrieves unit calls to the generative artificial intelligence model through menu information. The generative artificial intelligence model can be a language model based on a transformer structure. During the training phase, this model processes large-scale corpora through multiple encoding and decoding layers, each consisting of a multi-head attention sublayer and a feedforward network sublayer.

[0104] The model takes as input a sequence of characters representing prompts generated by the server, which is then segmented and embedded to form a vector sequence. The model uses a self-attention mechanism to weightedly sum the relationships between words, generating an internal context representation. During the decoding phase, the model progressively generates the next word in the output statement using an autoregressive approach, outputting a menu text containing multiple dishes.

[0105] The server can use supervised learning methods during the model training phase. It selects a large number of "condition description-menu scheme" pairs as training samples, uses cross-entropy as the loss function, and updates the model parameters through backpropagation. The server can iteratively reduce the loss value using optimization algorithms (such as gradient-based general optimization methods).

[0106] To enhance the model's sensitivity to health conditions, time conditions, and food categories, the server explicitly labels this information in the training data, making it appear in the prompt paragraphs and enabling the model to learn how to reflect constraints in generated paragraphs. The server can also expand the training samples and improve the model's robustness to diverse expressions through data augmentation methods, such as changing the order of conditions or adding synonyms.

[0107] After the model parameters are trained, the server performs forward propagation computation during the inference phase by calling the deployed inference engine (which can run on the GPU), and no further parameter updates are performed. The server can be configured to generate parameters, for example: - Maximum generation length; - Sampling temperature; - Rules to suppress repetition (e.g., based on a repetition penalty coefficient).

[0108] Through the specific structure and parameter control described above, the server can achieve stable output quality in menu generation tasks.

[0109] IV. Algorithm Processing for Menu Information Acquisition and Selection 1. Parsing and processing of menu information acquisition unit The server receives text output from the generative artificial intelligence model through a menu information retrieval unit. This text typically contains descriptions of multiple dishes, which the server converts into structured menu information using natural language parsing algorithms.

[0110] The server can use rule-based segmentation algorithms to split the text according to "serial number identifiers" or "dish name keywords". For each dish description, the server further parses fields such as dish name, main ingredients, and steps. For example, the server may segment the text using keywords such as "dish name", "main ingredients", "steps", and "approximate time".

[0111] The server stores the parsed results as a list of menu records, each of which includes: - Dish name field; - Ingredient list field; - Step list field; - Expected time field (extracted if described in the text, otherwise estimated by the server).

[0112] 2. Scoring and filtering of menu information selection units The server includes a menu information selection unit. The server calculates a score for each menu item in memory. The score consists of multiple metrics, such as: - Usage Expiration Priority Score: Bonus points are awarded based on whether the ingredients in the menu include high-priority objects; - Health Match Score: A score is calculated based on the degree of match between the fat, salt, and other conditions and the ingredients and cooking methods; - Time Matching Score: A score is calculated based on whether the estimated time is lower than the user-specified time limit; - Preference matching score: The score is increased or decreased based on whether the ingredients belong to the user's preferred category or the category rejected by the user.

[0113] The server can combine the scores from the above items into a total score using a weighted method. The weights can be preset or estimated from historical data through statistical learning. The server then sorts the multiple menu records according to the total score and selects the top few as the final output menu information.

[0114] By performing the above scoring and filtering on the server side, the present invention can effectively filter out menus that do not meet the priority of usage period or resource conservation goals, so that the results received by the terminal have been technically optimized.

[0115] V. Specific Examples 1. Examples of simple and healthy dinners The user enters the following conditions through the terminal: - For health: less oil and less salt are needed; - Suitable for children: Yes; - Time: Within 20 minutes; - Free text description: "I hope there will be more vegetables and poultry, and that it won't be too spicy."

[0116] After integrating the condition object, object information, and history, the server generates the following prompt statement: Please recommend a dinner menu for a user based on the following criteria: 1. Dishes should be as healthy as possible, with less oil and salt; 2. The dishes should be suitable for children's tastes, not too spicy, and should not use ingredients that are difficult to chew; 3. The overall cooking time should be kept within 20 minutes; 4. Prioritize using vegetables and poultry that need to be consumed as soon as possible; 5. Avoid using ingredients with strong odors that users have repeatedly rejected.

[0117] Please provide three suitable dishes. Each dish should include: the name of the dish, its main ingredients, a brief description of the cooking steps (no more than five steps), and the approximate cooking time. The server calls a generative artificial intelligence model to obtain the menu text containing 3 dishes, and selects the dishes with the highest total scores through parsing and scoring, and then returns it to the terminal for display.

[0118] 2. Examples of budget control and resource conservation The user enters a free text description in the terminal: "I want an affordable dinner, and I'll try to use up the food in the fridge that's about to expire." and specifies a cooking time of no more than 30 minutes.

[0119] Based on object information, the server detects that some vegetables and eggs are nearing their expiration date. Therefore, it adds a requirement to "prioritize consuming vegetables and eggs nearing their expiration date" to the generated prompt. The generative AI model then outputs a menu based on these ingredients. The server further scores the results based on usage period information and energy consumption estimates, selecting a menu that meets both health and time requirements while minimizing food waste.

[0120] VI. Technical Effects and Causal Relationships Through the specific module division and data processing structure described above, this invention does not simply replace human thought with a computer to think about menus. Instead, it constructs a technical pipeline within the server that unifies modeling of multi-source data, constraint fusion, prompt statement generation, deep model reasoning, and secondary result filtering.

[0121] The server maps conditional information, object information, and historical selection records into a unified data structure, and calculates object usage priority and preference features through rules and statistics, giving the prompts higher information density for the generative AI model. Because the model receives finely crafted prompts, its internal attention mechanism can more effectively focus on features related to the constraints, thereby improving the matching degree of the generated results in terms of health, timeliness, and resource utilization.

[0122] Meanwhile, the server's scoring and filtering process after generation treats the model output as a candidate set. Through a multi-index evaluation algorithm within the computer, the model is further processed, effectively reducing bias caused by model randomness and resulting in menus with more significant advantages in terms of lifespan and energy efficiency. This dual mechanism of "refined prompt construction + technical result filtering" fundamentally improves the accuracy and efficiency of the entire menu generation data processing flow.

[0123] Furthermore, by centrally processing user preferences and object information on a server, redundant calculations between terminals can be reduced, improving the overall system's computational efficiency. The server's structured management of object information and user preferences facilitates caching and reusing computation results, thereby improving response speed and reducing communication load.

[0124] VII. Alternative Implementation Methods In other implementations, the server can employ different generative artificial intelligence model structures. For example, it can add an auxiliary encoder for conditional encoding to the basic transformer structure, inputting health conditions, time conditions, etc., into the model in vector form. The server can also employ a multi-task learning approach, enabling the model to simultaneously learn "recipe recommendation" and "nutritional estimation" tasks to further improve the rationality of the output.

[0125] Regarding the generation of prompt statements, the server can employ a learnable template mechanism. This involves using a small neural network to automatically generate a prompt statement framework based on the conditional object, with the rule module then supplementing the details. This approach can improve expressive diversity while maintaining controllability.

[0126] Regarding menu selection, the server can design a trainable scoring function, automatically adjusting the weights of various indicators through supervised learning using historical feedback data. The server can also introduce reinforcement learning-based strategies to optimize menu selection based on long-term user satisfaction.

[0127] Through the above-mentioned various implementation forms, the present invention can achieve technical optimization of the menu generation task in different system scales and application environments, reflecting the improvement of the internal data processing structure and algorithm flow of the computer.

[0128] use Figure 11 The processing flow is explained.

[0129] Step 1: The user enters conditional information on the terminal. In the application interface on the terminal, users input conditional information related to menu generation through controls such as text input boxes, checkboxes, and drop-down lists. Input includes: free text descriptions (e.g., "Want a healthy and child-friendly dinner, ready in 20 minutes"), health options, time limits, and preference options.

[0130] The terminal collects these inputs in an event-driven manner and combines the current values ​​of each control into a condition object. The terminal stores the condition object in key-value pairs in local memory, which may contain fields such as "health condition", "time condition", "preference condition", and "free text".

[0131] The input for this step is the user's interaction events on the interface, and the output is a structured conditional object stored internally by the terminal. Based on the user's input, the terminal converts text data and option states into a unified data structure, preparing for subsequent network transmission and server-side computation.

[0132] Step 2: The terminal sends a condition object to the server. The terminal reads the condition object from local memory and uses the serialization module to encode the object into a network transmission format (such as JSON text). The terminal constructs a network request containing this JSON text and adds metadata such as user identifier and authentication information to the request header.

[0133] The terminal sends the request to the application interface address specified by the server via the network communication interface using the HTTPS protocol. Before sending, the terminal can initiate a loading animation to inform the user that the request is currently being processed.

[0134] The input for this step is a conditional object within the terminal, and the output is a conditional datagram transmitted across the network to the server. The terminal converts the data object in memory into a byte stream that can be transmitted over the network through serialization and protocol encapsulation.

[0135] Step 3: The server receives and parses the conditional data. The server receives network requests from the terminal at the application layer, calls the parsing functions provided by the framework, and extracts JSON text from the request body. The server then uses a parsing library to deserialize the JSON text into a conditional object internal to the server.

[0136] During the parsing process, the server checks the field types, fills in missing fields with default values, and sets invalid values ​​to predefined safe values. The server caches the parsing results in the session context or request context for subsequent module calls.

[0137] The input to this step is the conditional data message received by the network layer, and the output is a normalized conditional object running in the server's memory. The server converts the original text data into structured data that can directly participate in logical calculations through deserialization and verification.

[0138] Step 4: Server retrieves or updates object information The server invokes the object information retrieval unit to pull the latest list of object information from the subsystem connected to the storage container (such as the gateway or intermediate service of the refrigeration unit). The object information includes fields such as object category, general name, usage period, and quantity.

[0139] The server filters and deduplicates the retrieved list of object records, removing records missing core fields, and calculates the remaining time until the expiration date for each object based on the usage period field. The server stores these object records in an in-memory table structure and can synchronously update the corresponding entries in the backend database.

[0140] The input for this step is raw object information from the detection element and its gateway, and the output is a set of object records within the server, along with derived data such as the remaining days appended to each object. The server processes the raw detection data into structured resource information that can be used for priority calculation through field parsing and time calculation operations.

[0141] Step 5: Server generates user preference features Based on the user's identifier, the server queries the historical database for the user's menu selection history. The server reads multiple records, each containing information such as dish identifier, behavior type (accept, favorite, reject), and timestamp.

[0142] The server performs statistical operations on these historical records, such as counting the number of adoptions and rejections by ingredient category, and normalizes these counts to obtain a preference score for each category. The server then organizes these scores into preference feature vectors or category-weight mapping tables.

[0143] The input for this step is a historical selection record dataset, and the output is a feature structure representing user preferences. The server extracts preference parameters that can be directly used for constraints and ranking from discrete behavior records through data operations such as aggregation, counting, and normalization.

[0144] Step 6: Server computed objects use priority The server iterates through the collection of object records, calculating the remaining days based on the usage period and current time in each record. The server then maps the remaining days to priority levels according to preset rules, such as "0-1 days" being the highest priority, "2-3 days" being the medium priority, and "more than 3 days" being the low priority.

[0145] The server aggregates high-priority objects by object category. For example, it identifies which major categories (vegetables, poultry, etc.) appear frequently in the high-priority set and marks them as "priority consumption categories." The server stores these category labels in the object priority result structure.

[0146] The input to this step is a set of object records containing usage period information, and the output is a priority level for each object and a list of priority consumption categories summarized by type. The server converts the time-dimensional information into priority conditions that can be embedded in prompt statements through time difference calculations and category aggregation.

[0147] Step 7: Server construct prompt statement The server invokes the prompt statement generation unit, taking the normalized condition object, user preference features, and object priority results as input. The server first generates basic constraint text based on the condition object, such as health constraints, time constraints, and descriptions of the applicable population.

[0148] The server then inserts descriptions of "preferred food categories" and "food categories to avoid" into the text based on user preference characteristics. Subsequently, based on the object priority results, the server adds a constraint to the text: "prioritize consuming objects that are about to expire."

[0149] The server combines the above text fragments according to a predefined template to form a complete prompt statement. For example: Please recommend a dinner menu for a user based on the following criteria: 1. Dishes should be as healthy as possible, with less oil and salt; 2. The dishes should be suitable for children's tastes, not too spicy, and should not use ingredients that are difficult to chew; 3. The overall cooking time should be kept within 20 minutes; 4. Prioritize using vegetables and poultry that need to be consumed as soon as possible; 5. Avoid using ingredients with strong odors that users have repeatedly rejected.

[0150] Please provide three suitable dishes. Each dish should include: the name of the dish, its main ingredients, a brief description of the cooking steps (no more than five steps), and the approximate cooking time. The input for this step consists of conditional objects, preference feature structures, and object priority structures. The output is a natural language prompt that can be directly used by generative AI models. The server transforms multi-source structured data into high-information-density text input through string concatenation, template filling, and rule insertion.

[0151] Step 8: The server sends prompts to the generative AI model and performs inference. The server retrieves the generated prompt statement from the menu information unit and encapsulates it into the model call request, setting inference parameters such as model name, maximum output length, and temperature. The server then sends the request to the inference service that has deployed the generative artificial intelligence model via the model interface module.

[0152] The generative AI model receives prompts in the inference service, first converting the character sequence into a word sequence through a tokenizer, and then mapping it into a vector sequence through an embedding layer. The model performs matrix multiplication and nonlinear transformations in a multi-layer self-attention network to progressively compute the contextual representation, and then autoregressively generates the menu text during the decoding stage.

[0153] The server receives the generated result text returned by the inference service and caches it in memory.

[0154] The input for this step is the prompt statement and inference parameters constructed by the server, and the output is a multi-course menu text generated by a generative artificial intelligence model. The server uses remote calls and forward propagation computation to restore the high-dimensional vector operation results into candidate menus in natural language description form.

[0155] Step 9: The server parses and generates text as structured menu information. The server parses the menu text returned by the generative artificial intelligence model. Using a rule-based text segmentation algorithm, the server divides the long text into multiple segments based on sequence numbers, line breaks, or key phrases, with each segment corresponding to a description of a dish.

[0156] The server further identifies fields such as "dish name," "main ingredients," "steps," and "approximate time" within each segment. The server uses string lookups and regular expression matching to extract the content after key phrases, and then splits the step section into a list of steps by row or sequence number.

[0157] The input for this step is unstructured menu generation text, and the output is a list of menu records. Each menu record contains fields such as dish name, ingredient list, step list, and time estimate. The server uses text parsing and field extraction to transform the natural language output back into structured data that can be used for numerical scoring and logical judgment.

[0158] Step 10: The server scores menu records based on multiple metrics. The server calls the menu information selection unit to calculate a multi-dimensional score for each menu record. The server compares the ingredient categories appearing in the menu with the object priority results, and adds a "usage period priority score" to dishes that use high-priority objects.

[0159] The server compares the cooking methods and ingredient types in the dish description with health requirements. For example, it deducts points for dishes containing fried ingredients or high-fat ingredients, and adds points for steamed, boiled, or low-oil dishes. The server also compares the estimated time field with the user's time requirements, deducting points if the time exceeds the limit.

[0160] Meanwhile, based on user preference characteristics, the server adds points to dishes that use ingredients the user prefers and deducts points from dishes that contain ingredients the user frequently rejects. Finally, the server calculates the overall score for each menu record using a weighted summation method, combining the individual scores according to predetermined weights.

[0161] The input for this step is a structured list of menu records, object priority structure, health and time conditions, and preference features. The output is a set of menu records with a comprehensive score. The server quantifies the semantic constraints into sortable numerical indicators through comparison, conditional judgment, and weighted calculation.

[0162] Step 11: The server selects the target menu and generates response data. The server sorts the menu records in descending order based on their overall scores and selects the highest-scoring records as recommended menu items. The server then converts the selected menu records into a response data structure, removing intermediate calculated fields and retaining only the dish name, main ingredients, cooking steps, and time information required for terminal display.

[0163] The server serializes the response data into a transmission format (such as JSON text) and marks the status code and content type in the response header.

[0164] The input for this step is a set of menu records with comprehensive scores, and the output is simplified menu response data for the terminal. The server sorts and filters the candidate set to narrow it down to a target set that satisfies the multi-objective optimization results.

[0165] Step 12: The terminal receives and parses the menu response data. The terminal receives the response message returned by the server from the network interface and reads the JSON text within it. The terminal then calls a parsing library to deserialize the JSON text into a list of local menu objects.

[0166] The terminal stores these menu objects in memory and maps the fields to the list item data model according to the local interface layout requirements, in preparation for subsequent graphics rendering.

[0167] The input for this step is the menu response data message sent by the server, and the output is a list of menu objects within the terminal. The terminal uses deserialization to restore the network transmission result into a data structure that can directly drive the interface display.

[0168] Step 13: The terminal presents the menu to the user and receives feedback. The terminal updates the interface display based on the menu item list. In the list view, the terminal displays the name and brief description of each dish; when the user clicks on a dish, the terminal displays a complete list of main ingredients and steps in the details view.

[0169] Users can browse the menu on the terminal interface, select a dish to start cooking, or perform actions such as adding a dish to their favorites or removing it from recommendations. The terminal captures these action events, generates new action objects (e.g., "add to favorites" or "reject a dish"), and can subsequently send these action objects to the server to update the historical selection record.

[0170] The input for this step is the list of menu objects within the terminal and user operation events. The output is a user-visible menu interface and optional behavioral feedback data. The terminal, through interface rendering and event listening, transforms the server-side calculation results into an interactive user interface and provides behavioral input for the next round of personalized optimization.

[0171] Application Example 1 The process flow corresponding to the specific processing in Use Case 1 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. Furthermore, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0172] With the development of menu recommendation technology based on generative artificial intelligence models, although related systems can generate menu options based on users' natural language preferences, the following technical problems still exist in practical applications: First, in traditional solutions, servers typically input simple user preference information directly into generative AI models as natural language. This lacks structured processing and unified modeling of food data, shelf-life data, and multidimensional user constraints (health status, budget, time, allergies, etc.) from sensing devices. As a result, there is a discrepancy between the generated results and the user's actual constraints, and the overall data processing chain of the system has not been effectively optimized.

[0173] Second, existing technologies mostly use fixed prompt templates to call generative artificial intelligence models. The servers lack the ability to automatically adjust the content of prompts based on dynamic information such as inventory status, shelf life, and user history. They cannot guide the model to prioritize the use of near-expiry ingredients, control energy intake, and meet complex constraints such as delivery time during the generation process. As a result, it is difficult to achieve comprehensive optimization of food waste, electricity consumption, and costs at the computational level.

[0174] Third, server-side processing of the output of generative AI models often remains at the level of simple display, without fine-grained structured analysis and algorithmic processing of the output text. For example, there is a lack of efficient comparison algorithms and differential calculation mechanisms between "the required ingredients output by the model" and "standardized inventory ingredients collected by sensors," and a lack of automatic calculation modules based on multiple supply points and multiple delivery routes. Therefore, it is impossible to automatically generate executable order data and delivery decisions in the same calculation process, resulting in a fragmented processing chain from menu generation to order placement and delivery, with many redundant calculations and long response times.

[0175] Fourth, in terms of order generation and delivery coordination, traditional e-commerce or food delivery systems typically separate menu recommendation, inventory management, and delivery optimization. The servers do not form an integrated computing path in terms of software architecture and data flow design, which leads to the need for multiple cross-system queries and manual intervention during the order confirmation stage. This is not conducive to achieving unified algorithm optimization of cost constraints, delivery time constraints, and energy consumption constraints within a single information processing platform, thus limiting the overall system performance and scalability.

[0176] Therefore, a new information processing system is needed that integrates "condition collection, prompt generation, model invocation, result parsing, differential calculation, and order and delivery optimization." This system improves the efficiency, accuracy, and resource utilization of menu recommendation and order processing by introducing a dynamic prompt generation mechanism for generative artificial intelligence models, a differential calculation mechanism based on standardized ingredient data, and a supply and delivery optimization mechanism under multiple constraints on the server side.

[0177] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is achieved by the following means.

[0178] In this invention, the server includes a module for acquiring user condition data, including health status, object attributes, time conditions, cost conditions, and allergy information, from the user via an information input / output device; a module for acquiring food information data, including food type, quantity, and shelf life, from a detection device installed in a storage device and standardizing its units and names to generate standardized food data; a module for generating prompt statements based on the user condition data and the standardized food data, wherein the prompt statements include a condition description in natural language, a food description in natural language, and an instruction description for specifying the output format of the generative artificial intelligence model; and a module for inputting the prompt statements into the generative artificial intelligence model to obtain menu candidate information and extracting the necessary information for each menu item. The system includes several modules: a module for comparing required ingredient data with standardized ingredient data to determine sufficient and insufficient ingredients, and a module for calculating candidate items and recommended quantities based on a product information database; a module for generating menu candidate screen data for display on the user terminal based on menu candidate information, insufficient ingredients, and recommended quantities, and receiving user selections and quantity changes to generate order data; and a module for calculating delivery time, cost, and energy consumption based on receiving information, time conditions, and purchased ingredient information contained in the order data, utilizing multiple supply points and delivery route information to determine supply points and delivery conditions that meet predetermined time and evaluation conditions, storing order information in an order information database, and sending order data to the delivery service device. This allows for the structured and automated data processing before and after the generative artificial intelligence model within a single server-side computing architecture. By dynamically constructing prompts to precisely constrain model output, and then using standardized ingredient data and supply and delivery information for unified calculations after output parsing, it achieves integrated processing of menu generation, inventory utilization optimization, cost and energy consumption control, and delivery route optimization. This improves the data processing efficiency, resource utilization, and overall performance of the computer system in menu recommendation and order processing scenarios.

[0179] "System" refers to a combination of hardware and software consisting of at least one information processing device, storage device, detection device, user terminal and delivery service device interconnected through a communication network, and working together to perform menu generation, order generation and delivery optimization processing.

[0180] "Information processing device" refers to an electronic computing device with a processor and memory, such as a server or computer, used to execute programs to process and control user condition data, ingredient information data, menu candidate information and order data.

[0181] "Information input / output device" refers to a human-computer interface device used for data interaction between a user and an information processing device, such as a touch screen, keyboard, display, microphone, or speaker.

[0182] "User condition data" refers to various constraint information related to menu generation and order generation obtained from users through information input / output devices, including but not limited to health status, object attributes, time conditions, cost conditions, and allergy information.

[0183] "Health status" refers to information used to indicate a user's needs or restrictions in areas such as nutrition, chronic diseases, and weight control, such as preferences for low salt, low fat, controlled sugar, or high protein.

[0184] "Object attributes" refer to information used to represent the characteristics of the menu service object, such as categories like children, the elderly, athletes, or specific occupational groups.

[0185] "Time conditions" refers to constraints related to menu preparation time or delivery completion time, such as conditions for cooking to be completed within a predetermined number of minutes or delivery to be completed within a predetermined time.

[0186] "Fee conditions" refers to the upper limit or range of fees that a user can accept in a single meal or order, and other expenditure-related constraints.

[0187] "Allergy information" refers to restrictive information used to indicate that a user has an allergy or intolerance to a specific food or ingredient, such as an allergy to nuts, dairy products, or shellfish.

[0188] "Storage equipment" refers to physical devices used for storing food ingredients, such as refrigeration equipment, freezing equipment, or room temperature storage equipment.

[0189] "Detection device" refers to a sensing component installed in a storage device to obtain information related to food ingredients, such as a weight sensor, image sensor, radio frequency identification device, or barcode scanning device.

[0190] "Food information data" refers to data acquired by the detection device and recorded by the information processing device to indicate the status of food in the storage device, including at least the type, quantity and shelf life of the food.

[0191] "Ingredient category" refers to the labeling information used to distinguish different food or ingredient categories, such as the specific names of vegetables, meats, grains, or seasonings.

[0192] "Quantity" refers to measurement information related to the physical quantity of ingredients, such as mass, volume, quantity, or number of servings.

[0193] "Shelf life" refers to the date or time information used to indicate the deadline for recommending or ensuring the safe consumption of food.

[0194] "Standardized food data" refers to food data that is recorded using a unified format and unit of measurement, obtained by standardizing the units, names, and representation methods of the original food information data.

[0195] "Prompt statements" refer to natural language text generated by an information processing device and input into a generative artificial intelligence model. This text describes user conditions, stored ingredients, and model output format requirements to guide the model in generating menu candidate information that meets the constraints.

[0196] "Condition description" refers to the user condition data expressed in natural language in the prompt statement, which is used to specify constraints such as health, object, time, cost, and allergies to the generative artificial intelligence model.

[0197] "Ingredient description" refers to the standardized ingredient data expressed in natural language in the prompt statement, which is used to explain to the generative artificial intelligence model the available ingredients that can be prioritized.

[0198] "Instruction description" refers to the natural language instruction part in the prompt statement used to specify the requirements for the structure, format, or quantity of the output content of the generative artificial intelligence model.

[0199] "Generative artificial intelligence model" refers to an artificial intelligence model trained based on deep learning technology that can automatically generate text-based menu candidate information based on input prompts.

[0200] "Menu candidate information" refers to text or structured data containing one or more menu options, output by the generative artificial intelligence model based on prompts, including at least the dish name, required ingredients, and cooking-related information.

[0201] "Required ingredient data" refers to the data extracted from the menu candidate information, which represents the specific types and quantities of ingredients needed to generate each menu.

[0202] "Meeting requirements" refers to ingredients whose demand can be fully covered by the existing food inventory in the storage device.

[0203] "Insufficient ingredients" refers to ingredients whose required quantity exceeds the existing food inventory in the storage device, thus requiring additional purchases.

[0204] The "product information database" refers to a data set that records information such as available products, their specifications, packaging quantities, and prices, and is used to match insufficient ingredients with actual available products.

[0205] "Purchase candidate items" refers to one or more candidate products that can be used to supplement the missing ingredients after matching the missing ingredients with the product information database.

[0206] "Recommended purchase quantity" refers to the quantity of goods that is suggested to be purchased based on the insufficient demand for ingredients and the corresponding product packaging specifications.

[0207] "Menu candidate screen data" refers to structured display data used to display menu candidate schemes on the user terminal, including information such as menu name, ingredients that have been satisfied, ingredients that are in short supply, and recommended purchase quantity.

[0208] "User terminal" refers to an electronic device operated by a user and communicating with an information processing device, such as a smart terminal or computing terminal, used to display menu candidate screens and receive user input.

[0209] "Order data" refers to structured data generated by an information processing device after a user confirms the menu and the quantity of ingredients purchased, which represents a transaction request. This data includes delivery information, time conditions, and information about the purchased ingredients.

[0210] "Shipping information" refers to data such as address, contact person, and contact information related to the destination of the order.

[0211] "Supply locations" refer to physical locations used to provide services for purchasing and shipping food ingredients, such as warehouses, stores, or centralized distribution centers.

[0212] "Delivery route" refers to the combination of actual transportation routes that can be selected from the supply point to the delivery point, including transfer points, road segments and modes of transportation.

[0213] "Delivery time" refers to the time or an estimate thereof required from when an order is accepted until the purchased ingredients are delivered to the delivery location.

[0214] The "order information database" refers to a collection of stored data used to record and manage order data and its status changes.

[0215] "Delivery service device" refers to a computing device or system used to receive order data and perform delivery task management, route planning and status feedback.

[0216] "Cost conditions" refer to evaluation constraints used to limit or optimize the total expenditure amount during the order generation and supply location selection process.

[0217] "Energy consumption conditions" refer to evaluation constraints used to limit or optimize resource use, such as electricity or fuel consumption, during the selection of delivery routes and supply points.

[0218] "Evaluation value" refers to a numerical indicator calculated based on cost information, energy consumption information, and time information, used to comprehensively compare different supply locations and delivery conditions.

[0219] In one embodiment of the present invention, the server includes a processor, main memory, persistent storage device, and communication interface. The operating system running on the server can be a general-purpose server operating system, such as a UNIX kernel-based server operating system. The server runs application server software on this operating system, such as a web framework based on an interpreted language or an enterprise-level framework based on a virtual machine, to implement the various functional modules of the present invention. The server also establishes a secure network connection with the database system and the generative artificial intelligence model inference service through the communication interface.

[0220] In this embodiment, the terminal is a user-operated smart terminal or general-purpose computing terminal. The terminal can be a smart terminal running a mobile operating system or a personal computing terminal running a desktop operating system. The terminal runs the client application specific to this invention, or accesses a web interface based on Hypertext Markup Language via a browser. The terminal exchanges data with the server via wireless or wired communication.

[0221] In this embodiment, the storage device can be a household appliance or a commercial cold chain appliance with refrigeration function. The storage device contains a detection device, which includes at least one sensor module, such as a weight sensor module, an optical image sensor module, an RFID reader / writer module, and a barcode reader module. The detection device is connected to a local control unit via a wired bus or short-range wireless communication. The local control unit packages the collected food identification results and quantity measurement results into structured data and sends it to a server via a local area network or the Internet.

[0222] In this embodiment, during the initialization phase, the server loads multiple software modules into the main memory, including: a user condition management module, a food ingredient data standardization module, a prompt statement generation module, a generative artificial intelligence model interface module, a menu result parsing module, a food ingredient difference calculation module, an order generation module, a supply and delivery optimization module, and a log and monitoring module. The server also establishes a persistent connection with a relational database system, which can be deployed on a cloud database service to store user data, food ingredient data, product information, order information, and delivery information.

[0223] In this embodiment, the server uses a relational database to store structured data. The server stores user condition data in a "User Conditions Table," food ingredient information from storage devices in a "Food Ingredient Inventory Table," available product information in a "Product Information Table," and order records in an "Order Table" and an "Order Details Table." The server establishes foreign key relationships and index structures between these database tables to enable join queries and condition filtering with low time complexity in subsequent operations.

[0224] In this embodiment, the user operates a dedicated client application on a terminal. The terminal displays a conditional input interface containing multiple fields, such as dietary preferences, target population, delivery time restrictions, budget, and allergy information. The terminal converts the user's input in these fields into an internal data structure, serializes this data structure into text format, and sends it to the server via an encrypted communication channel.

[0225] In this embodiment, after receiving user condition data uploaded by the terminal, the server maps the data to an internal unified representation. The server sets standardized identifiers for each condition field, such as "health preference," "child-oriented," "delivery within 15 minutes," "budget per person not exceeding a certain value," and "prohibition of specific ingredients." The server uses these standardized identifiers to generate condition feature vectors for generative artificial intelligence models and then maps these features into natural language descriptions when generating prompts.

[0226] In this embodiment, the server reads the food records of the storage devices corresponding to the user from the food inventory table. The server standardizes the units and names in these records, for example, converting packaging units such as "package" and "bag" into mass units, and unifying the same food item represented by different texts with the same standard name. The server performs unit conversion by looking up preset conversion rules in the "Food Specification Table" and unifies names by looking up the "Food Synonym Table". The server uses the standardization results as "standardized food data", which is maintained in memory in a structured form for subsequent differential calculations and prompt statement generation.

[0227] In this embodiment, when the server constructs the prompt statement, it first converts the user's conditional data into a natural language conditional description. For example, the server can generate the following text fragment: "User requirements: healthy, child-friendly, delivery within 15 minutes, budget not exceeding 1000 yen per person."

[0228] If you are allergic to peanuts, please completely avoid peanuts and other nuts. The server also converts standardized ingredient data into natural language descriptions of the ingredients. For example, the server can generate the following text snippet: "The refrigerator already contains the following ingredients: 300g chicken breast, 150g broccoli, 4 eggs, and 2 tomatoes." The server then appends instructions to constrain the output format of the generative AI model. For example, the server might generate the following text snippet: "Please prioritize using the ingredients already available above to design three dinner menus, and minimize the types of ingredients that need to be purchased separately."

[0229] For each menu item, please output: 1) Dish Name 2) All ingredients and quantities required 3) Which ingredients are already in the refrigerator, and which need to be purchased? 4) Cooking steps 5) Estimated preparation time (minutes) Please answer in Simplified Chinese. The server combines the multiple text segments into a complete prompt statement through string concatenation. This prompt statement is stored in a memory buffer as plain text and then transmitted to the model inference service via the generative artificial intelligence model interface module.

[0230] In this embodiment, the generative artificial intelligence model invoked by the server is a deep learning-based sequence-to-sequence language model. This model employs a multi-layer self-attention network structure, containing several encoding and decoding layers, each consisting of a multi-head attention sublayer and a feedforward network sublayer. The model's parameters include multiple weight matrices and bias vectors, and the model is obtained through supervised or self-supervised training on a large-scale text corpus. During the inference phase, the server no longer adjusts the model parameters but instead controls the model's generative behavior through prompts.

[0231] In this embodiment, the server uses the prompt statement as the model's input sequence, which the model encodes into a high-dimensional vector representation. During the decoding phase, the model progressively generates the output text sequence based on this vector representation and the attention weight distribution. The server specifies generation parameters, such as maximum generation length, temperature coefficient, and sampling strategy, through an interface module. These parameters control the diversity and stability of the output to balance menu creativity and executability.

[0232] In this embodiment, the server configures the training process of the generative artificial intelligence model offline. During the training phase, the server uses a corpus containing various recipe texts, nutritional information texts, and menu structure examples to construct training sample pairs. The input is conditional prompt text, and the output is menu description text that meets the structural requirements. During training, the server uses the cross-entropy loss function to measure the difference between the model output and the target text, calculates the gradient through the backpropagation algorithm, and updates the model weights using optimization algorithms based on adaptive learning rate or momentum. The server can employ data augmentation techniques during training, such as random synonym rewriting, unit unification, and conditional rearrangement, to improve the model's robustness to diverse prompt statements.

[0233] In this implementation, the server does not simply rely on the model output during inference, but instead performs secondary structuring processing on the output. The server first uses parsing rules based on text tags or line-start keywords, or requests the model to output using rule-based numbering in the prompt, thereby segmenting the generated text into several parts. The server parses these parts, extracting the dish name, ingredient, step, and time fields into structured data structures. If necessary, the server uses regular expressions to identify numbers and units in quantity information, and then uses a unit conversion module to unify them to standard units.

[0234] In this implementation, the server performs difference calculations on the structured required ingredient data and the standardized ingredient data. The server performs the following operations for each menu item: It establishes a hash table structure in memory with ingredient codes as keys and loads the standardized ingredient data into this hash table; it iterates through the list of required ingredients for each menu item, looking up the inventory quantity in the hash table based on the standard name; it compares the required quantity with the inventory quantity; if the inventory quantity is greater than or equal to the required quantity, the ingredient is marked as a satisfied ingredient; if the inventory quantity is less than the required quantity, the difference is calculated as the insufficient quantity, and the ingredient is added to the insufficient ingredient list. The server uses integer or floating-point operations to perform quantity difference calculations and unit conversions in this process, resulting in near-linear time complexity, thus maintaining high processing speed even in scenarios with multiple menus and multiple ingredients.

[0235] In this embodiment, the server queries the product information database based on the list of insufficient ingredients. The server matches the correspondence between standard ingredient names and available products using index fields, and calculates the minimum product combination that can cover the insufficient quantity based on the product specification field. For example, when the insufficient ingredient is 50 grams of broccoli, and the product specification is 100 grams per serving, the server calculates a recommended purchase quantity of 1 serving. The server compares the unit price and packaging specifications among multiple possible products, selecting a product combination that meets the demand while having the lowest cost.

[0236] In this embodiment, the server integrates menu candidate information, available ingredients, insufficient ingredients, and recommended purchase quantities into "menu candidate screen data." The server generates a record structure for each menu item, containing the dish name, estimated preparation time, nutritional information, ingredient category information, and estimated cost. The server groups these records into an array, serializes them into a terminal-parseable format, and sends them to the terminal via the communication interface. Upon receiving this data, the terminal displays a card-style menu list on its screen. The terminal handles interface rendering locally and does not participate in complex differential calculations, thus achieving a clear division of computational load.

[0237] In this embodiment, the user can select from recommended menu items on the terminal and adjust the quantity of ingredients to be purchased if insufficient. The terminal then sends the user's selection to the server. After receiving confirmation from the user, the server generates order data based on the selected menu items and quantities. When generating the order data, the server incorporates the time and budget constraints from the user's conditions into subsequent calculations.

[0238] In this implementation, the server performs comprehensive calculations on the required ingredients and multiple supply points for the order within the supply and delivery optimization module. The server first reads available supply locations from the "Supply Point Table" and parameters such as distance, historical average transportation time, and estimated energy consumption for different routes from the "Delivery Route Table." The server uses distance calculation and weighted time estimation algorithms to calculate the estimated delivery time for each supply point and route combination. Simultaneously, the server reads product price information and route energy consumption estimation data to calculate the cost and energy consumption evaluation value for each combination. The server comprehensively considers time, cost, and energy consumption through a multi-objective evaluation function, selecting the supply point and delivery route with the optimal evaluation value while meeting the user's time and cost requirements. This process is not a simple rule-based judgment but rather an algorithmic operation to find the optimal solution within a multi-dimensional indicator space, thereby achieving joint optimization of resource utilization and performance within the computer.

[0239] In this embodiment, the server writes the selected supply locations, delivery routes, and order details into the order information database. During the writing process, the server utilizes database transaction mechanisms to ensure data consistency and employs an index structure to improve query and write speeds. Simultaneously, the server sends order summary data to the delivery service device, which performs task scheduling and route execution upon receiving the order. When the server subsequently receives status update information from the delivery service device, it updates the order status field and can dynamically adjust the terminal push strategy based on status changes.

[0240] In this implementation, the server dynamically generates and adjusts the prompt statements, thereby creating feedback constraints on the generative artificial intelligence model. When the server detects that a certain type of food is nearing its expiration date or has excessive inventory, it automatically adds additional conditions to the prompt statements, such as "Please prioritize menu items using chicken breast to reduce food waste." Because the server regenerates the prompt statements before each model call, it can adjust the model's output direction in real time based on the latest inventory and order data. This dynamic prompt statement control mechanism differs from the traditional method of manually writing fixed templates; it allows the computer to use the latest data to drive changes in the prompt statement content, thereby improving the consistency between model output and system constraints.

[0241] In this implementation, the server, through the aforementioned modular and data structure design, enables a series of processes—from user condition collection, prompt generation, model invocation, result parsing, inventory difference calculation, to order and delivery optimization—to be executed efficiently in a pipelined manner on a single computing platform. The server reduces the communication load caused by frequent database access by setting indexes and caching strategies for different data tables; it reduces redundant calculations and complex join operations by using basic data structures such as hash tables and arrays in memory for difference calculations; and it reduces the burden of backend correction and filtering by pre-coding complex constraints into prompt statements, making the model output closer to an executable solution. These measures collectively improve the overall processing speed and computational efficiency of the system, reduce the number of network interactions, and decrease the amount of data transmission between the server and the terminal.

[0242] In this embodiment, the server employs processing rules and unconventional processing sequences that differ from traditional manual operations. In the traditional model, people first manually design menus, then manually check inventory and arrange delivery. In this invention, the server first internally standardizes data and constraints, converting them into prompts. Then, a generative artificial intelligence model generates multiple candidate menus in a high-dimensional vector space. Finally, the server uses algorithms to filter and optimize decisions, automatically generating orders and delivery information. The server doesn't merely simulate linear human thought processes; instead, it employs parallel and batch computing strategies across multiple modules to perform combined calculations on multiple users, multiple menus, and multiple supply points, thereby significantly improving throughput and response speed at the system level.

[0243] In another implementation, the server can employ different types of generative AI models, such as lightweight models with smaller parameter sizes, to be deployed on edge servers or local gateway devices, reducing reliance on remote inference services. The server can select from multiple models based on network conditions and load factors between the endpoint and the server, striking a balance between latency and computational resource consumption.

[0244] In another implementation, the server can fine-tune the generative AI model for a specific region or dietary habits. During fine-tuning, the server uses a regional recipe dataset as training data to update the parameters of the base model, making the model more relevant to the region's ingredient names, cooking methods, and taste preferences. When the server calls the fine-tuned model, it can add regional information to the prompts, further improving the relevance and acceptability of the output.

[0245] In another implementation, the server may not use a relational database, but instead employ a key-value database or document database as the storage backend for order and ingredient information. In this case, the server adjusts the data structure mapping method, but still maintains the structured representation of user condition data, standardized ingredient data, menu candidate information, and order data, thereby ensuring that differential calculation and optimization algorithms can be executed correctly.

[0246] Through the above implementation forms, a tightly coupled technical system is formed between the server, terminal and storage device. This system uses generative artificial intelligence models and dynamic generation mechanisms of prompts, combined with standardized food data and multi-constraint optimization algorithms, to improve data management methods and computing processes within the computer. This results in improved menu generation accuracy, faster processing speed, reduced food waste, and increased utilization of distribution resources.

[0247] use Figure 12 The processing flow is explained.

[0248] Step 1: The user enters conditions on the terminal. Users launch the client application or open a web interface on their terminals. Input consists of the user's subjective needs, including dietary preferences (e.g., "healthy"), target attributes (e.g., "child-friendly"), time constraints (e.g., "delivery within 15 minutes"), cost constraints (e.g., "budget per person not exceeding 1000 yen"), and allergy information (e.g., "peanut allergy"). The terminal performs field integrity checks and numerical format validation on these inputs, combines the fields into an internal data record, serializes this record into structured text, and sends it to the server via encrypted communication. Output is a user conditional data message containing the aforementioned conditional fields.

[0249] Step 2: The server receives and standardizes user condition data. The server receives user condition data from a terminal as input, parses communication messages, and reconstructs the user condition data structure in memory. The server maps natural language or option values to internal standard codes, for example, mapping "healthy" to a health preference code and "child-oriented" to an object attribute code. The server converts time conditions to integer minutes and budget conditions to currency values. The output of the server is a user condition record containing standardized fields, which can directly participate in subsequent data operations.

[0250] Step 3: The server acquires and standardizes ingredient information data The server takes a user ID as input, queries the storage device ID bound to the user in the database, and further queries the ingredient inventory table to obtain the type, quantity, and shelf life information of ingredients recorded in the storage device. The raw data read by the server may contain different units and name representations. The server performs unit conversion (e.g., converting "packet" to grams) and name normalization (e.g., unifying "tomato" under two Chinese aliases to the same standard Chinese name "tomato") on these raw records, and obtains standardized quantity values and standard names through table lookup and arithmetic operations. The output of this processing is a standardized ingredient data set, where each record includes a standard ingredient code, a standard name, a quantity in a unified unit, and a shelf life timestamp.

[0251] Step 4: The server generates prompt statements for a generative artificial intelligence model The server takes standardized user condition records and a standardized ingredient data set as input, and performs string splicing and text formatting operations according to a preset template. The server first converts health preferences, object attributes, time conditions, cost conditions, and allergy information into natural language condition descriptions, then converts the standardized ingredient data into a natural language ingredient description list. The server also adds an instruction description of the output format, specifying the number of menus, field structure and language type that the generative artificial intelligence model shall return. By sequentially combining these text fragments, the server generates a complete prompt statement. The output of this step is a continuous natural language text, used as the input of the generative artificial intelligence model.

[0252] Step 5: The server invokes a generative artificial intelligence model to generate menu candidate information The server takes the prompt generated in step 4 as input and sends an inference request to the generative AI model through the model interface. The server includes parameters such as the model name, maximum generation length, and temperature coefficient in the request. The generative AI model performs internal vector operations and attention calculations in the cloud based on the prompt, generating menu description text word by word. The server receives the model's output text, verifies the response status, and extracts the valid menu description portion. The output of this step is one or more segments of text data containing menu candidate content.

[0253] Step 6: The server parses the menu candidate text and structures the required ingredient data. The server takes the menu candidate text returned by the generative artificial intelligence model as input and applies pre-defined text parsing rules to segment and identify fields in the text. Using line numbers, numbering symbols, or keywords, the server extracts information such as "dish name," "required ingredients and quantities," "cooking steps," and "estimated time" for each menu item. It then uses regular expressions to extract and segment the names, values, and units in the ingredient rows. The server subsequently maps each ingredient name to its internal standard ingredient name and performs unit conversions on the quantities, generating a structured list of required ingredient data. The output of this step is a structured representation of the menu candidate information, containing fielded data for each menu item and standardized records of its required ingredients.

[0254] Step 7: The server calculations have determined whether the required ingredients are sufficient or insufficient. The server takes the standardized ingredient data set obtained in step 3 and the required ingredient data for each menu obtained in step 6 as input. The server constructs a mapping structure in memory with standard ingredient codes as keys and inventory quantities as values, and then queries this mapping for each required ingredient for each menu item. The server uses numerical comparison operations to subtract the required quantity from the inventory quantity. If the result is greater than or equal to zero, the ingredient is marked as a satisfied ingredient; if the result is less than zero, its absolute value is the insufficient quantity, and it is added to the insufficient ingredient list. During this process, the server simultaneously calculates the total shortage and used inventory for each menu. The output of this step is a "satisfied ingredient list" and a "insufficient ingredient list" for each menu, along with their respective quantity information.

[0255] Step 8: The server calculates recommended purchase quantities based on a product information database. The server takes a list of insufficient ingredients as input and queries the product information database for the corresponding product entries for each insufficient ingredient. The server reads the product specification field (e.g., "100g per serving", "3 pieces per bag") and the unit price field, and performs division and rounding up to calculate the minimum number of servings required to meet the shortage. If multiple specifications are available, the server performs a minimization operation based on unit price or other evaluation indicators, selecting the combination with the lower cost. The server organizes the calculation results into "candidate purchase items" and "recommended purchase quantity" structures. The output is a dataset of insufficient ingredient purchase suggestions associated with each menu item, including product identifier, specifications, recommended quantity, and estimated cost.

[0256] Step 9: The server generates menu candidate screen data and sends it to the terminal. The server takes structured menu candidate information, the list of satisfied and insufficient ingredients obtained in step 7, and the recommended purchase data obtained in step 8 as input, and combines this data into a data structure suitable for front-end display. The server generates a screen data record for each menu item, including the dish name, tags (such as "healthy" or "kids-friendly"), estimated preparation time, total cost estimate, list of satisfied ingredients, list of insufficient ingredients and recommended purchase quantities, and a brief nutritional description. The server packages these records to form a menu candidate screen data set and sends it to the terminal via a communication interface. The output of this step is a structured data message for the terminal to render the interface.

[0257] Step 10: Users select menu items and adjust the quantity of ingredients to purchase on the terminal. The terminal takes the menu candidate screen data returned by the server as input and displays multiple menu cards and their corresponding ingredient information on the screen. Users browse the menu on the terminal, select one or more menus, and can increase or decrease the quantity of recommended ingredients using interactive controls on the interface. The terminal updates its local order draft data structure based on user actions and, upon user confirmation of the order, sends the order draft as output to the server via encrypted communication. The output order draft data includes fields such as the selected menu identifier, purchased ingredients and their adjusted quantities, delivery address, and expected delivery time.

[0258] Step 11: The server generates order data and performs supply and delivery optimization. The server takes the draft order data sent by the terminal as input, combines it with the current price in the product information database, the supply point inventory table, and the distance and time data in the delivery route table, and performs a series of calculations. First, the server checks whether each supply point has sufficient inventory to meet the order demand. Then, for each feasible supply point and its delivery route combination, it calculates the estimated delivery time and cost, and estimates energy consumption based on the route parameters. Using a defined evaluation function, the server integrates time constraints, cost conditions, and energy consumption conditions into a single evaluation value, and performs an optimal selection operation on all combinations. After selecting the optimal supply point and delivery route, the server generates formal order data, writing the supply point identifier, route information, and order details into the order database. The server's output is an order record containing complete order information, and simultaneously generates delivery instruction data to be sent to the delivery service device.

[0259] Step 12: The server sends the order data to the delivery service device and then sends feedback to the terminal. The server uses the formal order record generated in step 11 as input to construct the data message required for the delivery service, including the order number, pickup location, delivery address, delivery time requirements, and product details. The server sends this message to the delivery service device through a pre-defined interface. Simultaneously, the server generates order confirmation information for the terminal to display, including the order number, estimated delivery time, and a cost summary. The server sends this confirmation information back to the terminal through a communication interface. The terminal uses this confirmation information as input to update the interface, displaying the order confirmation status and estimated delivery time to the user, thus completing the entire processing flow from conditional input and generative AI model menu generation to actual order and delivery decisions.

[0260] Alternatively, an emotion engine for inferring user emotions can be combined. That is, the specific processing unit 290 can also use the emotion-specific model 59 to infer user emotions and perform specific processing using user emotions.

[0261] Example 2 The flow of a specific process in Example 2 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 as the "terminal."

[0262] In existing technologies, most solutions for managing items within storage devices (such as refrigeration equipment) rely on simple sensors to collect limited information such as quantity or temperature, or depend on users manually inputting food information. This presents the following technical problems: First, image data, physical quantity data, and transaction voucher data (such as shopping receipt images) are often processed in a fragmented manner, lacking a unified data fusion and structured modeling mechanism. This results in incomplete and inaccurate inventory information, making it difficult for computer systems to make highly reliable automatic decisions based on the actual inventory status. Second, generative artificial intelligence models typically only receive plain text prompts for reasoning, failing to integrate with the underlying inventory database and sensors. The deep integration of multi-source data, such as data and user health conditions, often results in menu solutions output by the model that have problems such as unavailable ingredients, mismatched quantities, and inconsistencies with user health goals. This limits the application effect of generative artificial intelligence in real-world home storage management scenarios. Thirdly, traditional systems often treat "food waste control" and "power consumption optimization" as independent logical processes. They lack a unified optimization mechanism in which the computer automatically considers storage period information, environmental status information, and equipment operating condition information in the same information processing flow. This leads to low resource utilization efficiency at the system level, and the computing device cannot effectively exert its collaborative control capabilities for energy and goods management in the real world.

[0263] Therefore, there is an urgent need for a comprehensive processing solution that can automatically integrate image recognition, numerical analysis, text recognition, and inventory modeling within a computer, and encode the resulting structured inventory information along with user-inputted conditional information and prompts into model input data that can be directly used by generative artificial intelligence models. This would enable generative artificial intelligence models to perceive real inventory constraints and health constraints during the menu solution generation stage, and further achieve joint optimization of food waste risk and electricity usage conditions within the same system framework. This would improve the automation level of data processing, the executability of inference results, and resource utilization efficiency from a computer technology perspective.

[0264] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Embodiment 2 is achieved by the following means.

[0265] In this invention, the server includes: a processing unit for acquiring conditional information and prompt statements from a user via an information input / output device; a processing unit for acquiring image data and physical quantity data from a storage device equipped with an imaging device and a detection device, and parsing the image data using an image processing program and a trained recognition model to obtain specific item type information, and parsing the physical quantity data using a numerical processing program to obtain specific item quantity information and status information, thereby generating inventory information; a processing unit for receiving image data representing transaction information, converting the image data into string data using a character recognition program, and extracting purchased item information from the string data to update the inventory information; and a processing unit for generating model input data containing descriptive information about the items in the storage device and the prompt statements based on the inventory information and the user's conditional information, generating prompt data containing instruction information for instructing a generative artificial intelligence model to generate a menu scheme based on the model input data, acquiring menu scheme data output by the generative artificial intelligence model, comparing the required item information contained in the menu scheme data with the inventory information to extract insufficient item information, and generating output data for prompting the user with the menu scheme data and the insufficient item information. This enables the establishment of an end-to-end data processing link on the server side, which automatically constructs structured inventory information from multi-source perception data and transaction data, and then integrates it with user condition information and prompts into the input of a generative artificial intelligence model. This allows the reasoning process of the generative artificial intelligence model to be directly driven by real inventory constraints and health constraints within the computer, and automatically derives out-of-stock information and resource utilization strategies within the same system framework. This improves the automation and decision-making quality of the computer's management of items in storage devices, reduces food waste, and optimizes power consumption, substantially improving computer technology in the fields of inventory management and menu generation based on generative artificial intelligence.

[0266] "System" refers to an overall technical solution consisting of multiple functional units, information processing devices, storage devices, input / output devices, etc., used to perform the data acquisition, data parsing, information generation and output processing described in this invention.

[0267] "Information processing device" refers to an electronic device that has computing capabilities and can execute program code to process, store, analyze, and communicate with external devices for the purpose of collecting data, including but not limited to servers, computer terminals, or embedded processors.

[0268] "Information input / output device" refers to a device used to receive input information from users and output processing results to users, including input interface, display interface, network communication interface and other human-computer interaction or data interaction interface.

[0269] "User condition information" refers to various constraints and requirements provided by users to generate menu plans, including but not limited to taste preferences, number of people, time limits, health requirements, nutritional goals, and budget conditions.

[0270] "Prompt statements" refer to a piece of natural language or structured text, input by the user or generated by the system, used as input to a generative artificial intelligence model to guide the generative artificial intelligence model to generate corresponding menu options or related content.

[0271] "Storage device" refers to equipment used to store food or other items and capable of interacting with information processing devices, including but not limited to refrigeration equipment, freezing equipment, or household or commercial equipment with storage functions.

[0272] "Imaging device" refers to an image acquisition device installed in or near a storage device for acquiring image data of the storage space or its related areas, including but not limited to cameras, image sensors or other image acquisition components.

[0273] "Detection device" refers to a device associated with a storage facility for measuring physical quantities related to stored goods or the environment, including but not limited to weight sensors, temperature sensors, humidity sensors, and other physical quantity sensors.

[0274] "Image data" refers to image information collected by an imaging device and stored in digital form to reflect the internal condition of the items and their environment within a storage device.

[0275] "Physical quantity data" refers to physical parameter information expressed in digital form output by the detection device, including but not limited to weight, temperature, humidity, and other physical measurements related to the item or environment.

[0276] "Image processing program" refers to a software program or software module that runs on an information processing device and is used to preprocess image data, extract features, and perform other image analysis operations.

[0277] A "trained recognition model" refers to a mathematical or parametric model trained on a pre-prepared dataset using machine learning or deep learning methods. It is used to detect or classify objects in image data to identify the types of items.

[0278] "Item category information" refers to the identification information obtained by parsing image data, used to indicate the category or type to which each item in the storage device belongs.

[0279] "Numerical processing program" refers to a software program or software module that runs on an information processing device and is used to analyze, calculate and infer physical quantity data in order to obtain the quantity, state or other derived numerical information of an item.

[0280] "Item quantity information" refers to numerical information representing the quantity or weight of various items in a storage device, based on physical quantity data and pre-set standard parameters.

[0281] "Status information" refers to information indicating the storage status, safety, and usage priority of each item in the storage device, including but not limited to whether it is nearing its expiration date, whether there is a risk of spoilage, and whether it should be used first.

[0282] "Inventory information" refers to a structured set of information that integrates information on the type of goods, quantity of goods, status information, and relevant time or location data, and is used to represent the overall inventory status of the current storage device.

[0283] "Transaction information" refers to information related to the purchase activity, such as the contents, quantity, price, and transaction time of the goods, which usually exists in the form of invoices, vouchers, or electronic records.

[0284] "Image data (transaction information)" refers to image data generated after tickets, vouchers, etc., used to represent transaction information are collected by an imaging device.

[0285] "Character recognition program" refers to an optical character recognition software program or module that runs on an information processing device and is used to convert image data containing text content into editable text data.

[0286] “String data” refers to text information represented as a sequence of characters, obtained by processing image data using a character recognition program.

[0287] "Purchase item information" refers to information extracted from string data that represents the name, quantity, and related attributes of each item purchased in one or more transactions.

[0288] "Model input data" refers to the set of input data constructed to drive generative artificial intelligence models to perform reasoning, including at least information related to menu generation, such as descriptive information about the items in the storage device, user condition information, and prompts.

[0289] "Hint data" refers to structured or textual data that provides instructions for the generative behavior of generative artificial intelligence models, and is used to explicitly or implicitly specify the generation goals, constraints, and output formats.

[0290] "Generative AI models" refer to AI models trained through machine learning that can automatically generate text content (such as menu options and related instructions) based on input model data and prompts, including but not limited to generative models based on neural networks or deep learning.

[0291] "Menu scheme data" refers to text or structured data output by a generative artificial intelligence model that represents one or more menu schemes, including information such as dish names, required items and their quantities, preparation steps, and related instructions.

[0292] "Required Item Information" refers to the information extracted from the menu scheme data, including the types of items required to implement the menu scheme, their quantities, specifications, etc.

[0293] "Insufficient Item Information" refers to information obtained by comparing the required item information with the inventory information, indicating that the current inventory does not contain or is insufficient to meet the requirements of the menu scheme, and the insufficient quantity of such items.

[0294] "Output data" refers to the result data generated by the information processing device and used to present to the user, which includes at least menu scheme data and information on missing items, and may further include health-related information, nutrition-related information and economic benefit information.

[0295] "Health condition information" refers to dietary restrictions proposed by users for physical health, disease management, or lifestyle, including but not limited to salt reduction, low fat, sugar control, allergen avoidance, and energy intake restriction.

[0296] "Nutritional condition information" refers to the condition information proposed by users or systems for the goal of nutritional balance, including but not limited to the intake targets or proportion requirements of nutrients such as protein, fat, carbohydrates, vitamins and minerals.

[0297] "Shelf life information" refers to time information such as shelf life, expiration date, or safe consumption period related to each item in the inventory information.

[0298] "Environmental status information" refers to parameter information related to the internal or surrounding environment of the storage device that can affect the preservation status of the items, including but not limited to temperature, humidity, and door opening and closing frequency.

[0299] "Operating condition information" refers to equipment operating parameter information related to the working status, power consumption mode, and operating time period of the storage device and its associated devices.

[0300] "Electricity usage conditions" refer to the constraints or environment related to electricity consumption at a given time or in a given scenario, including power consumption, time of day, electricity price level, and energy-saving strategies.

[0301] "Utilization planning information" refers to planned information calculated based on a combination of inventory information, shelf-life information, environmental status information, and operating condition information, used to guide the order of item use, cooking arrangements, and equipment operation methods.

[0302] "Economic benefit information" refers to information that represents the benefits or advantages that users gain in terms of economics or resource utilization, calculated based on the reduction of food waste, optimization of electricity consumption, and related cost savings.

[0303] In the following embodiments, the server, terminal, and user each perform different functions as entities. Each embodiment can be used independently or in combination. This invention is not limited to the specific hardware or software names exemplified below; these names are merely illustrative of possible implementations.

[0304] I. Overall System Composition In one embodiment, the server includes a processor, memory, a network interface, and interfaces for communicating with camera and detection devices (e.g., Ethernet interface, serial port interface, wireless communication module, etc.). The server runs applications on an operating system (e.g., a server platform based on a general-purpose operating system), including image processing programs, numerical processing programs, character recognition programs, database management programs, and generative artificial intelligence model inference programs.

[0305] In one embodiment, a terminal includes a mobile or fixed terminal with a display device and an input device, such as a smartphone or tablet. The terminal runs an application or browser to achieve network communication with a server, human-machine interface display, and user input collection.

[0306] In one implementation, users interact with the server through a terminal, including: inputting conditional information and prompts, uploading transaction information images, and browsing inventory information and menu options.

[0307] In one embodiment, the storage device includes a refrigerated space, an imaging device, a detection device, and a communication module. The imaging device can be a fixed camera or a camera with an adjustable viewing angle; the detection device can include a weight sensor, a temperature sensor, a humidity sensor, etc. The storage device transmits the collected image data and physical quantity data to a server via wired or wireless means.

[0308] II. Server-side data structure and module composition In one implementation, the server maintains multiple data tables or data sets in memory, including but not limited to: 1. Item Master Data Table: The server stores fields such as item category identifier, normalized representation of item name, standard unit weight, recommended storage temperature range, and default shelf life in the item master data table. This table is used for subsequent quantity estimation based on weight and shelf life deduction based on category.

[0309] 2. Inventory Information Sheet: The server creates a record in the inventory information table for each user, each item category, and each storage location. Fields may include: user ID, item category ID, current quantity or weight, estimated expiration date, storage status flag, last observation time, observation source (image / sensor / OCR), etc.

[0310] 3. Transaction Record Sheet: The server stores purchase records parsed by a character recognition program in the transaction record table. Fields may include: transaction time, item category identifier, quantity purchased, estimated expiration date, etc.

[0311] 4. User Condition Information Table: The server records the user's health condition information, nutritional condition information, and preference parameters in the user condition information table, such as whether they are low in salt, low in fat, vegetarian, daily target calorie range, and a list of allergens.

[0312] 5. Logs and training data tables: The server records the input data (inventory summary, user condition information, prompt statement) and output data (menu scheme, replenishment information) for each menu scheme generation in the log table for subsequent model optimization.

[0313] In one implementation, the server achieves its functionality through a modular software architecture, and the modules may include: - Image acquisition and preprocessing module; - Image recognition module (calls convolutional neural network model or object detection model); - Sensor data parsing module; - OCR parsing module; - Inventory fusion and update module; - Model input construction module; - Generative artificial intelligence model inference module; - Menu result parsing and verification module; - Output generation and notification module.

[0314] III. Comprehensive Analysis of Image and Sensor Data In one implementation, the server uses an image processing program and a trained recognition model to analyze images within the storage device. The server can employ a convolutional neural network-based object detection architecture, such as a detection network containing multiple convolutional layers, pooling layers, and feature pyramids. During the model inference phase, the server performs matrix multiplication, convolution operations, non-linear activation, and normalization operations, outputting multiple candidate boxes for each image frame. Each candidate box contains the probability distribution of the item category and its location coordinates.

[0315] When performing image processing, the server first performs color space conversion (e.g., from BGR to RGB), size scaling, and normalization, and then inputs the image tensor into the recognition model. The server selects the category corresponding to the maximum probability of the item category. If the maximum probability is lower than a preset threshold, the candidate box is discarded to reduce the false detection rate.

[0316] In another implementation, the server uses a numerical processing program to parse the weight and temperature data from the detection device. Based on the standard unit weight in the item master data table, the server estimates the quantity of items using division and threshold rounding rules. The server can perform differencing operations on historical weight sequences at the same location to infer item removal or placement events, thereby dynamically adjusting the quantity estimate. Based on a combination of temperature and storage time, the server estimates the spoilage risk using a rule table or simple regression model and generates a status information field.

[0317] In one implementation, the server merges image recognition results and sensor analysis results in the inventory fusion module. The server associates data by location and category number, performing consistency checks on data from different sources. For example, if the identified item type and weight variation in an image are significantly inconsistent, the server can flag the anomaly and trigger re-collection or request user confirmation. Through this cross-modal data alignment mechanism, the server reduces single-source errors and improves the accuracy of inventory information.

[0318] IV. OCR Analysis of Transaction Information and Inventory Update In one implementation, a user takes a picture of a shopping receipt or electronic invoice using a terminal, generating image data of the transaction information. The terminal then uploads the image to a server via a network interface.

[0319] In one implementation, the server uses a character recognition program to parse transaction images. The server first performs processing on the image, including grayscale conversion, binarization, skew correction, and page segmentation. Then, it calls an OCR engine to perform character recognition, obtaining string data. The server searches for possible product rows within the string data, either line by line or text block by block, and identifies fields such as product name, quantity, and unit price using regular expressions and keyword matching.

[0320] In the transaction parsing module, the server standardizes product names by performing fuzzy matching or vector similarity calculation with the standard names in the item master data table, mapping different descriptions of product names to a unified item category identifier. The server then writes the identified quantity data into the transaction record table and updates the inventory information table accordingly. At the same time, it calculates the initial retention period based on the item category and purchase date and writes it into the retention period information field.

[0321] V. Structure and Learning Methods of Generative Artificial Intelligence Models In one implementation, the server implements the generative AI model as a text generation model based on a transformer structure. This model includes a multi-layer encoder and a multi-layer decoder, or a decoding-only structure containing multi-head self-attention layers, feedforward network layers, and layer normalization layers. The model's input is a segmented and embedded text sequence, and its output is the probability distribution of the next word or tag.

[0322] During model training, the server constructs a training dataset using a large amount of historical menu text, ingredient information, and user condition information. The server employs cross-entropy as the loss function during training to calculate the error between the predicted word distribution and the true word labels. The server calculates gradients using backpropagation and updates the model weight parameters through stochastic gradient descent, momentum optimization, or adaptive learning rate optimization. The server can use data augmentation strategies during training, such as paraphrasing prompts and randomly shuffling inventory descriptions, to improve the model's robustness to different representations.

[0323] In one implementation, the server fine-tunes the base language model to enable it to understand inventory descriptions and health condition information. The server constructs samples from the fine-tuning dataset, combining "inventory description + user conditions + target menu scheme" into input-output pairs. Through training, the model generates recipe schemes that match the inventory and conditions when it encounters similar structures. The server can encode inventory information into semi-structured text, for example: "Current inventory: 300g chicken breast, 1 head of broccoli, 2 carrots, 6 eggs, 1 carton of milk. Health requirements: low salt, controlled oil." VI. Model Input Construction and Specific Forms of Prompt Statements In one implementation, the user inputs natural language prompts via a terminal. Examples include: "Based on the ingredients I currently have in my refrigerator, please recommend two home-style dishes suitable for tonight's dinner. Each dish should be ready within 30 minutes, suitable for two adults and one child, and should use as many ingredients as possible that are about to expire." - "Please check the inventory in my refrigerator and pantry, tell me what main ingredients I need to prepare for three meals a day for the next three days, and generate a shopping list suitable for a low-sodium diet, with a budget of less than 300 yuan." - "Based on the current inventory list, generate a weekly dinner plan, with 2 dishes and 1 soup per day, requiring a total calorie intake of around 1800 kcal per day. Prioritize ingredients that expire within 3 days, and list the dish names, required ingredients, and simple cooking methods by date." In one implementation, the server combines inventory information and user condition information within the model input construction module, placing them before or after prompt statements to form structured model input. For example, the server can concatenate text according to the following pattern: System Description: The following is a list of ingredients currently stored in the user's storage device: ... User requirements: ... User prompt message: ... Please generate the menu scheme based on the above conditions. The server uses dedicated delimiters to mark different information segments, enabling the model to distinguish between inventory, conditional, and user natural language components during attention calculations. This input structure design allows generative AI models to focus their attention on content related to inventory and conditions during reasoning, thereby reducing the generation of results using non-existent ingredients or violating conditions and improving reasoning accuracy.

[0324] VII. Menu scheme parsing, verification, and replenishment information generation In one implementation, after receiving a text sequence output by a generative artificial intelligence model, the server structures the text in the menu parsing module. The server then uses pattern matching or sequence labeling methods to identify the dish name, required ingredients and their quantities, and cooking steps from the text. The server can maintain a set of predefined formats, such as "Required Ingredients:..." "Steps:...", and extract information segmented based on these keywords.

[0325] In the inventory verification module, the server compares the required item information with the inventory information table. For each item category, the server performs a subtraction operation: the required quantity minus the inventory quantity. If the result is greater than zero, it is recorded as a shortage quantity, forming a shortage item information table. The server can further sort the shortage items according to their importance. For example, completely missing high-priority ingredients are marked as "critical shortage," and slightly insufficient ones are marked as "optional replenishment."

[0326] In the output generation module, the server combines menu scheme data, information on missing items, and possible alternative suggestions into output data and sends it to the terminal. Alternative suggestions can be generated by rules or simple algorithms; for example, when a certain vegetable is unavailable, alternatives with similar nutritional characteristics can be recommended based on the "nutritional similarity" field in the item master data table.

[0327] VIII. Terminal-side interface and interaction methods In one implementation, the terminal displays inventory information and menu scheme information returned by the server through a user interface. The terminal can display the inventory information in list format, using color to mark items that are about to expire or are at risk. When displaying menu schemes, the terminal can expand each dish into a list of dish name, required ingredients, and steps, which the user can view item by item.

[0328] In another implementation, users can provide feedback on the menu results on the terminal, such as marking a recipe as "not to their liking" or "the steps are too complicated." The terminal uploads this feedback to the server, which records it in a log table for subsequent improvements to model training data or adjustments to recommendation logic, thereby continuously optimizing the generative artificial intelligence model at the system level.

[0329] IX. Technical Effects and Improvements in Computer Technology In the above implementations, the server improves the computer technology itself through the following mechanisms: 1. Improvements at the data fusion level: Internally, the server integrates image data, physical quantity data, and transaction image data using a unified data structure, avoiding data redundancy and inconsistency issues caused by independent processing in traditional systems. By cross-validating sensor data with image recognition results, the server reduces the impact of single-source noise, improves inventory estimation accuracy, and thus reduces error propagation during subsequent inference.

[0330] 2. Optimization of model input structure: The server introduces explicit inventory and conditional segments during the model input construction phase, allowing the generative AI model to selectively focus on key fields in the attention layer. This input design represents a technical improvement to natural language model input encoding methods, enhancing the model's ability to jointly understand structured and unstructured information and reducing the often unrealistic outputs that occur with traditional "prompt-only" methods.

[0331] 3. Improvements in computational efficiency and communication load: The server performs image and sensor data parsing locally, only transmitting compressed text descriptions to the generative AI model instead of high-dimensional images or raw time-series data. This significantly reduces the dimensionality of the input data during model inference and decreases computational resource consumption. Because inventory and conditional information are encoded in text, the communication load between the server and the model is reduced, improving system response speed.

[0332] 4. Processing flow that combines non-traditional rules with learning: The server uses rules in some parts of the processing (e.g., weight difference inference of retrieval behavior, shelf-life inference rules, nutritional similarity substitution rules), and learning models (image recognition models, generative artificial intelligence models) in other parts. This hybrid strategy forms a specific processing pipeline within the computer. Unlike traditional single-rule systems or single-generative models, it can leverage the strong expressive power of deep learning while maintaining interpretability and stability, achieving higher overall accuracy and robustness.

[0333] 5. Learning methods and parameter updates: The server employs standard error functions (such as cross-entropy) and gradient descent optimization algorithms when training generative AI models. Regularization terms can be introduced to prevent overfitting, and learning rate scheduling and batch normalization improve convergence speed and numerical stability. The server constructs fine-tuning datasets using log data, allowing the model to continuously adapt to the inventory patterns and preferences of specific user groups, thereby substantially improving prediction quality over the long term.

[0334] 10. Multiple variations and alternative implementation forms In another implementation, the server can deploy the generative AI model on an external inference service node, sending model input data and receiving model output via remote calls through a network interface. In this case, the server still performs local data fusion, input construction, and result validation, thus ensuring compatibility with generative models of different sizes and architectures.

[0335] In other implementations, the server can use different types of recognition models, such as lightweight convolutional networks to adapt to edge computing nodes, or attention-based visual models to improve recognition accuracy in complex scenes. The server can also add or remove physical quantity data fields based on the type of sensor; for example, adding gas sensor data to detect deteriorating gases further refines the state information estimation.

[0336] In another implementation, the terminal can provide users with voice input functionality. The process of converting speech into text can be completed by the local speech recognition module on the terminal or the server side. The converted text is then used as prompts for generative artificial intelligence models.

[0337] In some implementations, users can also control the operating mode of the storage device through the terminal. For example, after the server generates a "centralized cooking time period" according to the menu plan, it sends instructions to the storage device to adjust the temperature or enhance cold preservation, so that the storage device operates in energy-saving mode or enhanced preservation mode during the critical time period. This realizes the linkage from information processing to physical equipment control, further demonstrating that the present invention is not just an abstract processing flow, but brings technical effects of energy optimization and food safety management at the level of equipment control in the real world.

[0338] use Figure 13 The processing flow is explained.

[0339] Step 1: The server retrieves raw data from the storage device.

[0340] Inputs: Image data from the imaging device, physical quantity data (weight, temperature, etc.) from the detection device.

[0341] The server receives image frames and sensor data packets through a network interface, caches the images in memory and stores them with timestamps, and parses the sensor data into structured records. The server performs preprocessing on the images, such as color space conversion, size scaling, and noise reduction, performs unit conversion and time synchronization on the sensor data, and outputs the preprocessed image data and aligned physical quantity data.

[0342] Step 2: The server uses recognition models and numerical processing programs to generate basic inventory information.

[0343] Input: Preprocessed image data, aligned physical quantity data, and item master data table (standard unit weight, category definition, etc.).

[0344] The server inputs image data into the trained recognition model, and obtains the item type and confidence score for each candidate region through convolution and classification operations. It then filters out low-confidence results to obtain item type labels and their locations. The server finds the standard unit weight based on the item type, performs division and rounding on the weight data for the same storage location to estimate the item quantity, and calculates the storage status based on temperature and historical time. Finally, the server combines the item type, quantity, location, and status to output a basic inventory information record set.

[0345] Step 3: The server updates the inventory information table.

[0346] Input: Inventory basic information record set, existing inventory information table.

[0347] For each basic inventory information record, the server searches the inventory information table for a matching record using the user identifier, item category, and location as keys. If a match is found, the server updates the quantity and status; otherwise, it inserts a new record. The server uses timestamp comparisons to determine whether to overwrite old observations and infers retrieval or replenishment events based on the difference between the old and new quantities, adjusting the retention period estimate accordingly. The output is the updated inventory information table.

[0348] Step 4: Users upload images of transaction information via their terminals.

[0349] Input: Physical paper or electronic receipt.

[0350] The user uses the terminal's camera to capture the receipt information. The terminal caches the image as a file and sends it to the server via a network request. The terminal includes the user's identifier and the capture time when uploading. The output is a transaction image request submitted to the server.

[0351] Step 5: The server parses transaction information and updates inventory using a character recognition program.

[0352] Inputs: Transaction image request (including image data and user ID), item master data table, and existing inventory information table.

[0353] The server performs grayscale conversion, binarization, and skew correction on the transaction images, then calls a character recognition program to convert the images into string data. The server uses regular expression matching and word segmentation to parse each product row into product name and quantity, and uses fuzzy matching to map the product name to the internal item category. The server adds the parsed purchase quantity to the corresponding item record in the inventory information table and recalculates the shelf life based on the purchase time and the item's default shelf life. The output is a transaction record table containing the transaction records and a revised inventory information table.

[0354] Step 6: Users input conditional information and prompts through the terminal.

[0355] Input: The user's dining needs and health needs.

[0356] Users select criteria such as number of people, meal time, cuisine preference, health requirements (e.g., low salt, low fat), and budget on the terminal interface. They then enter prompts in text boxes, for example: "Based on the ingredients in my refrigerator, please recommend two home-style dishes suitable for tonight's dinner. Each dish should be ready within 30 minutes, suitable for two adults and one child, and should use as many ingredients as possible that are about to expire." The terminal packages the criteria and prompts into request data and sends it to the server over the network. The output is a request message containing the criteria and prompts.

[0357] Step 7: The server constructs the input data for generative artificial intelligence models.

[0358] Input: Request message (conditional information, prompt statement), latest inventory information table, user conditional information table.

[0359] The server retrieves the current user's inventory data from the inventory information table based on the user's identifier, extracting the product name, quantity, and shelf life, and translating this into a natural language description. The server supplements or overwrites the submitted conditions from the user condition information table. The server concatenates the "inventory description," "condition description," and "user prompt" in a predetermined order, adding delimiters and tags to form the structured model input text. The output is the model input data used for generative artificial intelligence model inference.

[0360] Step 8: The server calls a generative artificial intelligence model to generate a menu scheme.

[0361] Inputs: Model input data, generative artificial intelligence model parameters.

[0362] The server segments and encodes the input data to the model, then inputs the resulting vector sequence into the embedding layer of the generative AI model. The vector then passes through a multi-head self-attention layer and a feedforward network layer, calculating the output vector at each position and the probability distribution of the next label. The server decodes and generates text progressively according to a maximum probability or sampling strategy until an end label is generated or the length limit is reached. The output is the original generated text containing one or more menu schemes.

[0363] Step 9: The server parses the generated text and extracts the required item information.

[0364] Input: The original generated text.

[0365] The server searches for keywords such as "dish name," "required ingredients," and "steps" according to a preset format, and segments the generated text into multiple recipe blocks. Within each recipe block, the server identifies the ingredient names and their quantities, maps the ingredient names to item category identifiers by matching them with the item master data table, and standardizes the quantities to a uniform unit. The output is structured menu scheme data, including a list of dish names, the required ingredients for each dish, and their quantities.

[0366] Step 10: The server performs inventory checks on the menu scheme and generates information on items that are out of stock.

[0367] Input: Menu scheme data, latest inventory information table.

[0368] For each item category of each dish, the server reads the current quantity information from the inventory table and performs a quantity difference calculation: the required quantity minus the inventory quantity. If the difference is greater than zero, it is recorded as insufficient; if the inventory record does not exist, it is recorded as completely missing. The server summarizes all insufficient entries, sorts them by importance or recipe priority, and generates a list of insufficient items. The output is menu scheme data with inventory verification results and insufficient item information.

[0369] Step 11: The server generates user-facing output data.

[0370] Input: Menu scheme data, insufficient item information, user health condition information.

[0371] The server generates health tips for each dish based on health information, such as marking it as low-salt or high-protein. The server organizes the menu scheme, list of insufficient items, and health tips into a unified output structure, generating a simple nutritional summary or economic explanation when necessary. The server encodes this output structure into a terminal-parsable response format and sends it to the terminal. The output is a response message containing menu details and restocking suggestions.

[0372] Step 12: The terminal displays the results and supports subsequent user interaction.

[0373] Input: The response message from the server.

[0374] The terminal parses the response message and displays the name of each dish, required ingredients, steps, and health tips in a list or card format on the interface, while also displaying a list of missing items in a separate area. The terminal allows users to click on a dish to view detailed steps or select missing items to add to their shopping list. If the user is not satisfied with the result, they can modify the prompts on the terminal, such as entering "remove the spice," "don't fry," or "change to a high-protein diet suitable for weight loss." The terminal then sends a new request to the server. The output includes the user-visible interface content and any subsequent requests.

[0375] Application Example 2 The process flow corresponding to the specific processing in Use Case 2 will be described below. The various parts of the system described below are implemented by the data processing device 12 and the intelligent device 14. In addition, the data processing device 12 is referred to as the "server" and the intelligent device 14 is referred to as the "terminal".

[0376] With the development of information processing and network communication technologies, users can access a wealth of recipe information and food delivery services through electronic terminals and online services in their daily diet management. However, the existing technologies have the following problems: (1) In most recipe recommendation systems, the information processing device only searches based on simple keywords, fixed rules or preset menus entered by the user. It lacks automatic acquisition and refined modeling of real-time food inventory information in the storage device, which leads to the disconnect between the recommendation results and the actual available food. Users still need to perform a lot of manual comparison and adjustment, which increases the operational burden.

[0377] (2) Existing systems typically embed recipe recommendation logic within applications, failing to fully utilize generative artificial intelligence models for comprehensive reasoning and natural language generation under various constraints (including health goals, cooking time, energy intake, etc.). The prompt statements are constructed in a simple and crude manner, failing to form high-quality model inputs tailored to specific users and inventory statuses, thus limiting the output quality and personalization of generative artificial intelligence models.

[0378] (3) Existing technologies often ignore the user's emotional state when generating menus or meal plans, and only make recommendations based on objective conditions. They cannot adaptively adjust the style and tone of prompts and notifications, resulting in a rigid and stiff interactive experience that makes it difficult to improve the user's subjective satisfaction. Especially when the user is in a state of fatigue, anxiety or anger, the lack of dynamic adjustment of the notification method may even increase the user's burden.

[0379] (4) Regarding the handling of missing ingredients, traditional solutions usually only prompt users with a list of items they need to purchase. This is not deeply integrated with the automatic ordering process of external services, requiring users to frequently switch between multiple applications or websites to manually place orders. This not only increases the number of manual operation steps, but also makes it difficult to uniformly optimize the efficiency of ingredient utilization and delivery timing on the server side.

[0380] (5) From a computer technology perspective, existing systems often treat "recipe generation," "inventory management," "emotion recognition," and "external ordering" as separate functional modules. They lack a unified processing flow and data structure on the server side that models user conditions, inventory status, and emotional information, automatically generates high-quality prompts, calls generative artificial intelligence models, and links with external services. This fragmented design results in the server being unable to complete state fusion and decision-making in a single centralized processing step, increasing multiple network round trips and redundant calculations, leading to low resource utilization efficiency and difficulty in maintenance and expansion.

[0381] Therefore, it is necessary to provide a system and method that performs unified data processing on the server side for user condition information, storage device inventory information, and emotional information, and automatically constructs prompts adapted to generative artificial intelligence models, generates diet plans, specific deficiencies, and links with external services. This will improve the user experience while substantially improving the processing efficiency and intelligence level of information processing devices in data fusion, model invocation, notification generation, and external service control.

[0382] The specific processing performed by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is achieved by the following means.

[0383] In this invention, the server includes means for receiving conditional and preference information related to diet from a user via an information input / output device; means for identifying the type and quantity of target objects in the storage device based on image data and detection data acquired by an imaging device and a detection device disposed in the storage device, and recording and updating them as inventory information; means for inferring an emotional state based on the user's facial expression data, voice data, or text data and acquiring the emotional state as emotional information; means for generating a prompt statement based on the inventory information, conditional information, preference information, and emotional information to instruct a generative artificial intelligence model to generate at least one diet plan, and inputting the prompt statement to the generative artificial intelligence model to obtain the diet plan; means for comparing the required components included in the diet plan with the inventory information and identifying components with insufficient inventory as insufficient components; means for determining the notification content and notification expression method based on the insufficient components and emotional information and notifying the user of the diet plan and insufficient components through the information input / output device; and means for generating ordering information for external services corresponding to the insufficient components and sending the ordering information to external services through a communication device. This allows for centralized modeling and fusion calculation of user conditions, physical circulation status, and emotional state on the server side using a unified data structure and processing flow. It automatically constructs highly relevant prompts to improve the effectiveness and personalization of generative artificial intelligence model calls. At the same time, it completes the generation of diet plans, the identification of deficiencies, and external ordering control through a one-time decision, reducing multiple rounds of human-computer interaction and repeated network calls. This substantially improves the overall performance of information processing devices in terms of resource utilization, response time, and interactive experience.

[0384] "System" refers to a collection of devices, including information processing devices, storage devices, communication devices, and information input / output devices, used to execute the processing steps described in this invention.

[0385] "Information processing device" refers to an electronic device that includes a combination of hardware and software with computational functions, used to analyze, calculate, store, and control external devices with input data.

[0386] A "server" refers to a computer device that, as an implementation of an information processing device, communicates with terminal devices via a network and performs data processing, model invocation, and control logic in the background.

[0387] "Information input / output device" refers to a human-computer interaction device used to receive user input information and present output information to the user, including display devices, touch devices, keyboards, pointing devices, speakers, microphones, etc.

[0388] "Storage device" refers to a container or equipment used to store food, goods or other objects, including refrigeration equipment, freezing equipment, ambient temperature storage cabinets, etc.

[0389] "Imaging device" refers to an image acquisition device used to acquire image data stored in a storage device, including cameras, photographing modules, etc.

[0390] "Detection device" refers to a sensor used to detect physical quantities inside a storage device and output detection data, including weight sensors, temperature sensors, humidity sensors, identification tag readers, etc.

[0391] "Image data" refers to digital image information acquired by an imaging device and used to represent scenes inside a storage device, including still image data and moving image data.

[0392] "Detection data" refers to measurement data related to the target object in the storage device output by the detection device, including weight data, temperature data, tag identification data, etc.

[0393] "Target object" refers to the object identified and managed within the storage device, including ingredients, food, condiments, and other consumable items.

[0394] "Inventory information" refers to a set of data that indicates the type, quantity, location, and shelf life of a target object within a storage device.

[0395] "Recorded information" refers to structured data such as inventory information, user information, emotional information, diet plans, and order information stored on storage media.

[0396] "User" refers to the entity that uses the system described in this invention to perform diet management, recipe acquisition and ordering operations, including individual users or group users.

[0397] "Conditional information" refers to the constraints or requirements that users input regarding their diet-related needs, including meal times, types of dishes, health goals, energy limits, and cooking time limits.

[0398] "Preference information" refers to data that reflects a user's long-term or short-term dietary preferences, taste preferences, dietary restrictions, and eating habits.

[0399] "Face data" refers to image or video data acquired through imaging devices and used to analyze the facial expression features of users.

[0400] "Voice data" refers to sound signal data acquired through audio acquisition devices and used to analyze user tone, emotional tendencies, and semantic content.

[0401] "Text data" refers to character information submitted by users through text input, including message text, search text, comment text, etc.

[0402] "Emotional state" refers to the user's psychological state obtained from the analysis of facial expression data, voice data, or text data, including categories such as joy, fatigue, depression, and anger.

[0403] "Emotional information" refers to a set of labeled data or parameters used to represent a user's emotional state and its intensity.

[0404] "Generative artificial intelligence models" refer to machine learning models that automatically generate natural language text or structured content based on input prompts, including language models based on neural networks.

[0405] "Prompt statements" refer to natural language or structured text input used to provide task descriptions, contextual information, and constraints to generative artificial intelligence models.

[0406] "Dietary plans" refer to dietary recommendations generated by generative artificial intelligence models and processed by the system, including recipes, meal plans, food combinations, and corresponding descriptions.

[0407] "Required components" refers to a list of ingredients, seasonings, and other necessary items needed to realize the various dishes or menus in a particular dietary plan.

[0408] "Insufficient constituent elements" refers to constituent elements that, after comparison with inventory information, are determined to be absent or insufficient in the storage device.

[0409] "Notification content" refers to the main text of the message generated by the system to provide users with dietary plans, deficiencies, or related information.

[0410] "Notification expression style" refers to the specific way in which the content of a notification is expressed in terms of tone, word choice, style, length, and presentation, and can be adjusted according to emotional information.

[0411] "External services" refers to service systems that operate independently outside the system and interact with the system through communication networks, including goods delivery services, online retail services, food supply services, etc.

[0412] "Ordering information" refers to order-related data generated by the system in response to a shortage of supply of essential elements for providing services to external parties. This data includes product type, quantity, delivery information, and payment information.

[0413] "Communication device" refers to communication hardware and its control modules used for sending and receiving data between the system and external services, including wired communication devices and wireless communication devices.

[0414] In one embodiment of the present invention, a server is located on the network side as an information processing device, and a terminal is located in the user environment as a human-machine interface device on the user side. The user interacts with the server through the terminal. The server implements the various functional modules of the present invention by executing a program stored in a non-transitory storage medium, said program running on a general-purpose processor or a dedicated processor.

[0415] I. Hardware and Software Composition The server employs a computer device with a multi-core processor and large-capacity main memory. The server runs a general-purpose operating system and the following software components: 1. The server uses database management software (such as a relational database management system or a key-value database) to store inventory information, user information, emotional information, and diet plan information.

[0416] 2. The server uses an image processing library (such as an open-source image processing library) to preprocess the images uploaded by the imaging device, including size normalization, color space conversion, and noise filtering.

[0417] 3. The server uses a deep learning framework (such as a general-purpose deep learning framework) to deploy a convolutional neural network model to classify and detect food images.

[0418] 4. The server uses generative artificial intelligence model interfaces (such as online interfaces or locally deployed interfaces based on large-scale language models) to receive and send natural language text.

[0419] 5. The server uses an emotion recognition module, which can achieve facial expression recognition through a deep learning framework, speech emotion recognition through an audio analysis module, or text emotion recognition through a text analysis module.

[0420] 6. The server uses a network communication module to exchange data with the terminal and external service providers through Hypertext Transfer Protocol or other application layer protocols.

[0421] The terminal uses a smartphone, tablet, or personal computing device with a client application or browser installed. It includes a display device, touchscreen, camera, and audio capture device. The terminal runs an operating system and user interface program to display dietary plans and notifications to the user, and to collect user input and emotion-related data.

[0422] The storage unit employs refrigeration equipment or other storage devices. An internal camera and detection system are installed within the storage unit. The camera can be a fixed digital camera used to periodically capture images of the storage space. The detection system may include weight sensors, ambient temperature sensors, and label readers to output detection data representing weight changes, temperature conditions, and label information for each shelf or location.

[0423] II. Server Program Structure and Data Structure The server implements the functions of this invention through a modular program structure. The server maintains multiple logical modules in the storage medium, including: an inventory identification module, a condition management module, a sentiment analysis module, a prompt statement generation module, a generative artificial intelligence invocation module, a deficiency element determination module, a notification generation module, and an external service linkage module.

[0424] 1. The server defines structured data records for inventory information in the database. Each record includes at least: target object identifier, target object category, quantity, unit, corresponding storage device identifier, and update time.

[0425] 2. The server defines records for user information, including: user identifier, preference information (such as preferred cuisine, frequently used ingredients, and foods to avoid), and health conditions (such as energy intake targets and nutritional targets).

[0426] 3. The server defines and records emotional information, including: user identifier, emotional tag (joy, fatigue, depression, anger, etc.), emotional confidence level, collection time, and source type (facial expression, voice, or text).

[0427] 4. The server defines and records the dietary plan information, including: plan identifier, menu list, list of required components for each dish, generation time, and associated prompt text.

[0428] This structured data layout allows the server to quickly access relevant information internally using an index, thereby shortening query time, reducing main memory usage, and improving overall processing efficiency.

[0429] III. Technical Details of Inventory Identification and Update The server uses an inventory identification module to identify and estimate the quantity of target objects within the storage device. This module utilizes a convolutional neural network model, which can have multiple convolutional layers, pooling layers, and fully connected layers, and is pre-trained using a large-scale labeled image set. During the inference phase, the server receives image data uploaded from the terminal, inputs it into the network, and outputs a probability vector and possible location information for each food category.

[0430] The server employs the following technical means during the inference process: 1. The server performs block segmentation and scaling on the image, dividing the large image into multiple sub-regions to independently identify local targets and improve detection accuracy.

[0431] 2. The server combines data from weight sensors to match the identified food categories with the weight changes of the corresponding shelves or areas, and uses the pre-stored standard weight of a single item to estimate the quantity of the target object.

[0432] 3. The server updates inventory information by performing a differential comparison between the current identification results and historical inventory records to determine newly added and disappeared target objects. To reduce the impact of noise, the server can apply algorithms such as threshold filtering and moving averages.

[0433] Through the aforementioned specific data fusion algorithm, the server can determine the type and quantity of target objects in the storage device with high accuracy and robustness, thereby providing reliable underlying data support for the subsequent generation of diet plans.

[0434] IV. Acquisition and Utilization of Emotional Information The terminal acquires the user's facial image through a camera device and the user's voice through an audio acquisition device. The server processes this data through a sentiment analysis module. The sentiment analysis module may include the following substructures: 1. The server uses a convolutional neural network to extract features from facial images, including facial key points and local texture features; 2. The server uses a recurrent neural network or a model with an attention mechanism to perform temporal analysis on the speech feature sequence, extract statistical features such as energy, pitch, and speech rate, and combine them with pronunciation patterns to predict emotion labels; 3. The server uses a pre-trained language model to classify the sentiment of the text data, analyzes word choice and syntactic structure to determine positive or negative emotions.

[0435] The server synthesizes multi-source sentiment data into a sentiment information record using weighted voting or Bayesian fusion. This feature-level and decision-level fusion allows the server to maintain high sentiment judgment accuracy even in noisy environments. The sentiment information is used not only to adjust dietary preferences but also to refine the tone and style of notification content, thereby enhancing the user experience.

[0436] V. Generation of Prompt Statements and Invocation of Generative Artificial Intelligence Models The server integrates inventory, condition, preference, and sentiment information into high-quality natural language text through a prompt generation module. Unlike simple keyword concatenation, the server employs a strategy combining template filling and rule combination when generating prompts. 1. The server selects the corresponding sentence template based on the user's current task type (e.g., "single recipe recommendation" or "multi-day meal plan generation").

[0437] 2. The server prioritizes inserting frequently consumed and soon-to-expire ingredients from the inventory information into the notification statements to encourage the generative AI model to pay more attention to these ingredients.

[0438] 3. The server will explicitly state the health conditions (such as "low calories", "high protein", "limit fat" etc.) in the prompt statement.

[0439] 4. The server selects different tones of voice based on emotional information. For example, it emphasizes "simple and short" when the user is tired, and "celebrated and ritualistic" when the user is joyful.

[0440] For example, in one embodiment, the server can generate the following prompt statement: "You are a smart menu recommendation assistant."

[0441] Ingredients currently in the refrigerator: 300g chicken breast, 2 tomatoes, 1 onion, 150g mozzarella cheese, no basil.

[0442] The user is feeling a bit tired and wants the process to be as simple as possible.

[0443] User requirements: Italian-style dinner, to be completed within 20 minutes, as low in calories as possible, avoiding nuts.

[0444] Please recommend three suitable recipes in Chinese, and provide the name of each dish, its main ingredients, a brief description of how to make it, and the estimated cooking time. For example, in a multi-day meal planning scenario, the server can generate the following prompt: Please generate a 7-day healthy eating plan in Chinese based on the following conditions: 1. User goal: Weight loss, with daily total calorie intake controlled below 1800 kcal; 2. User preferences: Primarily Chinese food, with occasional preference for Italian pasta; 3. Foods to keep in the refrigerator: chicken breast, eggs, oatmeal, tomatoes, broccoli, and carrots.

[0445] Please provide the menu names and brief descriptions for breakfast, lunch, and dinner for each day. The server inputs the aforementioned prompts into the generative AI model. This generative AI model can be a large language model based on a transformer architecture, featuring multiple layers of self-attention layers and feedforward networks. The server sets parameters such as maximum output length and sampling temperature during the call to control output diversity and stability. After receiving the natural language text output by the model, the server converts the text into structured dietary plan data through keyword extraction, pattern matching, and segmented parsing.

[0446] Through this collaborative design of "optimized prompt statements + model invocation", the server can improve the consistency between model output and actual user needs and inventory status, thereby reducing the number of secondary interactions and unnecessary requests, and reducing network communication burden.

[0447] VI. Determination of Insufficient Elements and Generation of Notifications The server uses a deficiency component determination module to perform precise calculations for missing ingredients in the generated meal plans. The server compares each required component of a dish with the inventory information item by item, using numerical difference calculations to determine if there are any shortages. To avoid decimal errors, the server can employ algorithms such as different unit conversions and standardization of the smallest unit of measurement to improve the accuracy of the determination.

[0448] When generating notification content, the server adjusts the style based on emotional information. For example, in an angry state, the server generates text with a mild tone and appropriate length, such as: “We have already selected a suitable recipe for you. When making the ‘Caprese’ dish, you only need to prepare a small amount of basil. If it is inconvenient for you to go out today, you can consider it later.” In a state of joy, the server can generate more exciting text, such as: "Great! With just a little more basil, I can make a delicious caprese and add a special touch to today!" The notification generation module uses a combination of preset sentence structures and generative artificial intelligence models: the server first constructs intermediate text containing key factual information, and then requests the generative artificial intelligence model to refine it naturally, thereby obtaining copy that is more in line with emotional state and user habits while maintaining factual accuracy.

[0449] VII. External Service Collaboration and Technical Effectiveness The server connects to external services via communication devices. After determining the insufficient components, the server can select appropriate supply channels according to contract rules and generate order information using a unified order data structure. Internally, the server employs queuing and asynchronous call mechanisms to reduce synchronization waiting time and improve overall throughput.

[0450] When a server sends order information to external service providers, it can combine or advance orders based on inventory forecasts and delivery time estimates, thereby reducing the total number of deliveries at the system level and saving transportation resources. This centralized scheduling logic on the server side not only simplifies the user operation process but also improves the utilization rate of network resources by reducing duplicate communication and redundant requests.

[0451] VIII. Explanation of Technical Effects and Causal Relationships Through specific data flow and algorithm design between the aforementioned modules, the server achieves the following technical effects: 1. The server identifies inventory by fusing image features with sensor data, which has higher accuracy and robustness compared to identification from a single data source, thereby reducing the waste of subsequent calculations caused by misjudgments.

[0452] 2. The server accelerates access to inventory and user data, reduces disk I / O operations, and improves query performance through structured data design and indexing mechanisms.

[0453] 3. By dynamically generating highly relevant prompts based on inventory and sentiment, the server improves the output hit rate of the generative AI model, enabling results that are closer to user needs under the same model configuration, and reducing the computation time and network bandwidth required for repeated requests.

[0454] 4. The server completes the centralized calculation process of "solution generation - missing data determination - external ordering" in one go, avoiding multiple requests and intermediate data access caused by multiple rounds of queries, manual confirmation and repeated ordering in traditional solutions, thereby reducing overall latency.

[0455] 5. By incorporating emotional information into the notification text generation process, the server increases the acceptance of notifications without increasing user interaction steps, indirectly reducing users' repeated queries to the system and lowering server load.

[0456] 6. By employing specific model structures, training methods, and data fusion strategies, the server integrates the previously fragmented "inventory management," "recipe generation," "emotion generation," and "order control" into a unified software architecture. This allows the system to complete more computational tasks within a single terminal request response cycle, thereby significantly optimizing overall computational efficiency.

[0457] IX. Alternative Implementation Methods In other implementations, the server can use locally deployed generative AI models or a combination of multiple models. For example, a smaller model can be used to quickly generate candidate solutions, and then a larger model can be used for refinement, thereby reducing the average computational cost while ensuring output quality.

[0458] The server can also use different network structures in the sentiment analysis part, such as graph convolutional networks for facial key point map analysis, or use lightweight models deployed on the terminal side for pre-analysis, and then upload the sentiment summary to the server, thereby further reducing the uplink communication volume.

[0459] Regarding storage devices, the camera device can be equipped with a multi-view camera array to reduce recognition errors caused by occlusion; the detection device can be equipped with a gas sensor to estimate the freshness of ingredients and incorporate inventory information, thereby guiding the generative artificial intelligence model to prioritize the use of ingredients that are about to spoil in the prompt statement.

[0460] Through the various implementation forms and alternative methods described above, this invention, while maintaining the core technical idea of ​​integrating inventory information, conditional information, and emotional information with a server as the center to generate optimized prompt statements and call a generative artificial intelligence model to output dietary plans, provides a flexible system implementation approach for different application environments. This results in substantial improvements to existing computer technology on multiple levels, such as processing speed, recognition accuracy, resource utilization, and user experience.

[0461] use Figure 14 The processing flow is explained.

[0462] Step 1: Users input conditional and preference information through the terminal.

[0463] Input: Raw interactive data such as user clicks, keyboard input, and voice input.

[0464] The terminal displays forms and options on the screen. Users can select meal type (such as breakfast, lunch, dinner), cuisine preference, health goals, cooking time limit, allergens, etc., and can also enter free text (such as "I want to eat a simple tomato dish").

[0465] The terminal parses the above information into structured fields, performs format validation and default value completion, and then packages it into conditional information and preference information.

[0466] Output: Structured data containing user identifiers, condition information, and preference information, which is sent to the server over the network.

[0467] Step 2: The terminal collects and uploads image and detection data stored in the storage device.

[0468] Input: Image frames from the camera device in the storage device and analog or digital signals from the detection device (weight sensor, temperature sensor, tag reader, etc.).

[0469] The terminal controls the camera device to capture images inside the storage device at a predetermined time point or when triggered by the user, converting analog signals into digital image data; the terminal reads measured values ​​such as weight, temperature, and tag ID from the detection device, and performs unit conversion and timestamp marking.

[0470] The terminal encapsulates the image data (compressed into JPEG or other formats) together with the detection data into a message, which is then sent to the server via the communication module.

[0471] Output: An upload message containing at least one image and its corresponding detection data.

[0472] Step 3: The server identifies inventory and updates inventory information based on image and detection data.

[0473] Input: Image data and detection data from the terminal.

[0474] The server uses image processing software to perform preprocessing on the image, such as scaling, cropping, and color normalization. The processed image is then converted into tensor form and input into the convolutional neural network. The server uses a deep learning framework to run forward inference, calculates the probability value for each candidate food category, and extracts the target bounding box position.

[0475] The server then matches the category of each identified target object with the weight and label information in the detection data, and calculates the number of target objects using a pre-stored standard weight for a single item or a label mapping table.

[0476] The server updates the target objects that are newly added, reduced, or have changed in quantity by comparing the current identification results with the existing inventory information in the database, and writes the updates to the inventory information table, while also recording the update time and possible usage period.

[0477] Output: Updated inventory information records, including the type, quantity, and related status of the target object.

[0478] Step 4: The terminal collects user emotion-related data and sends it to the server.

[0479] Input: User's facial image, voice clip, or text input.

[0480] With user authorization, the terminal uses a camera to obtain facial images or short videos of the user, a microphone to obtain voice data, or directly reads the text content entered by the user in the chat box or comment box.

[0481] The terminal compresses and encodes images and audio, adds user identifiers and timestamps, and transmits emotion-related data to the server via the network.

[0482] Output: A message containing user identification and sentiment observation data (images, audio, or text).

[0483] Step 5: The server analyzes emotion-related data and generates emotion information.

[0484] Input: Facial images, voice data, or text data from the terminal.

[0485] The server uses a sentiment analysis module to extract features from facial images, inputs the images into a pre-trained convolutional neural network model, and outputs probability distributions representing different emotion categories (such as joy, fatigue, depression, anger, etc.). For speech data, the server extracts acoustic features such as pitch, volume, and speech rate, inputs them into a sequence model for temporal analysis, and outputs emotion classification results. For text data, the server uses a language model to perform sentiment polarity analysis and emotion label determination.

[0486] The server uses a weighted fusion algorithm or Bayesian inference method to integrate the multi-source emotion judgment results into a main emotion label, calculate the corresponding confidence level, and write the result into the emotion information record table.

[0487] Output: Sentiment information including user sentiment labels and confidence levels.

[0488] Step 6: The server integrates conditional information, inventory information, preference information, and sentiment information to generate prompts.

[0489] Inputs: Conditional and preference information from step 1, inventory information from step 3, and sentiment information from step 5.

[0490] The server internally selects a template based on the task type (single recipe recommendation or multi-day meal plan), reads the target objects with a large quantity or about to expire from the inventory information, and combines them into a "current ingredients" list; the server reads the user's health goals, time limits, allergies, and other conditions, and converts them into natural language descriptions; the server adjusts the tone keywords based on emotion tags, for example, adding words such as "simple" and "time-saving" when the user is fatigued, and adding words such as "celebration" and "special" when the user is joyful.

[0491] The server concatenates the above content into a coherent natural language text according to a predefined template, which serves as a prompt for the generative artificial intelligence model.

[0492] Output: Prompt statements containing task description, inventory description, conditional constraints, and emotional context.

[0493] Step 7: The server sends prompts to the generative artificial intelligence model and obtains dietary plans.

[0494] Input: Prompt statement from step 6.

[0495] The server sends the prompts as input text to the model through the generative artificial intelligence model interface and sets parameters (such as maximum generation length, randomness temperature, etc.). Based on its internal multi-layer attention mechanism and language modeling structure, the generative artificial intelligence model encodes and decodes the prompts to generate natural language descriptions of multiple recipes or meal plans.

[0496] The server receives the text data output by the model and extracts the dish name, required components, main steps, and estimated cooking time from it using algorithms such as keyword search, pattern matching, and paragraph segmentation, transforming it into a structured dietary plan record.

[0497] Output: One or more structured meal plans, including a list of dishes and a list of required components.

[0498] Step 8: The server determines the deficient components in the diet plan.

[0499] Input: Dietary plan information from step 7 and inventory information from step 3.

[0500] The server iterates through the required components of each dish in each diet plan, and looks up the corresponding item in the inventory information for each component. The server compares the required quantity with the inventory quantity, calculates the difference using numerical subtraction, and if the difference is greater than zero, the component is recorded as a insufficient component, and the insufficient quantity is also recorded.

[0501] The server merges all deficient elements, removes duplicate entries, generates a list of deficient elements, and stores it in association with the corresponding dietary plan.

[0502] Output: A list of deficiencies and their amounts for each dietary plan.

[0503] Step 9: The server generates notification content and expression based on emotion information.

[0504] Input: List of deficiencies from step 8, dietary plan information from step 7, and emotional information from step 5.

[0505] The server selects different notification templates and language styles based on emotion tags: when the emotion is joy, the server adds positive and celebratory expressions to the text; when the emotion is fatigue or depression, the server emphasizes words like "simple," "easy," and "comforting"; when the emotion is anger, the server uses more gentle, neutral, and non-commanding expressions.

[0506] The server embeds the name and key features of the diet plan, as well as the names and recommended supplements of any deficiencies, into the template to generate a complete notification text. Alternatively, the server can send a preliminary factual description to a generative AI model to request text polishing, and then use the polished text as the final notification content.

[0507] Output: The notification text for the current user and its expression parameters (such as tone, length control, etc.).

[0508] Step 10: The server outputs dietary plans and deficiencies to users through the terminal.

[0509] Input: Dietary plan information from step 7 and notification content from step 9.

[0510] The server encapsulates the diet plan and notification content into a response message and sends it to the terminal. The terminal parses the message and displays multiple diet plans on the interface in the form of cards or lists. For each plan, it displays the dish name, estimated cooking time, and health label. The terminal lists the required components on the plan details page and prominently marks any missing components.

[0511] The terminal displays notification text from the server at the top of the interface or in a pop-up prompt box, allowing users to intuitively understand the reasons for the recommendation and the currently missing components.

[0512] Output: A list of dietary plans and prompts for deficiencies are displayed on the terminal device.

[0513] Step 11: Users select a dietary plan based on the information displayed on the terminal and decide whether to supplement the deficient components.

[0514] Input: The dietary plan, deficiency component prompts, and notification content displayed on the terminal interface.

[0515] Users can browse recommended diet plans on the terminal, click to select a plan as the actual preparation target, or return to reset the conditions; for any missing components, users can choose to purchase manually or place an order for external services through the buttons provided on the terminal.

[0516] The user's clicks and selections on the terminal are recorded and sent to the server as feedback information for subsequent preference updates.

[0517] Output: Identification of the user's selected diet plan and information indicating whether the order was made through an external service provider.

[0518] Step 12: The server generates subscription information for external services and sends it via a communication device.

[0519] Input: List of insufficient components from step 8 and user order instruction information from step 11.

[0520] The server extracts the name and required quantity of each component from the list of insufficient components, and generates a standardized order information record by combining it with the delivery address and contact information registered by the user in the system. The server then selects an appropriate external service interface according to the cooperation rules and converts the order information into the request format required by that interface.

[0521] The server uses communication devices to send order information requests to external providers and receives order confirmation results, including order number, estimated delivery time, and cost information. The server records the order results in the database and notifies the user of the order status through the terminal.

[0522] Output: Order information successfully submitted to external service providers, along with corresponding order confirmation data.

[0523] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires sound representing user input regarding the result of the specific processing. The control unit 46A sends the sound data representing user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0524] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0525] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0526] For example, the collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart device 14 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0527] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart device 14.

[0528] Second Implementation Method Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0529] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. A server can be cited as an example of the data processing device 12.

[0530] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0531] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and communication I / F 44 are also connected to the bus 52.

[0532] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0533] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0534] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0535] Figure 4 This illustrates an example of the main functions of the data processing device 12 and the smart glasses 214. For example... Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0536] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0537] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. In the emotion inference function (emotion-specific function) using the emotion-specific model 59, various inferences and predictions related to the user's emotions are performed, including inferences and predictions of the user's emotions, but this is not limited to this example. Furthermore, emotion inference and prediction may also include, for example, emotion analysis (parsing).

[0538] In the smart glasses 214, the processor 46 performs reception and output processing. The memory 50 stores the reception and output program 60. The processor 46 reads the reception and output program 60 from the memory 50 and executes the read reception and output program 60 on the RAM 48. The reception and output processing is implemented by the processor 46 operating as a control unit 46A according to the reception and output program 60 executed on the RAM 48. Furthermore, the smart glasses 214 has the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and these models can also be used to perform the same processing as the specific processing unit 290.

[0539] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0540] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0541] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0542] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0543] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0544] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A outputs the result of the specific processing to the speaker 240. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0545] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0546] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0547] For example, the collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the smart glasses 214 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0548] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the smart glasses 214.

[0549] Third Implementation Method Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0550] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. A server can be cited as an example of the data processing device 12.

[0551] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, display 343, and communication I / F 44 are also connected to the bus 52.

[0553] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the user 20's surroundings (e.g., the field of view defined by an angle equivalent to the field of vision of an average healthy person).

[0555] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0556] Figure 6 This illustrates an example of the main functions of the data processing device 12 and the head-mounted terminal 314. For example... Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0557] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0558] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0559] In the head-mounted terminal 314, the processor 46 performs the acceptance / output processing. The memory 50 stores the acceptance / output program 60. The processor 46 reads the acceptance / output program 60 from the memory 50 and executes the read acceptance / output program 60 on the RAM 48. The acceptance / output processing is implemented by the processor 46 operating as a control unit 46A according to the acceptance / output program 60 executed on the RAM 48.

[0560] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the head-mounted terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the head-mounted terminal 314 will be referred to as the "terminal".

[0561] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0562] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0563] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0564] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0565] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A outputs the result of the specific processing to the speaker 240 and the display 343. The microphone 238 acquires sound input representing the user's input regarding the result of the specific processing. The control unit 46A sends the sound data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0566] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 includes prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0567] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0568] For example, the collection unit is implemented by the control unit 46A of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the head-mounted terminal 314 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 to analyze the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 to generate a menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the head-mounted terminal 314 or the specific processing unit 290 of the data processing device 12 to provide the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0569] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the head-mounted terminal 314.

[0570] Fourth Implementation Method Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0571] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. A server can be cited as an example of the data processing device 12.

[0572] The data processing apparatus 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" as understood in this disclosure. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0573] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, controlled object 443, and communication I / F 44 are also connected to the bus 52.

[0574] Microphone 238 receives instructions from user 20 by receiving sounds emitted by user 20. Microphone 238 captures sounds emitted by user 20 and converts the captured sounds into sound data, which is then output to processor 46. Speaker 240 outputs sound according to instructions from processor 46.

[0575] Camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, to photograph the area around robot 414 (e.g., the field of view defined by a perspective equivalent to the field of vision of an average healthy person).

[0576] Communication I / F44 is connected to network 54. Communication I / F44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F44 and 26 is performed in a secure state.

[0577] The controlled object 443 includes a display device, LEDs (light-emitting diodes) for the eyes, and motors for driving the arms, hands, and feet. The posture or movement of the robot 414 is controlled by controlling the motors in the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0578] Figure 8 This illustrates an example of the main functions of the data processing device 12 and the robot 414. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The specific processing program 56 is stored in the memory 32.

[0579] The specific processing program 56 is an example of a "program" involved in the technology of this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0580] The memory 32 stores the data generation model 58 and the emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290.

[0581] In robot 414, the processor 46 performs the acceptance and output processing. The memory 50 stores the acceptance and output program 60. The processor 46 reads the acceptance and output program 60 from the memory 50 and executes the read acceptance and output program 60 on RAM 48. The acceptance and output processing is implemented by the processor 46 acting as the control unit 46A according to the acceptance and output program 60 executed on RAM 48.

[0582] Next, the specific processing of the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal".

[0583] Example 1 The process is the same as that of the specific process described in Embodiment 1 in the first embodiment above, so the description is omitted.

[0584] Application Example 1 The process is the same as that in the specific processing described in Application Example 1 of the first embodiment above, so the description is omitted.

[0585] Example 2 The process is the same as that of the specific process in Embodiment 2 described in the first embodiment above, so the description is omitted.

[0586] Application Example 2 The process is the same as that in the specific processing described in Application Example 2 of the first embodiment above, so the description is omitted.

[0587] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A outputs the result of the specific processing to the speaker 240 and the controlled object 443. The microphone 238 acquires sound input representing the result of the specific processing. The control unit 46A sends the sound data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the sound data.

[0588] Data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of data generation models 58 include ChatGPT (registered trademark) (accessible via the internet (URL: https: / / openai.com / blog / chatgpt)). Data generation model 58 is obtained through deep learning on a neural network. Input to data generation model 58 are prompt words containing instructions, and inference data such as sound data representing sound, text data representing text, and image data representing images (e.g., still image data or animation data). Data generation model 58 infers from the input inference data based on the instructions represented by the prompt words and outputs the inference result in one or more data forms, such as sound data, text data, and image data. Data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or induction. The specific processing unit 290 performs the aforementioned specific processing while using data generation model 58. The data generation model 58 can also be a model finely tuned to output inference results from prompts that do not contain instructions. In this case, the data generation model 58 can output inference results based on prompts that do not contain instructions. The data processing apparatus 12, etc., includes various data generation models 58, including AI other than the generation AI. AI other than the generation AI can be, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes, and can perform various processes, but is not limited to this example. Furthermore, the AI ​​can also be an AI agent. Furthermore, when the processing of the above-mentioned parts is performed by AI, the processing can be performed partially or entirely by AI, but is not limited to this example. Furthermore, the processing performed by the AI ​​including the generation AI can be replaced by processing in the rule base, and the processing in the rule base can also be replaced by processing performed by the AI ​​including the generation AI.

[0589] Furthermore, the processing of the aforementioned data processing system 10 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be performed by both the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0590] For example, the collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit uses the camera 42 or communication I / F 44 of the robot 414 to acquire step data, which is then processed by the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12, which analyzes the data from the collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12, which uses a generation AI to generate a menu. For example, the serving unit is implemented by the speaker 240 of the robot 414 and the control object 443 or the specific processing unit 290 of the data processing device 12, which provides the generated menu to the user. The correspondence between each unit and the device or control unit is not limited to the above examples and various changes can be made.

[0591] In the above embodiments, examples of specific processing by the data processing device 12 are given, but the technology disclosed herein is not limited to this, and specific processing may also be performed by the robot 414.

[0592] Furthermore, the emotion-specific model 59, acting as an emotion engine, can determine a user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine a user's emotion based on an emotion graph that serves as a specific mapping (see [reference]). Figure 9 The emotion-specific model 59 can also determine the robot's emotion, and the specific processing unit 290 performs specific processing based on the robot's emotions.

[0593] Figure 9 This is a diagram representing an emotion map 400 that maps multiple emotions. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotion is. On the outer side of the concentric circles, emotions representing states or behaviors arising from mood are arranged. Emotions are concepts that include feelings and mental states. Emotions generated by reactions occurring in the brain are arranged roughly to the left of the concentric circles. Emotions derived from situational judgments are arranged roughly to the right of the concentric circles. Emotions generated by reactions occurring in the brain and derived from situational judgments are arranged roughly above and below the concentric circles. Furthermore, "pleasant" emotions are arranged above the concentric circles, and "unpleasant" emotions are arranged below them. Thus, in the emotion map 400, multiple emotions are mapped based on the structure that generates emotions, and emotions that are likely to occur simultaneously are mapped close to each other.

[0594] These emotions are distributed at the three o'clock position of the emotion map 400, typically fluctuating between peace and anxiety. In the right half of the emotion map 400, situational awareness dominates over internal sensation, thus resulting in an impression of calm.

[0595] The inner side of the emotion map 400 represents the inner state, while the outer side represents behavior. Therefore, the further outward you are from the emotion map 400, the more visible the emotion becomes (manifested in behavior).

[0596] Here, human emotions are based on various balances such as posture and blood sugar levels. When these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotions in robots, cars, motorcycles, etc., can also be created in the following way: based on various balances such as posture and remaining battery power, when these balances deviate from an ideal state, it indicates an unpleasant state; when they approach the ideal state, it indicates a pleasant state. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a Brain Physiological Signal Analysis System for Voice Emotion Recognition and Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the sensory-dominated region, called "response," are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the situational cognition-dominated region, called "situation," are arranged.

[0597] In the emotion map, two types of emotions that promote learning are defined. One is a negative emotion on the situational side, in the middle or peripheral region of "repentance" or "reflection." This occurs when the robot experiences negative emotions such as "I don't want to experience this feeling again" or "I don't want to be blamed again." The other is a positive emotion on the response side, near the "desire" region. This occurs when there are positive feelings such as "wanting more" or "wanting to know more."

[0598] The emotion-specific model 59 inputs user input into a pre-trained neural network to obtain emotion values ​​representing each emotion shown in the emotion map 400, thereby determining the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network... Figure 10 As shown in the sentiment graph 900, it was trained in a way that sentiments that are configured close to each other have similar values. Figure 10 The text shows examples of emotions such as "peace of mind", "stability", and "reassurance" that have similar emotion values.

[0599] The above description focuses on the functions of the data processing device 12, but the system of this disclosure is not necessarily installed on a server. The system of this disclosure can also be installed as a general information processing system. This disclosure can also be installed, for example, as a software program running on a personal computer, an application running on a smartphone, etc. The method of this disclosure can also be provided to users in the form of SaaS (Software as a Service).

[0600] In the above embodiments, an example of a specific process being performed by a single computer 22 is given. However, the technology disclosed herein is not limited to this, and the specific process can also be distributed among multiple computers, including computer 22. For example, the data generation model 58 can be located on an external device of the data processing apparatus 12, where data is generated based on the input data.

[0601] In the above embodiments, examples of storing a specific processing program 56 in the memory 32 have been described, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0602] Alternatively, a specific processing program 56 may be pre-stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 according to the requirements of the data processing device 12.

[0603] In addition, it is not necessary to store all the specific processing program 56 in the storage device such as the server connected to the data processing device 12 via the network 54 or in the memory 32; a portion of the specific processing program 56 may be stored in advance.

[0604] As hardware resources for performing specific processes, various processors, as shown below, can be used. For example, a CPU can be listed as a processor, which functions as a general-purpose processor that performs specific processes by executing software, i.e., a program. Furthermore, processors can be listed as special-purpose circuits such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application-Specific Integrated Circuits), which are processors with circuitry specifically designed to perform specific processes. Each processor has built-in or connected memory, and each processor executes specific processes using that memory.

[0605] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resources for performing a specific process can be a single processor.

[0606] As an example of a single processor, there are two approaches: First, a processor is composed of a combination of one or more CPUs and software, which functions as a hardware resource to perform a specific process; second, as represented by a SoC (System-on-a-chip), a processor is used to implement the functionality of the entire system, which includes multiple hardware resources for performing a specific process, using a single IC (Integrated Circuit) chip. In this way, the specific process is implemented by using one or more of the aforementioned processors as hardware resources.

[0607] Furthermore, the hardware architecture of these various processors, more specifically, can utilize circuits that combine semiconductor elements and other circuit components. Moreover, the specific process described above is just one example. Therefore, without departing from the main point, unnecessary steps can certainly be deleted, new steps added, or the processing order changed.

[0608] The descriptions and illustrations above are detailed explanations of a portion of the technology disclosed herein, and are merely one example of the technology disclosed herein. For example, the above descriptions of the structure, function, effect, and results are just one example of the structure, function, effect, and results of a portion of the technology disclosed herein. Therefore, without departing from the spirit of the technology disclosed herein, unnecessary parts may be deleted, new elements added, or replacements may be made to the descriptions and illustrations above. Furthermore, to avoid confusion and facilitate understanding of a portion of the technology disclosed herein, explanations of common technical knowledge that do not require special explanation under the premise of being able to implement the technology disclosed herein have been omitted from the descriptions and illustrations above.

[0609] All documents, patent applications and technical specifications set forth in this specification are incorporated herein by reference to the same extent that each document, patent application and technical specification is specifically and individually described therein and referenced by reference.

[0610] In addition, the following notes are provided in response to the above explanation.

[0611] Example 1 (Note 1) An information processing system, characterized in that it comprises: A condition information receiving unit that operates on an information processing device and receives cooking-related condition information from a user through an information input / output section; An object information acquisition unit is used to acquire information about objects in a storage container by using detection elements, and to acquire the information about the objects as object type information and usage period information. Based on the condition information and the information of the object, the condition information is parsed by attribute, and the object information in the storage container is integrated with the user's historical selection record information to form a prompt statement for input to the generative artificial intelligence model. The prompt statement includes a prompt statement generation unit that includes health conditions, time conditions, preference conditions and object usage priority conditions. A menu information acquisition unit is used to input the prompt statement generated by the prompt statement generation unit into the generative artificial intelligence model, instruct the generative artificial intelligence model to generate cooking suggestion information, and acquire the cooking suggestion information as menu information containing multiple cooking schemes; A menu information selection unit for evaluating the menu information based on the usage period information and the historical selection record information, and selecting menu information from the menu information that helps reduce food waste and resource consumption; Menu information presentation unit for presenting the selected menu information to the user through the information input / output unit.

[0612] (Note 2) The information processing system according to Appendix 1 is characterized in that, The prompt statement generation unit is configured to parse the health status information and nutritional condition information contained in the condition information, and explicitly embed fat content restriction information, salt content restriction information and nutrient balance information in the prompt statement, thereby instructing the generative artificial intelligence model to generate health-oriented menu information.

[0613] (Note 3) The information processing system according to Appendix 1 is characterized in that, The prompt statement generation unit and the menu information selection unit are configured to, based on the usage period information of the objects in the storage container and the user's historical cooking record information stored in the information processing device, attach priority utilization conditions for suitable objects and conditions related to shortening cooking time to the prompt statement, and select menu information that helps reduce the amount of object waste and optimize the consumption of resources such as electricity from multiple menu information obtained from the generative artificial intelligence model, thereby providing users with economic benefits and convenience.

[0614] Application Example 1 (Note 1) An information processing system, characterized in that it comprises: A device for obtaining user condition data related to a menu from a user via an information input / output device, the user condition data including health status, object attributes, time conditions, cost conditions, and allergy information; A device for acquiring information data of stored food ingredients from a detection device installed in a storage device, the food ingredient information data including the type, quantity and shelf life of the food ingredients, and standardizing the unit of measurement and name of the food ingredient information data to generate standardized food ingredient data. An apparatus for generating a prompt statement based on the user condition data and the standardized ingredient data, the prompt statement including a condition description in natural language, an ingredient description in natural language, and an instruction description for specifying the output format of a generative artificial intelligence model. A device for inputting the prompt statement into a generative artificial intelligence model to obtain menu candidate information, and extracting the required ingredient data for each menu from the menu candidate information; An apparatus for comparing the required ingredient data with the standardized ingredient data to determine the ingredients that are already available and the ingredients that are insufficient for each menu, and for calculating the purchase candidate items and recommended purchase quantities for the insufficient ingredients based on a predetermined product information database. An apparatus for generating menu candidate screen data for display on a user terminal based on the menu candidate information, the insufficient ingredients, and the recommended purchase quantity, and for receiving the user's selection of the menu and changes to the purchase quantity of the ingredients in the menu candidate screen, thereby generating order data. An apparatus for calculating delivery time based on receipt information, time conditions, and purchased ingredients information contained in the order data, using information from multiple supply locations and delivery routes, thereby determining the supply locations and delivery conditions that can complete delivery within a predetermined time, storing the order information in an order information database, and sending the order data to a delivery service device.

[0615] (Note 2) The information processing system according to Appendix 1 is characterized in that, The information processing device is also used to compare the required ingredients of each menu with the standardized ingredient data based on the menu candidate information, evaluate the menu candidates that prioritize the use of ingredients with closer expiration dates in the storage device, and dynamically adjust the condition description in the prompt statement to increase the amount of ingredients with closer expiration dates used in the menu candidates generated by the generative artificial intelligence model, thereby reducing food waste.

[0616] (Note 3) The information processing system according to Appendix 1 is characterized in that, The information processing device is also used to reflect the user's cost conditions and energy consumption conditions in the order data. When determining the supply point and the delivery conditions, it uses food price information and information indicating the power consumption or fuel consumption of each delivery route to calculate the evaluation value of cost and energy consumption, and selects the supply point and delivery conditions that make the evaluation value meet the predetermined conditions.

[0617] Example 2 (Note 1) An information processing system, characterized in that it comprises: A unit for executing by an information processing device and obtaining conditional information and prompts from the user through an information input / output device; This unit is used to acquire image data and physical quantity data from a storage device equipped with an imaging device and a detection device, and to parse the image data with an image processing program and a trained recognition model to obtain specific item type information, and to parse the physical quantity data with a numerical processing program to obtain specific item quantity information and status information, thereby generating an inventory information unit containing the item type information, item quantity information and status information. A unit for receiving image data representing transaction information, converting the image data into string data using a character recognition program, and extracting purchase item information from the string data to update the inventory information; A unit for generating model input data containing descriptive information about the items in the storage device and the prompt statements based on the inventory information and the user's condition information, and generating prompt data containing instruction information for instructing the generative artificial intelligence model to generate menu schemes based on the model input data; A unit for acquiring menu scheme data output by the generative artificial intelligence model, comparing the required item information contained in the menu scheme data with the inventory information to extract insufficient item information, and generating output data for prompting the user with the menu scheme data and the insufficient item information.

[0618] (Note 2) According to the information processing system described in Appendix 1, the information processing device is configured to reflect health condition information and nutritional condition information obtained from the user into the model input data and the prompt data, thereby instructing the generative artificial intelligence model to generate a menu scheme that meets health considerations, and to include health-related information and nutrition-related information in the menu scheme data.

[0619] (Note 3) According to the information processing system described in Appendix 1, the information processing device is configured to: generate a menu scheme that prioritizes the use of items with a higher risk of waste based on the shelf life information and environmental status information of each item contained in the inventory information; and calculate utilization plan information that takes into account power usage conditions based on the operating condition information of the storage device and associated devices, thereby generating data to prompt the user about the economic benefits obtained by reducing food waste and optimizing power consumption.

[0620] Application Example 2 (Note 1) An information processing system, characterized in that it comprises: A device for receiving dietary-related conditional and preference information from a user via an information input / output device; A device for identifying the type and quantity of target objects in the storage device based on image data and detection data acquired by an imaging device and a detection device disposed in the storage device, and recording and updating the target objects as inventory information to the recorded information; A device for inferring an emotional state based on a user’s facial expression data, voice data, or text data, and for acquiring the emotional state as emotional information; Apparatus for generating prompt statements to instruct a generative artificial intelligence model to generate at least one dietary plan based on the inventory information, the condition information, the preference information, and the emotion information; A device for inputting the prompt statement into the generative artificial intelligence model and obtaining the dietary plan output by the generative artificial intelligence model; A means for comparing the required components included in the diet plan with the inventory information, and identifying the components that are in short supply from the required components as insufficient components. An apparatus for determining the content and expression of a notification based on the deficient elements and the emotional information, and for notifying the user of the dietary plan and the deficient elements through the information input / output device; An apparatus for generating ordering information for external services corresponding to the deficient constituent elements, and for sending the ordering information to the external service provider via a communication device.

[0621] (Note 2) The information processing system according to Appendix 1 is characterized in that, The device for generating the prompt statement is configured to receive the user's health status, nutritional conditions, cooking time conditions, or energy intake conditions as conditional information, and reflect the conditional information in the prompt statement, thereby instructing the generative artificial intelligence model to generate a health-oriented dietary plan, and to consider the usage period and inventory of the target object in the storage device when determining the dietary plan, so as to optimize power consumption while reducing food waste.

[0622] (Note 3) The information processing system according to Appendix 1 is characterized in that, The device for generating the prompt statement and determining the notification content is configured to modify the content and style of the prompt statement based on the emotional information. Specifically, when the emotional information indicates a joyful state, a celebratory diet plan is generated; when the emotional information indicates a fatigued or depressed state, a comforting diet plan with simple cooking steps is generated; and when the emotional information indicates an angry state, notification content with a mild expression is generated. The corresponding information is then provided to the user through the generative artificial intelligence model and the information input / output device.

Claims

1. An information processing system, characterized in that, include: processor; The processor is configured as follows: Receive user conditions through an interface for receiving conditional input from the user; Use sensors to obtain information about the food inside the refrigerator; Based on the acquired ingredient information and the user conditions, a prompt message is generated to instruct the generative artificial intelligence model to generate a contribution.

2. The information processing system according to claim 1, characterized in that, The processor is configured to reflect health-related conditions from the user in the prompt message and generate prompt messages to instruct a generative artificial intelligence model to generate health donations.

3. The information processing system according to claim 1, characterized in that, The processor is configured to consider the shelf life of food in the refrigerator when generating a donation to reduce food waste and to provide economic benefits to users by optimizing power consumption.

Images