System

Abstract

To provide a system according to an embodiment that analyzes pictures, and prepares an optimal menu table efficiently.SOLUTION: A system according to an embodiment comprises: a reception unit; an identification unit; a layout unit; and a proposal unit. The reception unit is configured to upload the picture desired to post in a menu. The identification unit is configured to analyze the picture uploaded by the reception unit, and identify a kind of cooking, or characteristic thereof. The layout unit is configured to prepare a cooking order of the menu or a layout thereof on the basis of information identified by the identification unit. The proposal unit is configured to propose an explanation of the cooking on the basis of the menu table prepared by the layout unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional techniques, the analysis of photos and optimization of layout when creating menu tables is time-consuming and difficult to do efficiently.

[0005] The system according to the embodiment aims to analyze photographs and efficiently create an optimal menu list. [Means for solving the problem]

[0006] The system according to the embodiment includes a reception unit, an identification unit, a layout unit, and a suggestion unit. The reception unit uploads photos to be included in the menu. The identification unit analyzes the photos uploaded by the reception unit and identifies the type or characteristics of the food. The layout unit creates a menu order or layout based on the information identified by the identification unit. The suggestion unit suggests descriptions of the food based on the menu list created by the layout unit. Effect of the Invention

[0007] The system according to the embodiment can analyze photos and efficiently create an optimal menu list. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The image analysis system according to the embodiment of the present invention is a system that creates an original menu table simply by uploading a photo to be included in the menu. In this image analysis system, a user uploads a photo to be included in the menu, and an image analysis AI analyzes the uploaded photo and creates a menu table that takes into account the order and layout of the menu. Furthermore, the AI ​​also suggests descriptions of dishes based on prior information and photos. For example, when a user uploads photos of appetizers, main dishes, desserts, etc., the image analysis AI analyzes the contents of each photo and identifies the type and characteristics of the dish. Next, the image analysis AI creates an easy-to-read menu table by arranging the dishes in the order of appetizers, main dishes, and desserts. In addition, the image analysis AI arranges the photos of each dish in an appropriate position to achieve a visually attractive layout. Furthermore, the image analysis AI suggests descriptions such as "healthy salad made with fresh vegetables" from a photo of a salad and "juicy steak served with special sauce" from a photo of a steak. This allows users to easily create attractive menu tables. This allows the image analysis system to easily create original menu tables for users, and the order and layout of the menu and descriptions of the dishes are automatically suggested, significantly reducing time and effort.

[0029] The image analysis system according to the embodiment includes a reception unit, a specification unit, a layout unit, and a proposal unit. The reception unit accepts uploading of photos that a user wants to put on a menu. The user can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos into the image analysis AI. The specification unit uses image analysis technology to analyze the uploaded photos and specify the type and characteristics of the dishes. For example, the specification unit specifies salad or soup from a photo of an appetizer, steak or pasta from a photo of a main dish, and cake or ice cream from a photo of a dessert. The specification unit analyzes the content of each photo using image recognition technology or machine learning algorithms. The layout unit considers the order and layout of the menu based on the information specified by the specification unit. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. In addition, the layout unit arranges the photos of each dish in an appropriate position to realize a visually attractive layout. The layout unit determines the order and layout of the menu taking into account the type and popularity of the dishes, the season, and the like. The suggestion unit suggests a description of the dishes based on the menu table created by the layout unit. The suggestion unit suggests a description of the dishes based on information collected from a preregistered database or the Internet. For example, the suggestion unit suggests a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with special sauce" from a photo of a steak. In this way, the image analysis system according to the embodiment automatically suggests the order and layout of the menu and the description of the dishes simply by the user uploading a photo. In this way, the image analysis system allows the user to easily create an original menu table, and since the order and layout of the menu and the description of the dishes are automatically suggested, it is possible to significantly reduce time and labor.

[0030] The reception unit accepts the upload of photos that users want to include in the menu. For example, users can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos into the image analysis AI. Specifically, when a user uploads photos through a dedicated web interface or mobile app, the reception unit quickly processes the photos and checks the format and resolution of the image files. In addition, the reception unit extracts metadata (e.g., the date and time of shooting and location information) from the uploaded photos and uses them as auxiliary information for analysis. The reception unit allows users to upload multiple photos at once and also provides a function to add tags and comments to each photo. This allows users to easily describe the content and features of the photos, making the subsequent analysis process proceed more smoothly. The reception unit temporarily stores the uploaded photos in cloud storage so that the image analysis AI can access them. In addition, the reception unit manages the user's account information and past upload history to make them easy to reuse and refer to. In this way, the reception unit plays a role in allowing users to easily upload photos and improving the efficiency of the entire system.

[0031] The identification unit uses image analysis technology to analyze the uploaded photos and identify the type and characteristics of the food. For example, the identification unit identifies salads and soups from photos of appetizers, steaks and pasta from photos of main dishes, and cakes and ice cream from photos of desserts. The identification unit uses image recognition technology and machine learning algorithms to analyze the content of each photo. Specifically, the identification unit uses a deep learning model to extract image features and compares them with a pre-trained database to identify the type of food with high accuracy. For example, a convolutional neural network (CNN) is used to analyze features such as edges, textures, and colors of an image, and based on this, the type of food is classified. Furthermore, the identification unit analyzes not only the characteristics of the food, but also information on the presentation and background to identify detailed characteristics of the food. For example, from a photo of a salad, the type of vegetables used and the type of dressing are identified, and from a photo of a steak, the degree of doneness and the type of sauce are identified. Based on this information, the identification unit registers detailed characteristics of the food in a database and makes them available for subsequent processes. Furthermore, the identification unit modifies the analysis results based on feedback from users to continuously improve the accuracy of the model. This enables the identification unit to analyze the uploaded photo with high accuracy and quickly identify the type and characteristics of the food.

[0032] The layout unit considers the order and layout of the menu based on the information specified by the specification unit. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges the photos of each dish in an appropriate position to achieve a visually attractive layout. Specifically, the layout unit automatically generates menu sections based on the types and characteristics of the dishes provided by the specification unit, and arranges the dishes corresponding to each section. For example, salads and soups are arranged in the appetizer section, steaks and pasta in the main dish section, and cakes and ice cream in the dessert section. The layout unit also optimizes the order and layout of the menu in consideration of the popularity and season of the dishes, the user's preferences, and the like. For example, seasonal dishes and menus that correspond to special events are arranged preferentially to attract the user's attention. The layout unit also arranges the photos of each dish in an appropriate size and position to achieve a visually balanced layout. For example, by displaying the photos of the dishes in a large size, the visual impact is emphasized and the user's appetite is stimulated. The layout unit also adjusts the design, font, color, and the like of the menu to create an overall sense of unity. This allows the layout unit to automatically create an easy-to-read and attractive menu table, thereby improving user satisfaction.

[0033] The suggestion unit proposes a description of the dish based on the menu table created by the layout unit. The suggestion unit proposes a description of the dish based on information collected from a database registered in advance or from the Internet. For example, the suggestion unit proposes a description such as "a healthy salad made with fresh vegetables" from a photo of a salad, and "a juicy steak served with a special sauce" from a photo of a steak. Specifically, the suggestion unit searches the database for a related description based on the type and characteristics of the dish provided by the identification unit, and automatically generates an appropriate description. For example, when a photo of a salad is identified, the suggestion unit generates a specific description such as "a healthy salad made with fresh lettuce, tomatoes, and cucumbers and finished with a special dressing." Furthermore, the suggestion unit collects the latest cooking information and trends from the Internet and proposes a description based on this. For example, in accordance with the recent health-conscious trend, it is possible to add a description such as "a salad made with organic vegetables" or "used a special gluten-free dressing." The suggestion unit modifies the description based on user feedback to provide a more appropriate and attractive description. In addition, the suggestion unit can provide not only a description of the dish, but also detailed information such as the background and history of the dish, cooking method, etc. This allows the user to fully convey the appeal of the dish through the menu, thereby increasing the effect of attracting the interest of customers.

[0034] The identification unit can identify the type and characteristics of a dish using image analysis technology. The identification unit, for example, uses image analysis technology to analyze the uploaded photo and identify the type and characteristics of a dish. For example, the identification unit identifies salad or soup from a photo of an appetizer, steak or pasta from a photo of a main dish, and cake or ice cream from a photo of a dessert. The identification unit analyzes the content of each photo using image recognition technology or machine learning algorithms. In this way, the type and characteristics of a dish can be accurately identified using image analysis technology. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input the uploaded photo to the generation AI and cause the generation AI to identify the type and characteristics of a dish.

[0035] The layout unit can determine the order or layout of the menu based on at least one of the type or popularity of the food, and the season. The layout unit determines the order or layout of the menu, for example, taking into consideration the type or popularity of the food, the season, etc. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges photos of each dish in an appropriate position to achieve a visually attractive layout. The layout unit determines the order or layout of the menu, for example, taking into consideration the type or popularity of the food, the season, etc. As a result, a visually attractive menu table can be created by taking into consideration the type or popularity of the food, the season, etc. A part or all of the above-mentioned processing in the layout unit may be performed, for example, using AI, or may be performed without using AI. For example, the layout unit can input information such as the type or popularity of the food, the season, etc. to the generation AI and cause the generation AI to determine the order or layout of the menu.

[0036] The suggestion unit can suggest a dish description based on information collected from a pre-registered database or the Internet. The suggestion unit suggests a dish description based on information collected from a pre-registered database or the Internet, for example. For example, the suggestion unit suggests a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with special sauce" from a photo of a steak. The suggestion unit suggests a dish description based on information collected from a pre-registered database or the Internet. This allows accurate suggestions of dish descriptions based on information from a pre-registered database or the Internet. Some or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input information collected from a pre-registered database or the Internet to a generation AI, and cause the generation AI to execute suggestions of dish descriptions.

[0037] The reception unit can accept that the user uploads photos of multiple dishes. For example, the reception unit accepts that the user uploads photos of multiple dishes. The user can upload photos of multiple dishes such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos to an image analysis AI. This allows the user to upload photos of multiple dishes. A part or all of the above-mentioned processing in the reception unit may be performed using an AI, for example, or may be performed without using an AI. For example, the reception unit can input photos of multiple dishes uploaded by the user to a generation AI and cause the generation AI to analyze the photos.

[0038] The suggestion unit can provide a description of a dish to the user. The suggestion unit, for example, provides a description of a dish to the user. The suggestion unit suggests a description of a dish based on information collected from a pre-registered database or the Internet. For example, the suggestion unit suggests a description such as "a healthy salad made with fresh vegetables" from a photo of a salad, or "a juicy steak served with a special sauce" from a photo of a steak. This improves the appeal of the menu by providing a description of the dish to the user. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit may input information collected from a pre-registered database or the Internet to the generation AI, and cause the generation AI to execute a suggestion of a description of the dish.

[0039] The reception unit can analyze the user's past upload history and select an appropriate upload method. The reception unit, for example, analyzes the user's past upload history and selects the optimal upload method. For example, the reception unit automatically displays the types of photos that the user frequently uploaded in the past as candidates. The reception unit can also preferentially suggest the upload method (audio, text, etc.) that the user used in the past. The reception unit can also predict and suggest the upload method to be used in a specific time period from the user's past upload history. In this way, the optimal upload method can be provided by analyzing the past upload history. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's past upload history data to the generation AI and cause the generation AI to select the optimal upload method.

[0040] The reception unit can perform filtering based on the user's current project and field of interest when uploading a photo. The reception unit performs filtering based on the user's current project and field of interest when uploading a photo, for example. For example, the reception unit preferentially uploads photos related to a project currently in progress by the user. The reception unit can also automatically filter photos with high relevance based on the user's field of interest. The reception unit can also preferentially display related photos when the user uploads photos based on a specific theme. This allows highly relevant photos to be uploaded by filtering based on the user's current project and field of interest. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input data on the user's current project and field of interest to the generation AI and cause the generation AI to perform filtering.

[0041] The reception unit can prioritize uploading highly relevant photos in consideration of the geographical location information of the user when uploading photos. For example, the reception unit prioritizes uploading highly relevant photos in consideration of the geographical location information of the user when uploading photos. For example, when the user is in a specific area, the reception unit prioritizes uploading photos related to the area. The reception unit can also prioritize uploading photos taken in a location close to the user's current location. The reception unit can also prioritize uploading photos related to the travel destination in the case where the user is traveling. In this way, by considering the geographical location information of the user, highly relevant photos can be prioritized. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the geographical location information data of the user to the generation AI and cause the generation AI to select highly relevant photos.

[0042] The reception unit can analyze the social media activity of the user when uploading a photo and upload related photos. The reception unit, for example, analyzes the social media activity of the user when uploading a photo and uploads related photos. For example, the reception unit preferentially uploads related photos based on photos shared by the user on social media. The reception unit can also analyze the content of the user's social media posts and upload highly relevant photos. In addition, if the user uses a specific hashtag, the reception unit can also preferentially upload photos related to the hashtag. In this way, highly relevant photos can be uploaded by analyzing the user's social media activity. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's social media activity data to the generation AI and cause the generation AI to select related photos.

[0043] The identification unit can optimize the identification algorithm by referring to past identification data when identifying a dish. The identification unit optimizes the identification algorithm by referring to past identification data when identifying a dish, for example. For example, the identification unit optimizes the identification algorithm based on data of dishes identified in the past. The identification unit can also improve the accuracy for a specific dish from the past identification data. The identification unit can also analyze the past identification data and adjust the parameters of the identification algorithm. As a result, the accuracy of the identification algorithm is improved by referring to the past identification data. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the past identification data to the generation AI and cause the generation AI to optimize the identification algorithm.

[0044] The identification unit can apply different identification algorithms depending on the category of the dish when identifying the dish. For example, the identification unit applies different identification algorithms depending on the category of the dish when identifying the dish. For example, the identification unit applies different identification algorithms for each category, such as appetizer, main dish, and dessert. The identification unit can also adjust parameters of the identification algorithm depending on the type of dish. The identification unit can also select an optimal identification algorithm based on the characteristics of the dish. This improves the identification accuracy by applying the identification algorithm depending on the category of the dish. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input dish category data to the generation AI and cause the generation AI to apply the identification algorithm.

[0045] The identification unit can perform identification by taking into consideration the geographical distribution of the cuisine when identifying the cuisine. For example, the identification unit performs identification by taking into consideration the geographical distribution of the cuisine when identifying the cuisine. For example, the identification unit adjusts the identification algorithm based on the geographical distribution of the cuisine. The identification unit can also improve the accuracy for the identified cuisine based on the geographical distribution. The identification unit can also adjust the parameters of the identification algorithm by taking into consideration the geographical distribution. In this way, the identification accuracy is improved by taking into consideration the geographical distribution of the cuisine. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI or may be performed without using AI. For example, the identification unit can input the geographical distribution data of the cuisine to the generation AI and cause the generation AI to adjust the identification algorithm.

[0046] The identification unit can improve the accuracy of identification by referring to literature related to the dish when identifying the dish. For example, the identification unit improves the accuracy of identification by referring to literature related to the dish when identifying the dish. For example, the identification unit optimizes the identification algorithm based on literature related to the dish. The identification unit can also improve the accuracy for the specific dish from the related literature. The identification unit can also analyze the related literature and adjust the parameters of the identification algorithm. As a result, the identification accuracy is improved by referring to literature related to the dish. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI or may be performed without using AI. For example, the identification unit can input literature data related to the dish to the generation AI and cause the generation AI to execute optimization of the identification algorithm.

[0047] The layout unit can optimize the layout algorithm by referring to past layout data when creating a layout. The layout unit can optimize the layout algorithm by referring to past layout data when creating a layout. For example, the layout unit optimizes the layout algorithm based on layout data created in the past. The layout unit can also generate a visually attractive layout from the past layout data. The layout unit can also analyze the past layout data and adjust parameters of the layout algorithm. As a result, the accuracy of the layout algorithm is improved by referring to the past layout data. A part or all of the above-mentioned processing in the layout unit can be performed using, for example, AI, or can be performed without using AI. For example, the layout unit can input the past layout data to the generation AI and cause the generation AI to optimize the layout algorithm.

[0048] The layout unit can apply different layout methods to each food category when creating the layout. For example, the layout unit applies different layout methods to each food category when creating the layout. For example, the layout unit applies different layout methods to each category, such as appetizers, main dishes, and desserts. The layout unit can also adjust parameters of the layout method according to the type of food. The layout unit can also select an optimal layout method based on the characteristics of the food. In this way, a visually attractive menu table can be created by applying different layout methods to each food category. Some or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input food category data to the generation AI and cause the generation AI to apply the layout method.

[0049] The layout unit can analyze changes in the layout based on the time of submission of the dish when creating the layout. The layout unit, for example, analyzes changes in the layout based on the time of submission of the dish when creating the layout. For example, the layout unit provides an optimal layout based on the time of submission of the dish. The layout unit can also adjust parameters of the layout according to the time of submission. The layout unit can also provide a visually attractive layout taking into account the time of submission. In this way, by analyzing changes in the layout based on the time of submission of the dish, an optimal layout can be provided. A part or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input data on the time of submission of the dish to the generation AI and cause the generation AI to perform an analysis of changes in the layout.

[0050] The layout unit can analyze the layout by referring to the market data related to the cuisine when creating the layout. For example, the layout unit analyzes the layout by referring to the market data related to the cuisine when creating the layout. For example, the layout unit provides an optimal layout based on the market data related to the cuisine. The layout unit can also generate a visually attractive layout from the market data. The layout unit can also analyze the market data and adjust parameters of the layout algorithm. In this way, a visually attractive layout can be provided by referring to the market data related to the cuisine. Some or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input the market data related to the cuisine to the generation AI and cause the generation AI to perform the analysis of the layout.

[0051] The suggestion unit can optimize the explanation algorithm by referring to past explanation data when proposing an explanation of a dish. For example, the suggestion unit optimizes the explanation algorithm by referring to past explanation data when proposing an explanation of a dish. For example, the suggestion unit optimizes the explanation algorithm based on the explanation data of dishes proposed in the past. The suggestion unit can also generate a visually attractive explanation from the past explanation data. The suggestion unit can also analyze the past explanation data and adjust parameters of the explanation algorithm. As a result, the accuracy of the explanation algorithm is improved by referring to the past explanation data. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the past explanation data to the generation AI and cause the generation AI to execute optimization of the explanation algorithm.

[0052] The suggestion unit can apply different explanation algorithms according to the category of the dish when proposing a description of the dish. For example, the suggestion unit applies different explanation algorithms according to the category of the dish when proposing a description of the dish. For example, the suggestion unit applies different explanation algorithms for each category, such as appetizer, main dish, and dessert. The suggestion unit can also adjust parameters of the explanation algorithm according to the type of dish. The suggestion unit can also select an optimal explanation algorithm based on the characteristics of the dish. As a result, by applying different explanation algorithms according to the category of the dish, the accuracy of the explanation is improved. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input dish category data to the generation AI and cause the generation AI to apply the explanation algorithm.

[0053] The suggestion unit can determine the priority of the explanation based on the time of submission of the dish when proposing the explanation of the dish. For example, the suggestion unit determines the priority of the explanation based on the time of submission of the dish when proposing the explanation of the dish. For example, the suggestion unit provides an optimal explanation based on the time of submission of the dish. The suggestion unit can also adjust the parameters of the explanation according to the time of submission. The suggestion unit can also provide a visually attractive explanation taking into account the time of submission. In this way, the optimal explanation can be provided by determining the priority of the explanation based on the time of submission of the dish. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the submission time data of the dish to the generation AI and cause the generation AI to execute the determination of the priority of the explanation.

[0054] The suggestion unit can improve the accuracy of the explanation by referring to literature related to the dish when proposing an explanation of the dish. For example, the suggestion unit can improve the accuracy of the explanation by referring to literature related to the dish when proposing an explanation of the dish. For example, the suggestion unit optimizes an explanation algorithm based on literature related to the dish. The suggestion unit can also generate a visually attractive explanation from the related literature. The suggestion unit can also analyze the related literature and adjust parameters of the explanation algorithm. As a result, the accuracy of the explanation is improved by referring to literature related to the dish. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input literature data related to the dish to the generation AI and cause the generation AI to optimize the explanation algorithm.

[0055] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0056] The reception unit can automatically evaluate the resolution and quality of photos uploaded by users and select photos with optimal resolution and quality. For example, the reception unit automatically filters out low-resolution photos and preferentially selects high-resolution photos. The reception unit can also evaluate the brightness and contrast of photos and select visually appealing photos. Furthermore, the reception unit can detect noise and blur in photos and select high-quality photos. In this way, the visual appeal of the menu table is improved by automatically evaluating the quality of photos uploaded by users and selecting optimal photos.

[0057] When identifying a dish, the identification unit can analyze the nutritional information of the dish and suggest appropriate dishes to health-conscious users. For example, the identification unit can analyze nutritional information such as calories, protein, fat, and carbohydrates of a dish and suggest low-calorie and high-protein dishes to health-conscious users. The identification unit can also detect dishes containing specific allergens and suggest appropriate dishes to users with allergies. Furthermore, the identification unit can analyze the vitamin and mineral content and suggest nutritionally balanced dishes. In this way, appropriate dishes can be suggested to health-conscious users by analyzing the nutritional information of the dish.

[0058] The layout unit can refer to the user's past menu creation history and customize the layout based on the user's preferences. For example, the layout unit can analyze the layout patterns of menus created by the user in the past and suggest a similar layout. The layout unit can also automatically apply colors and fonts preferred by the user to create a visually appealing menu list. Furthermore, the layout unit can take into account the order of dishes selected by the user in the past and suggest a similar order. This makes it possible to customize based on the user's preferences by referring to the user's past menu creation history.

[0059] When proposing a dish description, the suggestion unit can customize the description according to the cultural background and language of the user. For example, when the user speaks Japanese, the suggestion unit can provide an explanation based on Japanese food culture. When the user speaks English, the suggestion unit can also provide an explanation based on English-speaking food culture. Furthermore, the suggestion unit can take into account the religious background of the user and provide a dish description suitable for a specific religion. This enables customization according to the cultural background and language of the user, making it possible to provide a more appropriate dish description.

[0060] The reception unit can analyze metadata of photos uploaded by users and suggest appropriate dishes based on the date and time and location where the photos were taken. For example, the reception unit can analyze the date and time when the photos were taken and suggest dishes according to the season. The reception unit can also analyze the location where the photos were taken and suggest local specialties and dishes. Furthermore, the reception unit can analyze the situation in which the photos were taken (for example, a party or a picnic) and suggest dishes suitable for the situation. In this way, more appropriate dishes can be suggested by analyzing the metadata of photos.

[0061] The process flow of the first embodiment will be briefly described below.

[0062] Step 1: The reception department accepts the upload of photos that the user wants to include on the menu. The user can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception department inputs the uploaded photos into the image analysis AI. Step 2: The identification unit uses image analysis technology to analyze the uploaded photos and identify the type and characteristics of the food. For example, the identification unit may identify salad or soup from an appetizer photo, steak or pasta from a main dish photo, and cake or ice cream from a dessert photo. The identification unit uses image recognition technology and machine learning algorithms to analyze the content of each photo. Step 3: The layout unit considers the order and layout of the menu based on the information identified by the identification unit. For example, the layout unit creates an easy-to-read menu by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges photos of each dish in an appropriate position to create a visually appealing layout. The layout unit determines the order and layout of the menu taking into account the type and popularity of the dishes, the season, etc. Step 4: The suggestion unit proposes a description of the dish based on the menu created by the layout unit. The suggestion unit proposes a description of the dish based on information collected from a pre-registered database or the Internet. For example, the suggestion unit may suggest a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with a special sauce" from a photo of a steak.

[0063] (Example 2) The image analysis system according to the embodiment of the present invention is a system that creates an original menu table simply by uploading a photo to be included in the menu. In this image analysis system, a user uploads a photo to be included in the menu, and an image analysis AI analyzes the uploaded photo and creates a menu table that takes into account the order and layout of the menu. Furthermore, the AI ​​also suggests descriptions of dishes based on prior information and photos. For example, when a user uploads photos of appetizers, main dishes, desserts, etc., the image analysis AI analyzes the contents of each photo and identifies the type and characteristics of the dish. Next, the image analysis AI creates an easy-to-read menu table by arranging the dishes in the order of appetizers, main dishes, and desserts. In addition, the image analysis AI arranges the photos of each dish in an appropriate position to achieve a visually attractive layout. Furthermore, the image analysis AI suggests descriptions such as "healthy salad made with fresh vegetables" from a photo of a salad and "juicy steak served with special sauce" from a photo of a steak. This allows users to easily create attractive menu tables. This allows the image analysis system to easily create original menu tables for users, and the order and layout of the menu and descriptions of the dishes are automatically suggested, significantly reducing time and effort.

[0064] The image analysis system according to the embodiment includes a reception unit, a specification unit, a layout unit, and a proposal unit. The reception unit accepts uploading of photos that a user wants to put on a menu. The user can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos into the image analysis AI. The specification unit uses image analysis technology to analyze the uploaded photos and specify the type and characteristics of the dishes. For example, the specification unit specifies salad or soup from a photo of an appetizer, steak or pasta from a photo of a main dish, and cake or ice cream from a photo of a dessert. The specification unit analyzes the content of each photo using image recognition technology or machine learning algorithms. The layout unit considers the order and layout of the menu based on the information specified by the specification unit. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. In addition, the layout unit arranges the photos of each dish in an appropriate position to realize a visually attractive layout. The layout unit determines the order and layout of the menu taking into account the type and popularity of the dishes, the season, and the like. The suggestion unit suggests a description of the dishes based on the menu table created by the layout unit. The suggestion unit suggests a description of the dishes based on information collected from a preregistered database or the Internet. For example, the suggestion unit suggests a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with special sauce" from a photo of a steak. In this way, the image analysis system according to the embodiment automatically suggests the order and layout of the menu and the description of the dishes simply by the user uploading a photo. In this way, the image analysis system allows the user to easily create an original menu table, and since the order and layout of the menu and the description of the dishes are automatically suggested, it is possible to significantly reduce time and labor.

[0065] The reception unit accepts the upload of photos that users want to include in the menu. For example, users can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos into the image analysis AI. Specifically, when a user uploads photos through a dedicated web interface or mobile app, the reception unit quickly processes the photos and checks the format and resolution of the image files. In addition, the reception unit extracts metadata (e.g., the date and time of shooting and location information) from the uploaded photos and uses them as auxiliary information for analysis. The reception unit allows users to upload multiple photos at once and also provides a function to add tags and comments to each photo. This allows users to easily describe the content and features of the photos, making the subsequent analysis process proceed more smoothly. The reception unit temporarily stores the uploaded photos in cloud storage so that the image analysis AI can access them. In addition, the reception unit manages the user's account information and past upload history to make them easy to reuse and refer to. In this way, the reception unit plays a role in allowing users to easily upload photos and improving the efficiency of the entire system.

[0066] The identification unit uses image analysis technology to analyze the uploaded photos and identify the type and characteristics of the food. For example, the identification unit identifies salads and soups from photos of appetizers, steaks and pasta from photos of main dishes, and cakes and ice cream from photos of desserts. The identification unit uses image recognition technology and machine learning algorithms to analyze the content of each photo. Specifically, the identification unit uses a deep learning model to extract image features and compares them with a pre-trained database to identify the type of food with high accuracy. For example, a convolutional neural network (CNN) is used to analyze features such as edges, textures, and colors of an image, and based on this, the type of food is classified. Furthermore, the identification unit analyzes not only the characteristics of the food, but also information on the presentation and background to identify detailed characteristics of the food. For example, from a photo of a salad, the type of vegetables used and the type of dressing are identified, and from a photo of a steak, the degree of doneness and the type of sauce are identified. Based on this information, the identification unit registers detailed characteristics of the food in a database and makes them available for subsequent processes. Furthermore, the identification unit modifies the analysis results based on feedback from users to continuously improve the accuracy of the model. This enables the identification unit to analyze the uploaded photo with high accuracy and quickly identify the type and characteristics of the food.

[0067] The layout unit considers the order and layout of the menu based on the information specified by the specification unit. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges the photos of each dish in an appropriate position to achieve a visually attractive layout. Specifically, the layout unit automatically generates menu sections based on the types and characteristics of the dishes provided by the specification unit, and arranges the dishes corresponding to each section. For example, salads and soups are arranged in the appetizer section, steaks and pasta in the main dish section, and cakes and ice cream in the dessert section. The layout unit also optimizes the order and layout of the menu in consideration of the popularity and season of the dishes, the user's preferences, and the like. For example, seasonal dishes and menus that correspond to special events are arranged preferentially to attract the user's attention. The layout unit also arranges the photos of each dish in an appropriate size and position to achieve a visually balanced layout. For example, by displaying the photos of the dishes in a large size, the visual impact is emphasized and the user's appetite is stimulated. The layout unit also adjusts the design, font, color, and the like of the menu to create an overall sense of unity. This allows the layout unit to automatically create an easy-to-read and attractive menu table, thereby improving user satisfaction.

[0068] The suggestion unit proposes a description of the dish based on the menu table created by the layout unit. The suggestion unit proposes a description of the dish based on information collected from a database registered in advance or from the Internet. For example, the suggestion unit proposes a description such as "a healthy salad made with fresh vegetables" from a photo of a salad, and "a juicy steak served with a special sauce" from a photo of a steak. Specifically, the suggestion unit searches the database for a related description based on the type and characteristics of the dish provided by the identification unit, and automatically generates an appropriate description. For example, when a photo of a salad is identified, the suggestion unit generates a specific description such as "a healthy salad made with fresh lettuce, tomatoes, and cucumbers and finished with a special dressing." Furthermore, the suggestion unit collects the latest cooking information and trends from the Internet and proposes a description based on this. For example, in accordance with the recent health-conscious trend, it is possible to add a description such as "a salad made with organic vegetables" or "used a special gluten-free dressing." The suggestion unit modifies the description based on user feedback to provide a more appropriate and attractive description. In addition, the suggestion unit can provide not only a description of the dish, but also detailed information such as the background and history of the dish, cooking method, etc. This allows the user to fully convey the appeal of the dish through the menu, thereby increasing the effect of attracting the interest of customers.

[0069] The identification unit can identify the type and characteristics of a dish using image analysis technology. The identification unit, for example, uses image analysis technology to analyze the uploaded photo and identify the type and characteristics of a dish. For example, the identification unit identifies salad or soup from a photo of an appetizer, steak or pasta from a photo of a main dish, and cake or ice cream from a photo of a dessert. The identification unit analyzes the content of each photo using image recognition technology or machine learning algorithms. In this way, the type and characteristics of a dish can be accurately identified using image analysis technology. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input the uploaded photo to the generation AI and cause the generation AI to identify the type and characteristics of a dish.

[0070] The layout unit can determine the order or layout of the menu based on at least one of the type or popularity of the food, and the season. The layout unit determines the order or layout of the menu, for example, taking into consideration the type or popularity of the food, the season, etc. For example, the layout unit creates an easy-to-read menu table by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges photos of each dish in an appropriate position to achieve a visually attractive layout. The layout unit determines the order or layout of the menu, for example, taking into consideration the type or popularity of the food, the season, etc. As a result, a visually attractive menu table can be created by taking into consideration the type or popularity of the food, the season, etc. A part or all of the above-mentioned processing in the layout unit may be performed, for example, using AI, or may be performed without using AI. For example, the layout unit can input information such as the type or popularity of the food, the season, etc. to the generation AI and cause the generation AI to determine the order or layout of the menu.

[0071] The suggestion unit can suggest a dish description based on information collected from a pre-registered database or the Internet. The suggestion unit suggests a dish description based on information collected from a pre-registered database or the Internet, for example. For example, the suggestion unit suggests a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with special sauce" from a photo of a steak. The suggestion unit suggests a dish description based on information collected from a pre-registered database or the Internet. This allows accurate suggestions of dish descriptions based on information from a pre-registered database or the Internet. Some or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input information collected from a pre-registered database or the Internet to a generation AI, and cause the generation AI to execute suggestions of dish descriptions.

[0072] The reception unit can accept that the user uploads photos of multiple dishes. For example, the reception unit accepts that the user uploads photos of multiple dishes. The user can upload photos of multiple dishes such as appetizers, main dishes, and desserts. The reception unit inputs the uploaded photos to an image analysis AI. This allows the user to upload photos of multiple dishes. A part or all of the above-mentioned processing in the reception unit may be performed using an AI, for example, or may be performed without using an AI. For example, the reception unit can input photos of multiple dishes uploaded by the user to a generation AI and cause the generation AI to analyze the photos.

[0073] The suggestion unit can provide a description of a dish to the user. The suggestion unit, for example, provides a description of a dish to the user. The suggestion unit suggests a description of a dish based on information collected from a pre-registered database or the Internet. For example, the suggestion unit suggests a description such as "a healthy salad made with fresh vegetables" from a photo of a salad, or "a juicy steak served with a special sauce" from a photo of a steak. This improves the appeal of the menu by providing a description of the dish to the user. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit may input information collected from a pre-registered database or the Internet to the generation AI, and cause the generation AI to execute a suggestion of a description of the dish.

[0074] The reception unit can estimate the user's emotion and adjust the timing of uploading photos based on the estimated user's emotion. The reception unit, for example, estimates the user's emotion and adjusts the timing of uploading photos based on the estimated user's emotion. For example, when the user is stressed, the reception unit provides a simple interface and minimizes the steps of uploading photos. In addition, when the user is relaxed, the reception unit can provide detailed upload options and suggest a customizable upload method. In addition, when the user is in a hurry, the reception unit can prioritize voice input and enable quick uploading of photos. This improves the user experience by adjusting the timing of uploading photos according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0075] The reception unit can analyze the user's past upload history and select an appropriate upload method. The reception unit, for example, analyzes the user's past upload history and selects the optimal upload method. For example, the reception unit automatically displays the types of photos that the user frequently uploaded in the past as candidates. The reception unit can also preferentially suggest the upload method (audio, text, etc.) that the user used in the past. The reception unit can also predict and suggest the upload method to be used in a specific time period from the user's past upload history. In this way, the optimal upload method can be provided by analyzing the past upload history. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's past upload history data to the generation AI and cause the generation AI to select the optimal upload method.

[0076] The reception unit can perform filtering based on the user's current project and field of interest when uploading a photo. The reception unit performs filtering based on the user's current project and field of interest when uploading a photo, for example. For example, the reception unit preferentially uploads photos related to a project currently in progress by the user. The reception unit can also automatically filter photos with high relevance based on the user's field of interest. The reception unit can also preferentially display related photos when the user uploads photos based on a specific theme. This allows highly relevant photos to be uploaded by filtering based on the user's current project and field of interest. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input data on the user's current project and field of interest to the generation AI and cause the generation AI to perform filtering.

[0077] The reception unit can estimate the user's emotion and determine the priority of photos to be uploaded based on the estimated user's emotion. The reception unit, for example, estimates the user's emotion and determines the priority of photos to be uploaded based on the estimated user's emotion. For example, when the user is excited, the reception unit preferentially uploads visually attractive photos. In addition, when the user is relaxed, the reception unit can also preferentially upload photos that require detailed explanations. In addition, when the user is in a hurry, the reception unit can also preferentially upload photos that can be easily uploaded. This improves the user experience by determining the priority of photos to be uploaded according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0078] The reception unit can prioritize uploading highly relevant photos in consideration of the geographical location information of the user when uploading photos. For example, the reception unit prioritizes uploading highly relevant photos in consideration of the geographical location information of the user when uploading photos. For example, when the user is in a specific area, the reception unit prioritizes uploading photos related to the area. The reception unit can also prioritize uploading photos taken in a location close to the user's current location. The reception unit can also prioritize uploading photos related to the travel destination in the case where the user is traveling. In this way, by considering the geographical location information of the user, highly relevant photos can be prioritized. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the geographical location information data of the user to the generation AI and cause the generation AI to select highly relevant photos.

[0079] The reception unit can analyze the social media activity of the user when uploading a photo and upload related photos. The reception unit, for example, analyzes the social media activity of the user when uploading a photo and uploads related photos. For example, the reception unit preferentially uploads related photos based on photos shared by the user on social media. The reception unit can also analyze the content of the user's social media posts and upload highly relevant photos. In addition, if the user uses a specific hashtag, the reception unit can also preferentially upload photos related to the hashtag. In this way, highly relevant photos can be uploaded by analyzing the user's social media activity. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's social media activity data to the generation AI and cause the generation AI to select related photos.

[0080] The identification unit can estimate the user's emotion and adjust the method of identifying the type and characteristics of the dish based on the estimated user's emotion. The identification unit, for example, estimates the user's emotion and adjusts the method of identifying the type and characteristics of the dish based on the estimated user's emotion. For example, when the user is relaxed, the identification unit performs a detailed analysis and identifies the characteristics of the dish in detail. In addition, when the user is in a hurry, the identification unit can also perform a simplified analysis and identify only the main features. In addition, when the user is excited, the identification unit can preferentially identify visually attractive features. Thereby, by adjusting the method of identifying the type and characteristics of the dish according to the user's emotion, the identification accuracy is improved. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0081] The identification unit can optimize the identification algorithm by referring to past identification data when identifying a dish. The identification unit optimizes the identification algorithm by referring to past identification data when identifying a dish, for example. For example, the identification unit optimizes the identification algorithm based on data of dishes identified in the past. The identification unit can also improve the accuracy for a specific dish from the past identification data. The identification unit can also analyze the past identification data and adjust the parameters of the identification algorithm. As a result, the accuracy of the identification algorithm is improved by referring to the past identification data. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the past identification data to the generation AI and cause the generation AI to optimize the identification algorithm.

[0082] The identification unit can apply different identification algorithms depending on the category of the dish when identifying the dish. For example, the identification unit applies different identification algorithms depending on the category of the dish when identifying the dish. For example, the identification unit applies different identification algorithms for each category, such as appetizer, main dish, and dessert. The identification unit can also adjust parameters of the identification algorithm depending on the type of dish. The identification unit can also select an optimal identification algorithm based on the characteristics of the dish. This improves the identification accuracy by applying the identification algorithm depending on the category of the dish. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input dish category data to the generation AI and cause the generation AI to apply the identification algorithm.

[0083] The identification unit can estimate the user's emotion and adjust the display method of the identified result based on the estimated user's emotion. The identification unit, for example, estimates the user's emotion and adjusts the display method of the identified result based on the estimated user's emotion. For example, when the user is nervous, the identification unit provides a simple and highly visible display method. When the user is relaxed, the identification unit can also provide a display method including detailed information. When the user is in a hurry, the identification unit can also provide a display method that focuses on the main points. This improves the user experience by adjusting the display method of the identified result according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0084] The identification unit can perform identification by taking into consideration the geographical distribution of the cuisine when identifying the cuisine. For example, the identification unit performs identification by taking into consideration the geographical distribution of the cuisine when identifying the cuisine. For example, the identification unit adjusts the identification algorithm based on the geographical distribution of the cuisine. The identification unit can also improve the accuracy for the identified cuisine based on the geographical distribution. The identification unit can also adjust the parameters of the identification algorithm by taking into consideration the geographical distribution. In this way, the identification accuracy is improved by taking into consideration the geographical distribution of the cuisine. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI or may be performed without using AI. For example, the identification unit can input the geographical distribution data of the cuisine to the generation AI and cause the generation AI to adjust the identification algorithm.

[0085] The identification unit can improve the accuracy of identification by referring to literature related to the dish when identifying the dish. For example, the identification unit improves the accuracy of identification by referring to literature related to the dish when identifying the dish. For example, the identification unit optimizes the identification algorithm based on literature related to the dish. The identification unit can also improve the accuracy for the specific dish from the related literature. The identification unit can also analyze the related literature and adjust the parameters of the identification algorithm. As a result, the identification accuracy is improved by referring to literature related to the dish. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI or may be performed without using AI. For example, the identification unit can input literature data related to the dish to the generation AI and cause the generation AI to execute optimization of the identification algorithm.

[0086] The layout unit can estimate the user's emotion and adjust the order of the menu and the display method of the layout based on the estimated user's emotion. The layout unit, for example, estimates the user's emotion and adjusts the order of the menu and the display method of the layout based on the estimated user's emotion. For example, the layout unit provides a detailed layout and a visually attractive display method when the user is relaxed. The layout unit can also provide a simple and highly visible layout when the user is in a hurry. The layout unit can also provide a visually stimulating layout when the user is excited. This improves the user experience by adjusting the order of the menu and the display method of the layout according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the layout unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the layout unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0087] The layout unit can optimize the layout algorithm by referring to past layout data when creating a layout. The layout unit can optimize the layout algorithm by referring to past layout data when creating a layout. For example, the layout unit optimizes the layout algorithm based on layout data created in the past. The layout unit can also generate a visually attractive layout from the past layout data. The layout unit can also analyze the past layout data and adjust parameters of the layout algorithm. As a result, the accuracy of the layout algorithm is improved by referring to the past layout data. A part or all of the above-mentioned processing in the layout unit can be performed using, for example, AI, or can be performed without using AI. For example, the layout unit can input the past layout data to the generation AI and cause the generation AI to optimize the layout algorithm.

[0088] The layout unit can apply different layout methods to each food category when creating the layout. For example, the layout unit applies different layout methods to each food category when creating the layout. For example, the layout unit applies different layout methods to each category, such as appetizers, main dishes, and desserts. The layout unit can also adjust parameters of the layout method according to the type of food. The layout unit can also select an optimal layout method based on the characteristics of the food. In this way, a visually attractive menu table can be created by applying different layout methods to each food category. Some or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input food category data to the generation AI and cause the generation AI to apply the layout method.

[0089] The layout unit can estimate the user's emotion and determine the layout priority based on the estimated user's emotion. The layout unit, for example, estimates the user's emotion and determines the layout priority based on the estimated user's emotion. For example, the layout unit preferentially provides a detailed layout when the user is relaxed. Also, the layout unit can preferentially provide a simple and highly visible layout when the user is in a hurry. Also, the layout unit can preferentially provide a visually stimulating layout when the user is excited. This improves the user experience by determining the layout priority according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the layout unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the layout unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0090] The layout unit can analyze changes in the layout based on the time of submission of the dish when creating the layout. The layout unit, for example, analyzes changes in the layout based on the time of submission of the dish when creating the layout. For example, the layout unit provides an optimal layout based on the time of submission of the dish. The layout unit can also adjust parameters of the layout according to the time of submission. The layout unit can also provide a visually attractive layout taking into account the time of submission. In this way, by analyzing changes in the layout based on the time of submission of the dish, an optimal layout can be provided. A part or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input data on the time of submission of the dish to the generation AI and cause the generation AI to perform an analysis of changes in the layout.

[0091] The layout unit can analyze the layout by referring to the market data related to the cuisine when creating the layout. For example, the layout unit analyzes the layout by referring to the market data related to the cuisine when creating the layout. For example, the layout unit provides an optimal layout based on the market data related to the cuisine. The layout unit can also generate a visually attractive layout from the market data. The layout unit can also analyze the market data and adjust parameters of the layout algorithm. In this way, a visually attractive layout can be provided by referring to the market data related to the cuisine. Some or all of the above-mentioned processing in the layout unit may be performed, for example, using AI or may be performed without using AI. For example, the layout unit can input the market data related to the cuisine to the generation AI and cause the generation AI to perform the analysis of the layout.

[0092] The suggestion unit can estimate the user's emotion and adjust the way of expressing the dish description based on the estimated user's emotion. The suggestion unit, for example, estimates the user's emotion and adjusts the way of expressing the dish description based on the estimated user's emotion. For example, the suggestion unit provides a detailed explanation and a visually attractive way of expressing the dish description when the user is relaxed. The suggestion unit can also provide a concise and to-the-point explanation when the user is in a hurry. The suggestion unit can also provide a visually stimulating way of expressing the dish description when the user is excited. This improves the user experience by adjusting the way of expressing the dish description according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0093] The suggestion unit can optimize the explanation algorithm by referring to past explanation data when proposing an explanation of a dish. For example, the suggestion unit optimizes the explanation algorithm by referring to past explanation data when proposing an explanation of a dish. For example, the suggestion unit optimizes the explanation algorithm based on the explanation data of dishes proposed in the past. The suggestion unit can also generate a visually attractive explanation from the past explanation data. The suggestion unit can also analyze the past explanation data and adjust parameters of the explanation algorithm. As a result, the accuracy of the explanation algorithm is improved by referring to the past explanation data. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the past explanation data to the generation AI and cause the generation AI to execute optimization of the explanation algorithm.

[0094] The suggestion unit can apply different explanation algorithms according to the category of the dish when proposing a description of the dish. For example, the suggestion unit applies different explanation algorithms according to the category of the dish when proposing a description of the dish. For example, the suggestion unit applies different explanation algorithms for each category, such as appetizer, main dish, and dessert. The suggestion unit can also adjust parameters of the explanation algorithm according to the type of dish. The suggestion unit can also select an optimal explanation algorithm based on the characteristics of the dish. As a result, by applying different explanation algorithms according to the category of the dish, the accuracy of the explanation is improved. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input dish category data to the generation AI and cause the generation AI to apply the explanation algorithm.

[0095] The suggestion unit can estimate the user's emotion and adjust the length of the explanation based on the estimated user's emotion. The suggestion unit, for example, estimates the user's emotion and adjusts the length of the explanation based on the estimated user's emotion. For example, the suggestion unit provides a detailed explanation and a visually attractive expression method when the user is relaxed. In addition, the suggestion unit can also provide a concise and to the point explanation when the user is in a hurry. In addition, the suggestion unit can also provide a visually stimulating expression method when the user is excited. This improves the user experience by adjusting the length of the explanation according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0096] The suggestion unit can determine the priority of the explanation based on the time of submission of the dish when proposing the explanation of the dish. For example, the suggestion unit determines the priority of the explanation based on the time of submission of the dish when proposing the explanation of the dish. For example, the suggestion unit provides an optimal explanation based on the time of submission of the dish. The suggestion unit can also adjust the parameters of the explanation according to the time of submission. The suggestion unit can also provide a visually attractive explanation taking into account the time of submission. In this way, the optimal explanation can be provided by determining the priority of the explanation based on the time of submission of the dish. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the submission time data of the dish to the generation AI and cause the generation AI to execute the determination of the priority of the explanation.

[0097] The suggestion unit can improve the accuracy of the explanation by referring to literature related to the dish when proposing an explanation of the dish. For example, the suggestion unit can improve the accuracy of the explanation by referring to literature related to the dish when proposing an explanation of the dish. For example, the suggestion unit optimizes an explanation algorithm based on literature related to the dish. The suggestion unit can also generate a visually attractive explanation from the related literature. The suggestion unit can also analyze the related literature and adjust parameters of the explanation algorithm. As a result, the accuracy of the explanation is improved by referring to literature related to the dish. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input literature data related to the dish to the generation AI and cause the generation AI to optimize the explanation algorithm.

[0098] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0099] The reception unit can automatically evaluate the resolution and quality of photos uploaded by users and select photos with optimal resolution and quality. For example, the reception unit automatically filters out low-resolution photos and preferentially selects high-resolution photos. The reception unit can also evaluate the brightness and contrast of photos and select visually appealing photos. Furthermore, the reception unit can detect noise and blur in photos and select high-quality photos. In this way, the visual appeal of the menu table is improved by automatically evaluating the quality of photos uploaded by users and selecting optimal photos.

[0100] When identifying a dish, the identification unit can analyze the nutritional information of the dish and suggest appropriate dishes to health-conscious users. For example, the identification unit can analyze nutritional information such as calories, protein, fat, and carbohydrates of a dish and suggest low-calorie and high-protein dishes to health-conscious users. The identification unit can also detect dishes containing specific allergens and suggest appropriate dishes to users with allergies. Furthermore, the identification unit can analyze the vitamin and mineral content and suggest nutritionally balanced dishes. In this way, appropriate dishes can be suggested to health-conscious users by analyzing the nutritional information of the dish.

[0101] The layout unit can refer to the user's past menu creation history and customize the layout based on the user's preferences. For example, the layout unit can analyze the layout patterns of menus created by the user in the past and suggest a similar layout. The layout unit can also automatically apply colors and fonts preferred by the user to create a visually appealing menu list. Furthermore, the layout unit can take into account the order of dishes selected by the user in the past and suggest a similar order. This makes it possible to customize based on the user's preferences by referring to the user's past menu creation history.

[0102] When proposing a dish description, the suggestion unit can customize the description according to the cultural background and language of the user. For example, when the user speaks Japanese, the suggestion unit can provide an explanation based on Japanese food culture. When the user speaks English, the suggestion unit can also provide an explanation based on English-speaking food culture. Furthermore, the suggestion unit can take into account the religious background of the user and provide a dish description suitable for a specific religion. This enables customization according to the cultural background and language of the user, making it possible to provide a more appropriate dish description.

[0103] The reception unit can analyze metadata of photos uploaded by users and suggest appropriate dishes based on the date and time and location where the photos were taken. For example, the reception unit can analyze the date and time when the photos were taken and suggest dishes according to the season. The reception unit can also analyze the location where the photos were taken and suggest local specialties and dishes. Furthermore, the reception unit can analyze the situation in which the photos were taken (for example, a party or a picnic) and suggest dishes suitable for the situation. In this way, more appropriate dishes can be suggested by analyzing the metadata of photos.

[0104] The reception unit can estimate the user's emotion and customize the photo upload interface based on the estimated user's emotion. For example, when the user is stressed, the reception unit can provide a simple and intuitive interface to minimize the steps of uploading photos. When the user is relaxed, the reception unit can provide detailed customization options to allow the user to freely select the upload method. Furthermore, when the user is excited, the reception unit can provide a visually appealing interface to allow the user to enjoy the upload experience. This improves the user experience by customizing the interface according to the user's emotion.

[0105] The identification unit can estimate the user's emotions and adjust the method of feeding back the dish identification results based on the estimated user's emotions. For example, the identification unit can provide detailed feedback and explain the dish identification process when the user is relaxed. Also, the identification unit can provide brief feedback and display only the main identification results when the user is in a hurry. Furthermore, the identification unit can provide visually appealing feedback to allow the user to enjoy the identification results when the user is excited. This improves the user experience by adjusting the feedback method according to the user's emotions.

[0106] The layout unit can estimate the user's emotion and dynamically change the layout of the menu based on the estimated user's emotion. For example, the layout unit can provide a detailed and visually rich layout when the user is relaxed. The layout unit can also provide a simple and highly visible layout when the user is in a hurry. Furthermore, the layout unit can provide a visually stimulating layout when the user is excited, making the menu table enjoyable. This improves the user experience by dynamically changing the layout according to the user's emotion.

[0107] The suggestion unit can estimate the user's emotions and adjust the tone and style of the dish explanation based on the estimated user's emotions. For example, the suggestion unit can provide a calm and detailed explanation when the user is relaxed. The suggestion unit can also provide a concise and to-the-point explanation when the user is in a hurry. Furthermore, the suggestion unit can provide a visually stimulating and energetic explanation when the user is excited. This improves the user experience by adjusting the tone and style of the explanation according to the user's emotions.

[0108] The suggestion unit can estimate the user's emotion and adjust the length of the dish description based on the estimated user's emotion. For example, the suggestion unit can provide a detailed description and a visually appealing presentation when the user is relaxed. The suggestion unit can also provide a concise and to-the-point description when the user is in a hurry. Furthermore, the suggestion unit can also provide a visually stimulating presentation when the user is excited. This improves the user experience by adjusting the length of the description according to the user's emotion.

[0109] The process flow of the second embodiment will be briefly described below.

[0110] Step 1: The reception department accepts the upload of photos that the user wants to include on the menu. The user can upload photos of multiple dishes, such as appetizers, main dishes, and desserts. The reception department inputs the uploaded photos into the image analysis AI. Step 2: The identification unit uses image analysis technology to analyze the uploaded photos and identify the type and characteristics of the food. For example, the identification unit may identify salad or soup from an appetizer photo, steak or pasta from a main dish photo, and cake or ice cream from a dessert photo. The identification unit uses image recognition technology and machine learning algorithms to analyze the content of each photo. Step 3: The layout unit considers the order and layout of the menu based on the information identified by the identification unit. For example, the layout unit creates an easy-to-read menu by arranging the dishes in the order of appetizer, main dish, and dessert. The layout unit also arranges photos of each dish in an appropriate position to create a visually appealing layout. The layout unit determines the order and layout of the menu taking into account the type and popularity of the dishes, the season, etc. Step 4: The suggestion unit proposes a description of the dish based on the menu created by the layout unit. The suggestion unit proposes a description of the dish based on information collected from a pre-registered database or the Internet. For example, the suggestion unit may suggest a description such as "healthy salad made with fresh vegetables" from a photo of a salad, or "juicy steak served with a special sauce" from a photo of a steak.

[0111] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0112] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0113] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0114] Each of the multiple elements including the above-mentioned reception unit, identification unit, layout unit, and suggestion unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the reception unit is realized by the control unit 46A of the smart device 14, and accepts the upload of a photo that the user wants to put on the menu. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and analyzes the uploaded photo using image analysis technology to identify the type and characteristics of the dish. The layout unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and considers the order and layout of the menu based on the information identified by the identification unit. The suggestion unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and proposes an explanation of the dish based on the menu table created by the layout unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0115] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0116] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0117] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0118] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0119] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0120] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0121] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0122] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0123] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0124] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0125] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0126] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0127] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0128] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0129] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0130] Each of the multiple elements including the above-mentioned reception unit, identification unit, layout unit, and suggestion unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the reception unit is realized by the control unit 46A of the smart glasses 214, and accepts the upload of a photo that the user wants to put on the menu. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and analyzes the uploaded photo using image analysis technology to identify the type and characteristics of the dish. The layout unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and considers the order and layout of the menu based on the information identified by the identification unit. The suggestion unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and proposes a description of the dish based on the menu table created by the layout unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0131] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0132] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0133] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0134] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0135] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0136] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0137] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0138] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0139] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0140] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0141] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0142] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0143] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0144] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0145] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0146] Each of the multiple elements including the above-mentioned reception unit, identification unit, layout unit, and suggestion unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the reception unit is realized by the control unit 46A of the headset type terminal 314, and accepts the upload of a photo that the user wants to put on the menu. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and analyzes the uploaded photo using image analysis technology to identify the type and characteristics of the dish. The layout unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and considers the order and layout of the menu based on the information identified by the identification unit. The suggestion unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and proposes an explanation of the dish based on the menu table created by the layout unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0147] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0148] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0149] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0150] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0151] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0152] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0153] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0154] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0155] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0156] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0158] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0159] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0160] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0161] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0162] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0163] Each of the multiple elements including the above-mentioned reception unit, identification unit, layout unit, and suggestion unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the reception unit is realized by the control unit 46A of the robot 414, and accepts the uploading of a photo that the user wants to put on the menu. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and analyzes the uploaded photo using image analysis technology to identify the type and characteristics of the dish. The layout unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and considers the order and layout of the menu based on the information identified by the identification unit. The suggestion unit is realized, for example, by the identification processing unit 290 of the data processing device 12, and proposes an explanation of the dish based on the menu table created by the layout unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0164] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0165] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0166] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0167] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0168] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0169] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0170] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0171] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0172] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0173] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0174] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0175] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0176] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0177] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0178] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0179] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0180] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0181] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0182] (Appendix 1) There is a reception desk where you can upload photos you want to include on the menu. an identification unit that analyzes the photo uploaded by the reception unit and identifies a type or feature of a dish; a layout unit that determines an order or layout of the menu based on the information specified by the determination unit; a suggestion unit that suggests a description of a dish based on the menu table created by the layout unit; A system characterized by: (Appendix 2) The identification unit is Identifying the type and characteristics of food using image analysis technology 2. The system of claim 1, (Appendix 3) The layout unit: Decide on the order or layout of your menu based on food type, popularity, and / or seasonality 2. The system of claim 1, (Appendix 4) The suggestion unit, Suggests dish descriptions based on pre-registered databases and information collected from the internet 2. The system of claim 1, (Appendix 5) The reception unit is Allows users to upload photos of multiple dishes 2. The system of claim 1, (Appendix 6) The suggestion unit, Providing a description of the dish to the user 2. The system of claim 1, (Appendix 7) The reception unit is Infer user emotions and adjust the timing of photo uploads based on the estimated user emotions 2. The system of claim 1, (Appendix 8) The reception unit is Analyze the user's past upload history and select the appropriate upload method 2. The system of claim 1, (Appendix 9) The reception unit is Filter photos as they are uploaded based on the user's current projects and interests 2. The system of claim 1, (Appendix 10) The reception unit is Estimate user emotions and prioritize photos to upload based on the estimated user emotions 2. The system of claim 1, (Appendix 11) The reception unit is When uploading photos, it considers the user's geographic location to prioritize uploading more relevant photos. 2. The system of claim 1, (Appendix 12) The reception unit is When uploading photos, analyze your social media activity and upload relevant photos 2. The system of claim 1, (Appendix 13) The identification unit is Infer user emotions and adjust the method of identifying food types and characteristics based on the estimated user emotions 2. The system of claim 1, (Appendix 14) The identification unit is When identifying dishes, past identification data is referenced to optimize the identification algorithm. 2. The system of claim 1, (Appendix 15) The identification unit is When identifying cuisines, apply different identification algorithms depending on the cuisine category. 2. The system of claim 1, (Appendix 16) The identification unit is Inferring a user's emotion and adjusting the way certain results are displayed based on the inferred user emotion 2. The system of claim 1, (Appendix 17) The identification unit is When identifying a dish, the geographical distribution of the dish is taken into consideration. 2. The system of claim 1, (Appendix 18) The identification unit is When identifying a dish, refer to related literature to improve the accuracy of the identification. 2. The system of claim 1, (Appendix 19) The layout unit: Infer user emotions and adjust menu order and layout display methods based on the inferred user emotions 2. The system of claim 1, (Appendix 20) The layout unit: When creating a layout, the layout algorithm is optimized by referring to past layout data. 2. The system of claim 1, (Appendix 21) The layout unit: When creating a layout, apply different layout techniques to different food categories. 2. The system of claim 1, (Appendix 22) The layout unit: Infer user emotions and prioritize layouts based on the estimated user emotions 2. The system of claim 1, (Appendix 23) The layout unit: When creating the layout, analyze the changes in the layout based on the time of dish submission 2. The system of claim 1, (Appendix 24) The layout unit: When creating the layout, refer to the relevant market data for the cuisine to analyze the layout. 2. The system of claim 1, (Appendix 25) The suggestion unit, Infer user emotions and adjust the way food descriptions are presented based on the estimated user emotions 2. The system of claim 1, (Appendix 26) The suggestion unit, When suggesting dish descriptions, the description algorithm is optimized by referencing past description data. 2. The system of claim 1, (Appendix 27) The suggestion unit, When suggesting dish descriptions, apply different description algorithms depending on the category of the dish 2. The system of claim 1, (Appendix 28) The suggestion unit, Infer user sentiment and adjust the length of explanations based on the estimated user sentiment 2. The system of claim 1, (Appendix 29) The suggestion unit, When suggesting dish descriptions, prioritize descriptions based on when the dish was submitted 2. The system of claim 1, (Appendix 30) The suggestion unit, When proposing a dish description, the accuracy of the description is improved by referring to related literature on the dish. 2. The system of claim 1, [Explanation of symbols]

[0183] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. There is a reception desk where you can upload photos you want to include on the menu. an identification unit that analyzes the photo uploaded by the reception unit and identifies a type or feature of a dish; a layout unit that determines an order or layout of the menu based on the information specified by the determination unit; a suggestion unit that suggests a description of a dish based on the menu table created by the layout unit; A system characterized by:

2. The identification unit is Identifying the type and characteristics of food using image analysis technology 2. The system of claim 1 .

3. The layout unit: Decide on the order or layout of your menu based on food type, popularity, and / or seasonality 2. The system of claim 1 .

4. The suggestion unit, Suggests dish descriptions based on pre-registered databases and information collected from the internet 2. The system of claim 1 .

5. The reception unit is Allows users to upload photos of multiple dishes 2. The system of claim 1 .

6. The reception unit is Infer user emotions and adjust the timing of photo uploads based on the estimated user emotions 2. The system of claim 1 .

7. The reception unit is Analyze the user's past upload history and select the appropriate upload method 2. The system of claim 1 .

Images